Hack it to me: Pushing the AI envelope to produce new products at breakneck speed

Ever heard of a hackathon? It’s essentially an intensive period of strategizing and coding, with the aim of creating a new digital product. It typically involves working around the clock in shifts.

We were itching to organize another one, as it had been some time, so we recruited 18 participants across all three of our international offices in Montreal, Nantes, and London to showcase their skills inside of this unique container. We assembled four teams of colleagues who were excited to be part of this high-octane experience (and who were willing to sleep at the office).

The focus of this hackathon was to explore the uses of generative AI and its potential to eliminate inefficiencies in day-to-day life. Our company already has a fair bit of experience in this area, but because we’re constantly striving to gain more skills in the areas that matter most, we used the event as an opportunity to evaluate and test new tools and frameworks in the space, approach problems from fresh perspectives, and gain even more knowledge that can be applied to future projects.

In this blog post, we’ll share the four teams' approaches, challenges faced, and outcomes.

24 hours to experiment with generative AI

We’re only beginning to scratch the surface of AI’s potential to solve everyday problems, improve efficiencies, foster innovation, increase accessibility, and enhance accuracy. But one thing is clear. AI is kind of a big deal, and it’s here to stay.

To get everyone at Osedea up to speed on the latest advancements in AI technology, including LLMs, ChatGPT, and potential use cases, we held a workshop with AI expert, Nicholas Nadeau ahead of our hackathon to help equip us for success. We then invited each team to choose their preferred technologies to work with—the only rule being that they had to make use of AI tools.

Here’s how it went ...

Team 1: OpenPizza

The OpenPizza team aimed to solve the problem of labour shortages currently faced by restaurants (resulting in congested drive-throughs and long waits when placing take-out orders by phone) by automating tasks with AI. They developed an app which uses speech-to-text, ChatGPT, and text-to-speech technology to create a hassle-free ordering and drive-through system for food.

The team used a combination of OpenAI's Whisper and GPT-3.5, and worked with various text-to-speech solutions throughout the night, such as the ElevenLabs API and the Coqui TTS model repository. Although they only had time to complete work on the pizza ordering cycle, they had plans to implement AI long-term memory with Pinecone, voice recognition, and face tracking to identify customers and cater to their preferences. Examples of this included facilitating recurring orders, and suggesting menu items based on user preferences.

To create the system, the OpenPizza team used various technologies to synthesize GPT3's responses. However, they faced some challenges in guiding ChatGPT to respond in a consistent parsable format, as well as with a lag in response time, which made for confusing dialogue. The team also found that running trained models locally sometimes led to strange audio results.

Despite the hiccups they ran into, the OpenPizza team was able to successfully automate phone and drive-through orders with their AI-based ordering/drive-through system.

Team 2: WasteNot

The WasteNot team aimed to address the problem of food waste using AI. They used predictive analytics, image analysis, and computer vision to create personalized meal planning, financial reporting, and inventory management to tackle the issue.

The team turned to ChatGPT to identify keywords and names for logo generation during the ideation process, while also leveraging Miro and Figma. Their approach involved identifying the items in a user's grocery cart and on their receipts from past shopping trips to provide personalized suggestions for reducing food waste. They also integrated OpenAI and other AI libraries to generate solutions in real-time, so that users would be empowered to make more informed decisions about their grocery purchases throughout the week.

While building in some user flows with data extraction from receipts, WasteNot faced some challenges with the Azure API. Object detection was tricky, as the team obtained too many abstract labels compared to the objects they were trying to detect. They tried to re-inject the output of the Google Vision model into ChatGPT to get a filtered list of items, but this method wasn’t very effective.

In the end, due to the 24-hour time constraint, the team's MVP (Minimum Viable Product) was limited to one feature. That being said, while making grocery shopping more efficient was the team's ultimate goal, they learned a lot about how AI can play a crucial role in reducing food waste and promoting sustainability by optimizing grocery inventory, identifying optimal storage conditions and expiration dates, facilitating food donations to charities, identifying optimal composting methods, promoting biodiversity, and conserving water resources used in food production.

Team 3: PerfectionistAI

If you’ve ever tried to upgrade the look of a room in your home, you know how much trial and error can be involved. The PerfectionistAI team aimed to solve the problem of time-consuming room redecorating by building an AI interior design helper. The idea was to give users the ability to take a photo or video of a room in their home and then give voice commands to add paintings to walls, change furniture into other furniture, add rugs to the floor, and add dogs to couches (because why not?). The end goal would be to help prevent repeated trips to the store to return items that didn’t look or function as expected.

The team used the hackathon to develop an app that leveraged OpenAI's Whisper to convert speech to text. The text output from Whisper was then fed into StableDiffusion via a prompt, which would trigger the identification of a mask. For example, if the prompt was to remove a white lamp from the table, a mask would be created to isolate the area around the lamp. The generative modelling capabilities would then make the requested changes to the region where the lamp was, without affecting the rest of the image. The team noted that OpenAI had difficulty with complex masks, but generally produced good results when the mask was large. It was also a challenge to engineer prompts to perform complex. For example, to replace an object in an image, 15 steps are required in StableDiffusion.

Overall, the PerfectionistAI team was pleased with the easy access to models via APIs, affordable inference, access to cutting-edge technology and academic research, rapid open-source contribution, and powerful language models. However, they also faced several challenges, including the complexity of AI technologies and concepts, difficulties in estimating AI difficulties, limitations in AI hardware and infrastructure, non-deterministic behaviour of ChatGPT, and issues with dependency management.

Team 4: Titre en Tête Team

The Titre en Tête team created an addictive movie guessing game app to provide a fun and engaging way for movie enthusiasts to test their cinematic knowledge and skills. The game involves playing against ChatGPT and requires players to give precise clues (without saying too much).

The team used Bubble, Chalice, The Movie Database API, and ChatGPT to make their vision a reality in just 24 hours. The biggest hurdle for them was to build an app with 50% non-dev team members, with some team members needing to work on other projects simultaneously. ChatGPT also produced some random, unexpected results, but overall the stack was still a good choice for immediate deployment and testing with others. The team faced a learning curve with Bubble, but they got more comfortable with it in the final three hours of the hackathon.

In conclusion, the Titre en Tête successfully built their movie guessing app, learning valuable lessons about teamwork, time management, and technology. They demonstrated the benefits of low-code development and API integration while creating a fun and engaging product. Future improvements to the app might include refining the AI's movie guess accuracy, adding more game modes, and incorporating user feedback. They felt that building a small gamification product proved to be useful practice working with AI integrations to improve their expertise.

Key takeaways

At the end of the hackathon, each team presented their projects to the rest of the company. Despite the limited time frame they had to work with, each team managed to produce something innovative and exciting using the latest advancements in AI technology.

Here are some of the key takeaways:

1. AI can be used to solve everyday problems - From reducing food waste to automating drive-through orders, the teams showed tangible ways that AI can be used to tackle inefficiencies and improve our lives.
2. Collaboration is key - Each team had a mix of designers and developers taking part, and by working together and leveraging each other's skills and expertise, they were able to create something that was greater than the sum of its parts.
3. AI is still evolving - While AI has come a long way in recent years, there are still limitations to the technology that need to be addressed. The teams discovered this firsthand, as they faced challenges with things like speech recognition and image analysis.
4. The potential of AI is limitless - Despite the challenges, the teams were able to create some truly innovative and exciting projects and it’s exciting to think about what the future holds for generative AI.

Join us in exploring AI’s potential

The hackathon was a great success! We’re so proud of what our teams were able to accomplish in just one day—just imagine what they can achieve in two weeks or one month! We believe that AI has the potential to change the world for the better, and we’re committed to exploring its uses and pushing the boundaries of what's possible in this exciting domain.

If you have an AI product idea, or if you’re interested in learning more about how AI can help solve problems in your industry, we invite you to work with us! Contact us to book a 1:1 lunch & learn with us today and let's talk about the best tools and frameworks and frameworks for your specific needs, and how we can help you achieve your goals.