Galileo AI's groundbreaking prompt-to-UI tool ✨

Plus: Co-founder Arnaud’s insights on building AI for design...

February 23, 2024

CV Deep Dive

Today, we’re talking to Arnaud Benard, co-founder of Galileo.

Galileo is a prompt-to-UI generative AI tool for designers and builders - or, as Arnaud describes it, ChatGPT for interface design. Founded in late 2022 within the highly-selective South Park Commons, Galileo went viral in 2023 with demos showcasing its early text-to-UI capabilities. Arnaud and co-founder Helen Zhou launched their public beta two weeks ago, and have already experienced strong uptake from the design and founder communities.

Alongside their launch, Galileo also announced a $4.4 million seed round led by Khosla Ventures, as well as a host of other industry experts including Julie Zhuo (former VP of Design, Meta), David Hoang (VP of Marketing and Design, Replit), Howie Liu (CEO, AirTable), Dan Becker (Design Leadership, Google), Anisha Jain (VP of Design, Cruise), Matt Robinson (CEO, Nested) and Joel Hellermark (CEO, Sana).

In this conversation, Arnaud walks us through his and Helen’s vision for Galileo, building AI for designers, and their time in South Park Commons.

Let’s dive in ⚡️

Read time: 8 mins

Our Chat with Arnaud 💬

Arnaud - welcome to Cerebral Valley. Firstly, walk us through your time at Google Research and what led you to co-found Galileo in the first place?

In 2018, I joined Google Research in Mountain View to work on language models - which wasn’t an area of high interest at the time. My first task was to work on Google’s Android keyboard - where you would type and have an algorithm predict the next word. The model was very simple - it could only predict one word at a time. The year I joined - 2018 - was when the Transformer model (BERT) started taking off. 

I was lucky with my timing at Google. In my few first months, language models weren’t very advanced. But their capabilities dramatically improved over the three years I was there. At some point, I thought “Well, if we continue drawing the line from here to where things can go, and the improvements don’t stop, we're going to see the creation of entirely new applications”. That realization opened my eyes to the exciting opportunities for starting a company.

My co-founder, Helen, is a design expert with experience at Meta and leading a team of designers at Cruise. We decided to leverage our combined skills to pursue the opportunities emerging from AI breakthroughs. This collaboration led to the inception of Galileo.

How would you describe Galileo to somebody who's never heard of the product?

At its core, Galileo is a text-to-UI tool. Users can describe the UI they want in a simple text prompt, and Galileo creates it for them. It's like using ChatGPT, but focused on generating interface designs. 

For example, if you need a dashboard design to track your nutrition intake, you simply describe it, and Galileo generates a few UI options for desktop or mobile in under a minute, which you can then export to a design tool like Figma. Additionally, Galileo offers a feature where you can upload a wireframe or screenshot, and it will generate the interface based on that image input. Compared to traditional design tools that require manually drawing buttons and forms, we want to make the design process easy, fast, and magical through Galileo.

Who are your users today? Who’s finding the most value in using Galileo?

From our private beta, Galileo’s user base falls into 3 distinct groups.

First, designers looking to speed up their process find Galileo very handy. Instead of spending hours drawing a mockup, they can create them in minutes with our tool.

Second, founders use Galileo to quickly prototype and gather customer feedback. They can achieve way faster product iteration than previously possible.

Third, software engineers who prefer to visualize the design before coding see a lot of benefits. For instance, an engineer working on a news app used Galileo to assemble all the screens before starting on the code.

I would say these three groups - designers, founders, and software engineers - are currently getting the most value from Galileo.

How has the product evolved since you founded Galileo in 2022?

When we started the company in late 2022, our biggest question was whether it was technically possible to generate good UIs with AI. Back then, the top model available was GPT-3-davinci, we spent a few weeks to get the initial concept working, but I can tell you that our first results were not impressive. 

Still, we dedicated months in R&D to strive for the best generation quality possible. Helen and I were working side-by-side and tried a lot of methods to get to the quality level we desired. In the end, it really paid off. Comparing the designs generated in January 2023 to January 2024 shows a magnitude of improvement in both quality and speed. Sometimes you need a bit of tenacity to get the best results.

In October 2023, we started rolling out a private beta, which quickly gained traction. People love using Galileo AI to visualize their ideas and iterate through concepts quickly. Interestingly, many users directly move to code after getting the designs from Galileo.

Overall, my biggest belief is that the user interface is the front door to the software creation process. What makes Google great is that it's the first place you go to search for information. Similarly, what makes Galileo great is that it's the first place you go for software building. At the start of a project, Galileo can significantly boost your productivity and broaden your exploration. What's more exciting is the potential of where we can go from here, as technology continues to evolve monthly.

How do you keep up with the pace of AI breakthroughs internally? Does it force you to adapt or change on a regular basis?

Our core principle is not to be dogmatic about what you build. I think this is crucial - resisting the urge to cling to something because of the time invested in it. You need to build in a way that allows you to shift to new models really fast. During my years at Google, part of my job involved staying ahead of technological trends by reviewing new papers weekly. Over time, I’ve built the muscles to be able to discern a true technology shift from a one-off demo that works well, which has been invaluable in building Galileo.

That said, I think every founder in AI, including us, needs to be adaptable to change. For example, the recent demo of Groq could output 500 tokens per second. If we told somebody that last year, that would have sounded ludicrous. It’s important to apply that same logic to the way we build products - meaning, what if you continue projecting to the point where we can get 5,000 tokens a second? That's the type of progress that all founders need to think about and map their roadmap with those possible changes.

Are there any interesting stories from your early users that have surprised or delighted you since Galileo’s launch?

The most rewarding feedback is from our users who told us “Hey, I tried five or six other UI generation products and Galileo’s output quality is the best.” This makes us incredibly happy, considering the amount of effort and thought we've put into every aspect of our product. I'd say that our technological investments have truly paid off. 

In terms of surprising use cases, I’d say we’re often amazed to see users combine Galileo with development tools that we never anticipated. We’ve seen people shipping iOS apps with designs entirely generated by Galileo - which is incredible. 

What makes Galileo stand out as the creative tool that every designer and builder would want to use?

First, our team spends the majority of our time and investment on quality. When we started in 2022, we looked at the future and asked ourselves “What would success look like?” The answer was always clear - to generate the highest quality. If you look at the realm of text-to-image, people naturally lean towards the highest quality outputs. This belief guided our feature development and priorities.

Second, we think Galileo significantly benefited from the blend of AI and design expertise within our founding team. My co-founder, Helen, has almost a decade of experience in design and has worked closely with me to ensure our model considers crucial design principles such as layout, UI elements, color, and font. I also made sure she was equipped with the knowledge of model training, fine-tuning models, and conducting evaluations. This partnership between AI and design is essential for getting the best outcome.

What have some of the main technical challenges been with building Galileo?

The hardest part on a technical level is that we need to invent the field from scratch. There are no existing benchmarks, papers, or models for generating UIs that are both functional and visually pleasing. Out-of-the-box language models don’t understand design nuances and can only generate simple layouts like login pages. I also read most UI generation academic papers and their outcome didn’t meet our quality bar. It quickly became clear that we had to create our own infrastructure, models, and datasets to get the results we wanted. It took a lot of engineering effort, but it paid dividends.

This period of R&D really taught us that we needed to invest a lot of time in high-quality research to stand out in the market. Today, we still try to take big technological bets as a company. I’m not sure I would recommend this for every startup, but it worked out for us.

As an AI startup, how do you balance research versus productization?

Ultimately, our main focus as a company is on outcomes and delivering value to our customers. Even if a research idea sounds very attractive mathematically, it might not be the right thing to attack as a team. We need to evaluate the potential rationally - research ideas are like investments, where you have a timeline of outcomes to deliver and you need to stack-rank them. There might be a paper from a team at Google or Facebook that did something appealing, but they also have like 20 engineers and a lot more computing. It might not be realistic for a startup to chase the same goals given the resources.

How is Galileo thinking about multi-modal in 2024, especially with the release of SoRA last week? And are you seeing more traction with text-to-UI or image-to-UI?

We were so impressed with the quality of models like SoRA. A few years ago, that type of model could only generate short blurry videos. If you look at models like GPT-4V, the AI is capable of understanding what’s in an image. It’s super valuable for us because interface design is a visual medium and having that layer of understanding can improve our product.

We already invested in a multimodal model with our image-to-UI feature. You can upload a wireframe/screenshot with a prompt and the AI will take care of the rest. Our users find it helpful for cases where it’s easier to express ideas through an image instead of words.

In terms of the difference, I’d say text-to-UI is very popular. I believe that a flaw with a lot of AI products today is that they expect everyone to be a prompt engineer, which is a major challenge for the user as it's not always obvious what needs to be typed into the prompt. This is why we leaned towards using a chat interface - because behind the scenes, we rewrite the users’ prompts in a way that can be easily understood by our AI.

So, if you type “I want an app for my dog”, the AI will say, “That's very high level. Let me ask you a clarifying question. Do you mean a dog walking app? And do you want an explore screen or do you want a setting screen?” That translation layer is what people like a lot, versus if you just had an empty box. We found that designers are amazing at prompting because they know UI elements extremely well, but if it's just you and I trying to design an app, having that translation layer is super helpful. This is especially true as people can be too high-level when describing what they’re looking for. That's where Galileo can help.

There’s obvious fear from designers about being replaced by AI. Tell us how you’re building Galileo to empower designers, instead of replacing them?

My co-founder Helen shared her take on this very topic in a recent interview. Our joint perspective is that designers spend a lot of time doing repetitive and mundane tasks within their design tool, and AI can help accelerate these tasks - which enables designers to spend more time on product thinking and strategy. These are the areas where humans can shine. 

A huge strength of designers is that they have context on both the business and the user, and can prioritize effectively what features need to be built - which is something that AI is unlikely to ever be able to match. So, if AI can accelerate the velocity of designers, they will have more time to focus on higher-impact tasks, and it will be a win for everyone.

Tell us about the team culture at Galileo. What do you look for in prospective team members, and are you hiring?

We work in an area that is evolving fast. The two most important ethos we value are 1) moving fast and 2) being outcome-driven.

I believe that having strong guardrails and engineering rigor is essential for moving fast. For example, I am very proud of our team's preparations for our public launch that we started preparing for scalability and stability weeks ahead. The traffic exploded on launch day, and was manageable thanks to our proactive setup, allowing the team to monitor systems and have plans ready for potential issues. It was an exciting day for everyone! It’s unrealistic to move fast without having a strong engineering culture.

Being outcome-driven means making sure every model update improves our generation quality. We've built a rigorous evaluation process for model updates, drawing on practices I've learned at Google. If the change doesn’t meet the bar, we’ll simply not ship it. This ensures our users consistently receive improved experiences. 

Our team is small, just four people, which I think is a key factor in staying competitive and nimble. It’s a culture built for those who like deep technical challenges and rapid development. If this interests you, reach out to me at [email protected]

How has South Park Commons played a role in the early story of Galileo, especially given the density of AI talent at the accelerator?

What I appreciate most about South Park Commons is their strong support for our vision. We had a rough prototype for our text-to-UI technology in late 2022, which, admittedly, was quite clunky. When we demonstrated it to all the SPC partners, their reaction was very positive, with comments like, “I've never seen anything like this before. You both should definitely pursue this.” Such feedback was quite encouraging, especially since ideas can be incredibly vulnerable in their early stages. 

SPC also has a lot of members with strong AI backgrounds - so it was really convenient to engage in deep conversations with subject matter experts over a casual coffee. There were times when the person would tell us “I’m the one that built that algorithm you’re talking about!”. That network of AI experts in one place has been very beneficial!

Conclusion

To stay up to date on the latest with Galileo, follow them on X(@Galileo_AI), join their Discord community and start creating at https://www.usegalileo.ai.

Read our past few Deep Dives below:

If you would like us to ‘Deep Dive’ a founder, team or product launch, please reply to this email ([email protected]) or DM us on Twitter or LinkedIn.