Julius is transforming computation with AI 📈

Join Slack • All Events • Jobs

Deep Dive

This week, we’re featuring Rahul Sonwalkar, the CEO and Founder of Julius AI.

Julius is a startup on a mission to transform computation using AI. It was launched as ‘Code Interpreter on steroids’, and quickly gained attention for being used by finance professionals and data analysts to crunch numbers. There are 300k+ users who have used the tool since its launch in mid-2023.

Rahul is also a key member of AI Tinkerers, one of SF’s premiere AI research events that brings together AI engineers, founders and researchers. Let’s dive in!

Our Chat with Rahul 💬

Rahul! Welcome to CV AI. Tell us about yourself and give us a bit of background on Julius.

Hey! I'm Rahul Sonwalkar, and I'm the founder and CEO of Julius AI. 

Julius is the world's first computational AI. We're focused on giving LLMs the ability to do computations, crunch numbers and be strong at doing math. We launched 26 weeks ago and have grown to 300k total users who are using Julius for use-cases like analyzing datasets, training small models, and doing heavy computation that they couldn’t with ChatGPT or other AI tools today.

About our team - we’re four hackers, and we believe that AI-powered computation is fundamental to us getting to AGI. Before Julius, we built and launched a number of smaller AI projects. We’re a fast-moving unit - most things that we ship, we build within a week.

We’re also constantly learning from our users - for example, one thing users currently want to be able to do is to modify the code that Julius generates for them. So that's our number one focus today. At our core, we care deeply about talking to users, solving their problems, and working towards our long term mission. 

The “AI Data Analyst” is originally how you positioned Julius. How has that vision evolved in the last 6 months or so?

Data analysis is where we started, and that's still where users find us extremely useful. When OpenAI came out with Code Interpreter, our team found it amazing but also realized it had a lot of limitations. That’s why we started Julius - to provide a better and more performant alternative.

Our initial focus was on nailing the UX around Code Interpreter - mainly, letting users upload a dataset and having the AI freely analyze it, generate visualizations, and write and run the code to complete the task. A couple of months in, we saw that the opportunities around an AI freely writing and running code could go far beyond simple data analysis. 

Who are your users? Who’s benefitting most from incorporating Julius into their work or lives?

The number one thing users use us for is data science computations, and a lot of that computation is powered by code. We know that a major pain point in AI computation is hallucination - and the way to keep these models grounded in reality is to ask them to strictly output code. This approach forces them to walk through the logical steps for how they arrived at an answer, and then run that code to reproduce it. Seriously, it’s hard to lie in code, and that’s what’s really powerful! 

So, data analysis is what a lot of users use us for, and that remains a core focus. However, our meta goal is to get much better at the code execution and computation that comes with it. That goes for data analysis, data science, and scientific calculations in general, too. 

And is there a genuine difference between how developers and non-technical users use Julius?

Stats-wise, under 5% of our users are devs - we’re mostly used by scientists, finance and grad students, two-thirds of whom don’t know how to write code! And that's the beauty of Julius - we’re founded on the premise that the cost of building software is trending to zero. Everyone is going to have this really powerful AI that can just write custom software for you on the fly.

An example we think about a lot: you’ll soon be able to pull out your phone and ask the AI “can you pull the data from here, run this analysis on it and give me the results?”, and the AI will look at the data, pull it, write a script to do that for you ad hoc, and give you the results you want. And that’s just the start.

So, since we know that the cost of creating software is going to 0 as we approach AGI, our question is: how do we democratize it? This is what we're focused on most. Amazingly, I would say 60 or 70% of our users have never written code in their life. 

What has surprised you most about the way users are using Julius?

This is wild. So, we originally launched Julius to interact with databases, since we thought everyone has a database they can connect to the AI and ask questions about. But it turns out that people don't have databases as much as they have spreadsheets. There are way more users who export data from a Google Sheet or SaaS software - or even in research, where the dataset is a CSV or JSON file. 

So, our market and the number of users who use us for spreadsheets quickly overpowered databases - from the start, it was 95% spreadsheets and 5% databases. That was a pretty huge realization for us - so right before the launch, we added spreadsheets in there, too!

You’ve definitely been asked this a lot, but can you explain how Julius is a definite improvement over OpenAI’s Code Interpreter?

Number 1, Julius throws fewer errors. Code Interpreter will often output broken code, and then goes into an error loop where it has to figure out what to do next. Julius works smoothly - and that actually requires a lot of work, because you have to understand the common pitfalls that the AI goes into and solve them preemptively. 

So, how do you intelligently solve those errors? You pull a bunch of different models and say, “here's the error I have”. You collect a ton of data on the different errors people run into. And you become better at fixing certain categories of them. There are 7-8 categories that are super popular, which we’ve studied extensively - asking why does the AI fail here, and how can we make it better for this purpose? So number one is Julius has better error self correction. 

Number 2, Julius is a better experience. A ton of people have posted about Julius being smoother and more intuitive, and this is because we’ve focussed primarily on the moment when users come to Julius and expect the AI to write correct code. 

For example: a user solving a math problem on Code Interpreter might upload an image of it, to which ChatGPT hallucinates a solution for them. With Julius, the AI knows it has access to a code execution environment (a virtual machine), so even if the input is an image, it will parse it as a math problem that gets solved by installing the right Python modules and running that code. So, we are hyper-focused on these use cases, which allows us to build a much better experience. 

In terms of the code that Julius generates, how do you ensure it’s correct? What’s the methodology around unit testing or executing different iterations?

So we run the code straight up, immediately. As soon as a user finishes writing code, we run it. And if the code has errors, then it will inevitably break. I've talked to a lot of founders who've talked about RAG, and one of the challenges there is that depending on the snippets you pull out of a document, the AI could potentially still hallucinate. But if the code is broken, you know in the coding system that it's simply broken - with no hallucinations.

This is also where self-correction comes in - which is one of the things where we’ve extensively focused our energy. It's not about just hooking up a code execution environment to an AI, but obsessively looking at where it breaks and fixing all of those things. And GPT-4 is of course the best model out there for code execution, but it has a lot of gaps. Our focus has been: how do you use smaller experts to cover those blind spots? 

What’s the hardest technical challenge involved in building Julius?

I would say: building the infrastructure to be able to spin up these virtual sandboxes. We recently crossed a billion seconds of sandbox runtime, which is super powerful, but also challenging to maintain. When a user logs in to Julius, they get a VM, and it stays for an hour after usage before it auto-destructs - keeping a user’s data super secure since each user has their separate VM. Getting this to run smoothly and doing the provisioning of these VMs at scale can be super challenging. 

And here’s a story for you: on December 13, we went viral in India, and we had thousands of users log in during a short space of time. At the time, our system would provision these VMs at the moment you logged in. On that day, we literally ran out of VMs because so many of the requests to our backend were timing out, and they piled up so fast that it took our whole backend out. Julius was down for 12 hours…

This was super frustrating, because even though it was cool that we went viral, we quickly ran out of VMs, and we had to basically rewrite a bunch of our infrastructure to be able to handle such scale. We're going to continue running into some of those issues, but I think these are good problems to have. So infrastructure is definitely a big technical challenge. 

The second challenge is building the internal systems to be able to improve our error self-correction. This is a big part of keeping things smooth - deeply understanding when code written by the AI breaks, understanding why it broke and the ways in which to not let it happen again. And sometimes, the answer is not even like, hey, let's have a fallback. The answer can often be, can we do something better UX-wise? But just having those checks in place is really useful. 

What are your goals for Julius in 2024? What are you hoping to achieve this year?

I want to get to a million users and 10 billion seconds of code execution runtime - and that’s the floor. The goal would be everyone in the world has a powerful AI code monkey - Julius - in their pocket. And what’s fascinating is that this is the worst that these models are ever going to be, so in a year from now, things could be very different. A year ago, GPT-4 wasn't even GA. So I'm super bullish about the next twelve months.

Lastly, tell us about your involvement in AI Tinkerers, which has become the #1 AI event in San Francisco.

Alex Graveley - inventor of GitHub CoPilot - posted a Tweet about AI Tinkerers in Feb 2023. And one thing to know is that I and our entire team are all hackers and tinkerers. Our plans are never set in stone - we just have goals that we chase and we’re always hacking on stuff internally.

So I reached out to Alex and said “I’d love to help you out”. And he was like, “let’s do it”. The first AI Tinkerers was in Austin, and the second was in SF at the start of 2023. And what’s crazy is more than three times as many people signed up as the venue allowed for safety reasons. And it was crazy, like really cool projects. 

Plus, what was interesting about that event was that it wasn't like a VC networking event. It was just like, whatever project you have, you can come show it to everyone. And people showed up with half-baked projects, some really ambitious projects. All of them were so cool. 

One of my favorite things about AI Tinkerers is when someone’s demoing something and it breaks. I love that, because it's not like you're showing slides or a recorded demo - it's like a real demo of a thing that you're actually trying to build. And to me, it's rare to find events like that. 

Final Note…

That’s a wrap for our second Deep Dive of 2024. Follow Rahul on Twitter to learn more about his work, and check out AI Tinkerers for upcoming events.

Sponsors: If you would like us to ‘Deep Dive’ a founder, team or product launch, please get in touch at [email protected] or DM us on Twitter.

Join Slack | All Events | Jobs | Home