• Cerebral Valley
  • Posts
  • Refuel is rethinking data-management in the age of AI 🌐

Refuel is rethinking data-management in the age of AI 🌐

Plus: Rishabh & Nihit on Refuel LLM, data enrichment and AI research...

March 27, 2024

CV Deep Dive

Today, we’re talking with Rishabh Bhargava and Nihit Desai, Co-Founders of Refuel.

Refuel is a platform that helps enterprise ML teams clean, label and enrich data at scale. The company’s mission is to use AI to power data workflows such as data labeling, cleaning, enrichment and extraction, so that ML and product teams across startups and enterprises can build data-driven products and train their own AI models with confidence. Founded in 2021, Refuel is built upon the idea that data is the true lifeblood of the current AI revolution, and that without proper data management processes in place, AI will not reach its full potential in the enterprise stack. 

Today, Refuel has hundreds of teams using its platform for data cleaning, enrichment and extraction, including at companies like Enigma, Spot AI and TeachFX. In 2023, the young startup announced a $5m Seed round led by General Catalyst and XYZ Ventures, in order to expand its team and work towards a public launch.

In this conversation, Rishabh and Nihit walk us through the founding premise of Refuel, why AI data is the lifeblood of today’s AI products, and Refuel’s goals for the next 12 months.

Let’s dive in ⚡️

Read time: 10 mins

Our Chat with Rishabh and Nihit 💬

Rishabh and Nihit - welcome to Cerebral Valley! Firstly, tell us about your background and what led you to co-found Refuel?

Hey there! I’m Rishabh, one of the co-founders of Refuel. My co-founder Nihit and I both come from machine learning and data systems backgrounds, and we met during grad school at Stanford. We’ve known each other for about a decade now, and founded Refuel just over two years ago.

Before Refuel, I was an early ML engineer at a company called Primer.AI in 2017, where we were building a tool to ingest the world's unstructured text information - things like news, social media, SEC filings - and give people an interface where they could just type a question and get a full-fledged report to read. So, very much ahead of our time back then - this now takes a few lines of code! I then had a stint at a company building data pipelines at the terabytes scale, and that company got acquired by Cloudera a couple of years ago.

My co-founder, Nihit, was at Meta working on trust and safety and content moderation problems prior to Refuel. He was focused on building scalable techniques for understanding large swaths of multimodal data, and then scaling it out to Facebook scale in near real time. He left Facebook at the end of 2021, and we’ve both been building Refuel ever since. 

We started Refuel to accelerate the era of AI abundance by solving the core bottleneck – data. No company has ever told us that they’ve got plenty of data that’s clean, structured and ready to use. Today, most businesses are generating and ingesting tons of data that remains unused simply because it's unstructured and hard to deal with. At the same time, there is a realization that data is the lifeblood of modern products - we're generating and collecting data at exponential scale, but our ability to understand it and make it usable is bottlenecked by human effort. 

It doesn’t have to be this way - our goal with Refuel is to unlock the value of AI for every business.

For an ML engineer or team who hasn’t heard of Refuel before, how would you describe what Refuel does? 

Refuel is a platform that helps teams clean, label and enrich data at scale by leveraging LLMs, while delivering superhuman (and better than GPT-4) accuracy.

If you’ve ever been in a room where someone has said - “We’d love to start this project, but we don't have engineering resources to get our data in order for the next two quarters” or, “if we’re to go out and clean and label our data, we'd have to hire this massive team of contractors, and who's going to manage that?” - then Refuel is meant for you. Data is a massive chore for most teams and if the problem of getting to clean, structured data doesn't get solved, then a lot of the promise of AI will simply never be realized. 

Working with Refuel is a simple three-step process. First, point Refuel at where your data sits and describe your data operation in natural language. Refuel’s models will run on your data to produce the desired outputs, while flagging any low-confidence results for review. Second, deploy this task to run in batch or real-time mode (at any latency), by training and deploying custom models fine-tuned for your tasks. Finally, set up your task for continual improvement with further feedback and retraining built-in. 

All in all, Refuel is a complete solution for teams to solve data cleaning, data extraction and data labeling problems at scale.

Who are your users today? Who’s finding the most value in using Refuel?

Our customers are startups and enterprises across several different verticals - such as financial services, marketplaces and e-commerce, where data quality, classification, extraction and matching problems run rampant. Our users include data scientists who want to create labeled datasets quickly, developers who are building highly accurate LLM applications, and even operations teams looking to automate time-consuming workflows. 

As a concrete example: imagine you're a marketplace and you’re ingesting products from different vendors, improving their quality/descriptions, and then making these products available on your platform for end consumers. A common pain point you'll run into is standardizing this data into your schema and then improving the quality to make the product look appealing to consumers. So, you’ll either spend a bunch of human time reviewing and editing data, or you'll have to hire a large machine learning team for training and improving many different ML models systems.

Instead, you can leverage Refuel by pointing us at the raw datasets being ingested, and describing how you want your data to be normalized and enriched. You can write down simple instructions or use one of our templates - for example, an enrichment template that we have - and then Refuel will help you pick the best LLMs for your task that you can further customize with simple thumbs-up and thumbs-down feedback.

The value for teams is a simple, consistent workflow for leveraging, fine-tuning and deploying high quality models and applications to production – typically outperforming GPT-4 and human quality, while scaling to any data volumes and with minimal effort. 

It feels like you’re taking a full-stack approach to the data enrichment problem. Would you say that’s a unique differentiation point between yourselves and competitors? 

This is a huge part of why we're building Refuel the way we're building it – as an end-to-end platform. Many of our users are people who are not machine-learning experts - these are folks who are experts in their domain and know how they want the problem solved, but don't have the ML expertise themselves. There are all sorts of questions that come up when leveraging LLMs to solve data problems – “What does a good prompt look like”, “Which LLMs are good for my task and how do I evaluate them?”, “When should I use few-shot prompting and which examples should I use?” “How do I prepare datasets for training?”, and more. But, as long as you can point Refuel at your data, and you know what the correct output should look like, you can get tons of leverage from Refuel. 

Essentially, Refuel ends up becoming a way for users to turn their internal expertise into an operationalized process that produces LLMs that are going to work well in production. It’s highly valuable having all of this in one place, such that the product can suggest how to structure your prompts and improve them behind the scenes, all the way to helping you assess the quality of models/data for fine-tuning and continually improving in production. Refuel ends up being one consolidated place where a lot of this can happen.

Additionally, we also have our own LLM that we've instruction-tuned on close to 3,000 data sets and many billions of tokens across multiple verticals, which is purpose-built for what we think of as data applications - classification, enrichment, extraction and more. What we’ve found with customers is that performance out of the gate is oftentimes at parity or above the most powerful LLMs like GPT-4. More importantly, it's very data-efficient to fine-tune it for specific customer applications - with 500 examples and less than 30 minutes of training, you can get to something that beats a fine-tuned GPT 3.5 or GPT-4.

Walk us through the customer experience of using Refuel - are you seeing any use-cases that are extra-compelling? 

We can illustrate this through one of our customers, where their internal revenue operations team are our customers. Oftentimes, this team will want higher quality leads and to hyper-personalize their messaging when talking to a potential customer, and so they’ll leverage Refuel for enriching potential leads - information like the company’s size, location of headquarters, industry and so on. One entertaining story is that they’ll ask Refuel to generate a list of fun history facts about their customer’s location before they go and talk to them - which is a use case we didn’t anticipate when we started working with them. Once they’ve set this up within Refuel, it runs automatically for every new data lead that enters their pipeline. 

Another trend we're seeing is that a lot of customers are interested in Refuel because they're already thinking about fine-tuning and customizing models for specific use-cases. Natural questions that come to their mind are “How far can we take this? Can we customize models on a per industry level, and customize LLMs on a per-customer basis?” This is one of the trends we’re finding super interesting because, to be able to do this successfully, you need very fine-grained control over your data and the set of models that you're training, and have the relevant infrastructure to be able to pull this off. If done well, the level of quality that you can achieve for any given customer is really high.

What’s the biggest technical challenge associated with building Refuel? 

There are multiple, but the foremost is around LLM output quality and reliability. LLMs are fundamentally a very new piece of technology, especially in terms of leveraging them at scale and applying them to solve real-world problems. Even though the research has been around for five or six years, the community collectively has twelve months of experience, and the bar for reliability and quality is fairly high for most customers that we work with across critical business use cases. Specifically, the challenge is getting high-quality outputs that you can then rely on as a business.

To give you a concrete example, one of our customers has a lot of domain-specific data for which they've built a large list of rules and heuristics over a decade. They've had a large team of expert humans building and managing this list, which is complex as a single change may break a bunch of things elsewhere. Now, this customer is using Refuel to replace a lot of these rules with a single LLM that is exceptional at that one task - and we're seeing success there. But, being able to deliver that experience for all types of business problems that exist - across data classification, extraction and more - and making it turnkey, is one of the hardest parts of building Refuel.

How do you incorporate AI research into your product development process, given the pace of breakthroughs on a weekly basis? 

Firstly, Cerebral Valley has been a great source of information for keeping up with new AI developments - almost as a proxy for what’s top of mind and important for the community. It’s true that every week, a bunch of new ideas come up and we leverage Twitter/X to stay on top of the best research.

In terms of actually deciding what research to incorporate into our product development roadmap, we don’t have a very systematic process today – a lot of it is building intuition for how relevant a piece of research will be to the product that we're building and to our customer base. We try to have a sense of the impact of new research before going too deep into it, and our priorities are driven by customer conversations, or by our aggressive product roadmap. As an example, very early on we heard from users that estimating “confidence” of LLM outputs to better detect hallucinations was critical, and so we spent a number of cycles going deep into the research. 

What are you going to be most focussed on for the next 6-12 months? Customer growth? Infrastructure? ML research? 

All of the above. We’ve seen significant growth in terms of the number of customers using Refuel, and we expect to continue to grow that significantly. For example, we've gone from having processed a million data points across our customers a few months ago, to 12 billion data points today. That’s a 1,000x increase in the volume of data that we’ve processed - and so we’re seeing that scale happen in real time. 

Additionally, a lot of our effort over the next 12 months is going to be spent on improving our platform - making it more robust, capable and friendlier for teams to get to their desired outcomes. So, for us, the next 6-12 months are very much about growing the number of customers that we have, while making sure that our platform continues to become better, more robust, and able to handle more use cases.

Lastly, tell us about the team culture at Refuel. What do you look for in prospective team members, and are you hiring?.

Culture at Refuel can be framed along a few different axes. When hiring, we look for a demonstration of real-world impact, and the desire to create more. We’re also extremely customer-centric – one of the ways this shows up for our engineering team is that we’ll happily leverage everything that's been built already, because ultimately, what we care about most is delighting our customers. 

We also place a heavy premium on moving fast, given how small of a team we are and how fast the ecosystem is moving. We're okay with shipping things that might be less than perfect as long as we maintain this culture of learning, iterating, and shipping the next version out as soon as possible. Tied to that, we also really look for an ‘ownership mentality’ around the work - this is critical for us given our broad product surface area, and the number of important decisions we need to make on behalf of our customers every single day.

Conclusion

To stay up to date on the latest with Refuel, follow them on X(@refuelai) and learn more at Refuel.

Read our past few Deep Dives below:

If you would like us to ‘Deep Dive’ a founder, team or product launch, please reply to this email ([email protected]) or DM us on Twitter or LinkedIn.