Today, we're talking with Luke Kim, CEO & founder of Spice AI

Spice AI is giving every AI agent its own data stack. The Bellevue, Washington-based company builds an open-source, Rust-based data and AI engine that gives development teams a single SQL endpoint to federate, accelerate, and search data across data lakes, warehouses, and operational databases. It’s lightweight enough to run on a Raspberry Pi, and as of today, distributed enough to take on Spark-scale compute.

Founders Luke Kim and Phillip LeBlanc come from deep infrastructure roots. Luke was the co-creator of Azure Incubations in Microsoft’s Azure CTO office, where he helped create projects like Dapr. Phillip spent a decade building distributed systems and big data platforms at GitHub and Microsoft. They started Spice in 2021 on a conviction that AI would be infused into every application - and that data, not models, would be the bottleneck.

Spice now runs global production workloads at enterprises like Twilio and Barracuda - Barracuda reports 100x faster queries at 50% lower cost. And today the team launches Spice 2.0: distributed multi-node query built on Apache Ballista, the general availability of the Spice Cayenne acceleration engine built on the Vortex columnar format, real-time CDC replication for analytics on live operational data, and the enterprise controls - Cedar policy, RBAC/ABAC, governance - that come with growing enterprise adoption.

We sat down with Luke to talk about the journey from reinforcement learning to the agentic era, why the model was never the hard part, and what it takes to give 10,000 agents their own sandboxed data stacks.

Let’s dive in ⚡️

Read time: 8 mins

Luke - you and Phillip both come from deep infra backgrounds at Microsoft and GitHub. What problem convinced you to start Spice in 2021, and how did the vision evolve from there?

I spent the last couple of years of my career at Microsoft in the CTO’s office in Azure, where I built an incubator called Azure Incubations. We did projects across compute, distributed systems, open source, databases, blockchain, and of course AI - and being in the CTO’s office, we worked with OpenAI at the time. This was back in 2019 and 2020, before ChatGPT launched, but you could see the general direction of where everything was going. I really saw the opportunity that AI was going to be infused into everything going forward - which would prove true within four or five years - but also the challenge: if you’re not building AI-native today, you’re going to be left behind.

In addition, Phillip and I were working on another project around neurofeedback - where you take data from the brain over an EEG, process it, and give feedback back to the brain. You’re consuming very large amounts of data - gigabytes per minute - and you have a very small time window to make an intelligent decision. So from that microscale all the way up to Azure scale, I was seeing the same problem: it’s all about the data. How can you consume data across whatever use case or business you’re in, use AI to make intelligent decisions, and feed it back into the system so it can be self-improving?

At the time, most companies had only one or two data teams, and you’d go to the centralized team and ask them to help you with your AI project. But because AI would be infused into everything, every team would essentially need to be able to build these solutions. So the question became: how can we enable every team to build their own AI solutions on a foundation of data platforms they can deploy themselves, for each AI use case?

So the constraint you kept seeing was data - rather than the models themselves?

I’ve built across all the generations of software - desktop, client-server, cloud - and throughout all of them there’s an enduring pattern: garbage in, garbage out. You can have the best models you want, but if you’re giving them the wrong data, or they don’t have access to all the data, they can’t give you good insights and they can’t give you good decisions.

You describe Spice as giving every agent its own sandboxed data stack - what does that mean in practice?

It used to be that you went to the data team and said, “I want to build an AI application,” and they would set up a whole bunch of infrastructure for you. That infrastructure would include things like Spark, ETL pipelines, connectors back to the operational stores in your business, transformations, a serving layer, connections over to models. That sprawling infrastructure not only takes a long time to build - often months, sometimes years - it’s also very expensive just to move data around, and you need a whole team with expertise to operationalize it.

Now, instead of one or two AI applications at your company, every team has to build multiple - every feature of every product is being built with AI. So how do you empower each of those teams, even each agent type, to have its own data stack? With Spice, you take all of that infrastructure a centralized team would have deployed and you have it in a single binary - a single engine that’s deployable anywhere. You get petabyte-scale, high-performance query federation across 40 different backend sources - everything from SQL databases to streaming systems to Iceberg and Delta, even things like IMAP for email or CSV files on FTP servers - the entire data platform, contained in something you can run anywhere and deploy alongside any application.

As an example, we’re kicking off a project right now where the customer wants one physically isolated query engine for every agent - and they have 10,000 agents. In a Databricks or Snowflake world, you wouldn’t deploy 10,000 warehouses. But what you can do is deploy 10,000 Spice instances, each one with its own data stack that integrates with  Databricks and your operational stores.

Spice 1.0 went GA in January 2025 as an extremely lightweight single-node binary. What did that release prove, and where did it land first?

The premise was a very lightweight binary that could deploy anywhere - on anything from a Raspberry Pi or Jetson Nano all the way up to cloud scale. Super simple, anyone can run it, with all of those features included. And the real premise was that running AI next to your data would lead to far better outcomes. If your AI doesn’t have access to all the data it needs, it’s not going to be accurate. If the data is too stale, it won’t give you up-to-date information. We shipped in January 2025 with something like 15 data connectors.

Speed is especially important for AI, because the faster data access is, the more turns the agent gets to reach an intelligent answer - it can query many sources multiple times and get better answers. So we also included acceleration engines in 1.0, initially built on SQLite and DuckDB, that provided consistently fast data to agents and applications - and it works just as well for dashboards and data applications. Under the hood, Spice materializes a working set of data that’s sandboxed, physically isolated, secure, and intentionally provisioned. And because the acceleration is often deployed alongside your application, you get extremely low-latency,  localhost access to that data at single-digit milliseconds. 

[Editorial note: accelerators are pluggable - teams can choose DuckDB, SQLite, Arrow in-memory, or Spice’s own Cayenne engine depending on requirements. Optionality across federation, acceleration, and search - all built in open source - is a core part of the Spice story.]

The engine itself has a longer history, though - could you walk us through the journey to that 1.0?

When we first started in 2021, the vision was always to help people build applications that learn and adapt. We called them intelligent applications back then - of course, everyone calls them agents today. We started off in reinforcement learning, but what we found was that our biggest bottleneck was the data: how do you access, at that time, hundreds of terabytes of data very, very fast to feed into these systems and get intelligent answers out?

We didn’t want to build all of this ourselves initially, so we built on systems that already existed - Dremio, DuckDB, a number of others. What we quickly realized is that none of these systems were designed for AI. So at the end of 2023 we decided to build the system we always wanted to have - and the system we saw everyone else needing. The first line of code was December 2023, built from the ground up in Rust on amazing open source projects we contribute to, like Apache DataFusion and Arrow. 

We launched on Hacker News in March 2024 and were on the front page most of the day. Barracuda reached out to us from that post - we were working with them by April 2024 and had a production deployment a couple of months later.

Barracuda reports 100x faster queries at 50% lower cost; Twilio runs Spice in its control plane. What do those look like day to day?

Where Spice really excels is query over operational data - even analytical query over operational data, and that’s reflected in those use cases. In Barracuda’s case, we use our acceleration technology to provide that 100x speedup over very large amounts of data directly over their data lake to their end customer applications. And it’s similar for Twilio - a lot of operational data they wanted to accelerate and provide consistently fast to their applications.

That sets us up for 2.0. What did agentic workloads change - why did enterprise agents outgrow the single-node model?

The vision for Spice has always been to provide that full data platform across all your data - what you used to get from your data team, all in a single box. Naturally, you have to grow beyond single-node to do large-scale distributed compute - the things you would do in systems like Spark. The single-node engine was getting us to about the single-terabyte range, and we started doing projects with larger customers in the five-terabytes-and-beyond range. That’s when we decided to build out the rest of the vision: full multi-node distributed compute.

The second thing we found is that you really want both - very fast, low-latency query serving hot data back to your applications and agents, and the ability to reach back into your entire history for large-scale compute. Pairing our single-node engine with a distributed cluster gets you the best of both worlds.

Third, organizations wanted to do large-scale analytical query over their operational data without disrupting their operational stores - MySQL, Postgres, DynamoDB, MongoDB, and so forth. Instead of running big aggregate analytical queries on those stores, you can bolt on a Spice analytical node and run those queries without giving your agents direct access to your operational stores and interrupting your business.

Fourth, we started to hit the limits of our initial accelerators, including DuckDB, around that one-to-five-terabyte range. So we built an entirely new query acceleration engine ourselves - Spice Cayenne - on Vortex, a Linux Foundation project and a very high-performance file format similar to Parquet, with the ability to scale much higher, essentially into the petabyte range.

And finally, we started to get a lot more enterprise use. So in 2.0 we added all the enterprise features you would want: full Cedar policy, which lets you define policy and do row and column-level filtering with OIDC support; full RBAC and ABAC; and you can connect up to your catalog and get all of your governance and lineage. With 2.0 you now get that complete data platform you would have gone to your data team for - in a very easy-to-deploy box that any feature team can spin up themselves.

One of the bolder 2.0 claims is running analytics on live operational data - without Kafka or Debezium. How does that work, and why does it matter for agents?

What we really see in the AI era is demand for super-fresh data. If you’re asking your agent about something and it’s answering on 24-hour-old data, it’s not going to give you the most up-to-date answers, and at worst, it gives you wrong answers. We built Spice Cayenne specifically for this. We have native CDC replication from your operational stores - we’ll connect over to Postgres, to MongoDB Streams, to DynamoDB Streams, replicate data consistently from those systems at high throughput, and then enable high-performance analytical query over that data in an isolated, sandboxed way.

So now you get single-digit-second freshness, you can do all your analytical queries without interrupting your operational store, and you can feed that data to your agents. And agents, of course, are probabilistic - when they go and fetch data, they could ask all types of different questions. You don’t have to worry about an agent bringing down your operational store with a big, heavy query, because that isolated analytical data platform sits alongside your operational platform.

Spice is fully open source and written in Rust. Why have both stayed non-negotiable?

We build on amazing open source projects. We’re big contributors to Apache DataFusion, Arrow, Iceberg, Delta, and now the Vortex project - and we contributed a whole set of table providers back to DataFusion, which enables connecting to those 40 different sources. Our principle has always been that the single-node engine will never hold anything back: it has to be the fastest, highest-performance single-node query engine you can get to serve data to agents. You can download it onto your laptop today and start querying data out of S3 within literally two minutes - and all of that is free and open source, and we intend to keep it that way.

Once you get to enterprise scale - large Spark-level compute jobs, row and column-level filtering, governance, those types of things - we have a commercial offering called Spice Enterprise that includes those features.

Some readers might assume this puts you head-to-head with the Databricks and Snowflakes of the world - but you’re a Databricks partner. How does that work?

They’re a great partner - we have something like 10+ different integrations with Databricks services today. We connect over to SQL Warehouse and Spark Connect, we integrate with Mosaic AI, and we’ve collaborated together on the delta-kernel-rs library. We deploy really, really well alongside Databricks.

The differentiation is the ability to run many, many Spice instances in physically isolated ways - that’s something they don’t do. Like the project I mentioned earlier: you wouldn’t deploy 10,000 warehouses, but you can deploy 10,000 Spice instances. And because Spice is so lightweight, you can deploy it on Raspberry Pis, on Jetson Nanos, at the edge, in completely air-gapped environments - again, not something you would normally do with a Databricks or Snowflake of the world.

Beyond the headline features, what makes 2.0 exciting for the team?

2.0 really marks the Spice Cayenne engine maturing into a high-performance query engine that scales to terabytes and petabytes of data, but is also an extremely fast consumer of operational CDC data for analytical query. It’s purpose-built for operational data - which is mutating data. You often change it in place. A lot of columnar engines out there, like ClickHouse for example, are primarily optimized for append-only data - logs, metrics, and so forth. We can consume operational data that’s changing in place at super high throughput, and then provide analytical query over it and serve it to your agents.

Going from single-node to full Spark-level distributed compute, built on the Apache Ballista project that we’ve done a lot of work on, is also super exciting. And we have a couple of differentiators there. One is multi-active scheduling: in a lot of these systems you have a driver and executors, and that driver is often a single point of failure - you provide high availability by failing over to a second driver. What we do instead is provide horizontally scalable schedulers, where each one can receive queries and fan out across a shared executor pool - and if any one goes down, another picks up the query and resumes it. You get highly-available, multi-node distributed compute.

So the Cayenne engine, multi-node distributed compute, and this high-throughput CDC replication from your operational stores are the three things that are really exciting about 2.0 - along with all the enterprise features like policy and governance that are just table stakes in the enterprise today.

Looking ahead - what comes after 2.0? What’s the five-year vision?

Schema evolution is actually already implemented - that’s in 2.1, which is releasing very soon. And we’ve had full hybrid search within the platform since about version 1.5: vector search, full-text search, regex, and SQL search - with full re-ranking. We also connect over to partner services like Amazon S3 Vectors, where you can keep a hot tier of vectorized data within Spice for very low-latency queries and store long-tail historical vectors in S3 Vectors, and Spice automatically handles the query across both tiers. Search primarily works on single-node today, so we’ll be expanding it across the full multi-node distributed cluster. And for very high-scale CDC ingest, we’re doing more work to scale that across all of the multi-node executors too - so you can stream millions of changes per second into the system.

But the bigger arc: when we launched in 2021, I think we were a little bit ahead of where the market was. The real vision was - how do you help people build truly intelligent applications that learn and adapt? That means a closed feedback loop. A lot of what you see in RAG today is one-directional: you provide data to the system, you pull it out. But how do you close that loop - where you have operational data, you get insights out of it with analytical query, you provide that to your AI, and your AI gives you additional feedback and data that goes back into your operational store, where you can start making operational control and decisions off it? Once you get that full feedback loop, you start to get the real autonomous application or agent - one that learns, adapts, and gets smarter over time automatically. That’s really the vision of where we take this: closing that loop.

And we already do this today in our own development using AI. As we’re building, we make sure we have all the tests and infrastructure to actually give feedback back to our AI agents as we code. If you can close that feedback loop, you get more iterations and improve so much faster.

It’s launch day when this goes out. For folks reading this - what should they do first, and who should get in touch?

It’s super easy to get started. We have a full cloud-managed product - go to spice.ai, click sign up, and you’ll get a free-tier instance of Spice automatically with all the single-node features. When you want to go multi-node and larger scale, we have a full product there. And if you just want to get started with the open source project, go to the Spice github repo - you can be running it on your laptop within two minutes, and querying Parquet on S3 with super-fast accelerated query a couple of minutes after that.

Stay up to date on the latest with Spice, follow them on LinkedIn and X.

Read our past few Deep Dives below:

If you would like us to ‘Deep Dive’ a founder, team or product launch, please reply to this email ([email protected]) or DM us on Twitter or LinkedIn.