Kolena Introduces AI-agents for Document Analysis and Understanding 🎛

CV Deep Dive

Today, we’re talking with Mohamed Elgendy, Co-Founder and CEO of Kolena.

Kolena is on a mission to revolutionize AI quality and testing for developers and businesses alike. With Mohamed’s extensive background in AI product development at Amazon, Twilio, and Rakuten, Kolena has become a leader in AI quality tools, hosting the inaugural AI Quality Conference in 2024 to set new standards for the industry. Now, Kolena is taking AI beyond developers and into the hands of business users with their latest product, Restructured.

Restructured is designed to help knowledge workers unlock insights from unstructured data like documents, emails, and contracts, bringing AI-powered data analysis to industries such as law, finance, and consulting. By blending LLM capabilities with advanced search and retrieval methods, Restructured offers a powerful, user-friendly solution for navigating complex data environments.

In this conversation, Mohamed shares Kolena’s journey, the development of Restructured, and how the product is transforming how businesses interact with their data.

Let’s dive in ⚡️

Read time: 8 mins

Our Chat with Mohamed 💬

Introduce yourself and give us a bit of background on yourself and Kolena. What led you to co-found Kolena?

I’m Mohamed Elgendy, CEO and co-founder of Kolena. My background is in data science and AI product development at companies like Amazon, Twilio, and Rakuten. At Kolena, we’ve always been committed to building cutting-edge AI Quality and Testing tools for developers. Our unique access to collaborate with leading AI companies and enterprises has allowed us to have a front-row seat to the evolving needs of the industry and confront and solve some of the most complex challenges in AI development. 

In June 2024, we took a significant step forward by hosting the inaugural AI Quality Conference in San Francisco. Our mission was clear: to work with the industry and regulators in defining gold standards for AI quality, setting a higher bar for developers across the field.

The conversations at the conference were illuminating. Again and again, we heard the same request from both developers and business leaders: “Can you extend the same data quality capabilities you built for developers to our business users?” The goal was to empower non-technical teams with the tools to quickly and accurately extract useful information from the enormous, unstructured datasets they deal with daily.

This was an eye-opening moment for us. Not every company gets the opportunity—or the overwhelming demand—to extend their technology beyond its original audience to reach every part of the organization. We knew we had uncovered something important.

The reality is that 80% of enterprise data is unstructured, locked away in documents, slides, emails, and systems that don’t talk to each other. Extracting meaningful insights from this data often requires an army of employees and countless hours of manual effort—an inefficient, unsustainable process for today’s fast-paced organizations.

We saw an opportunity to change this. With Restructured, we are bringing the principles of AI quality — accuracy, reliability, and privacy — into the hands of knowledge workers. This new product is designed to empower business users to access and analyze unstructured information seamlessly, without technical expertise or the need for complex processes.

Just as we’ve solved the hardest problems for developers, we are now solving them for the average knowledge worker. Restructured will unlock the full potential of enterprise data, transforming how businesses operate and delivering a smarter, faster way to work.

We are incredibly excited about this next chapter and what it means for our customers. This is more than just a product—it’s a movement toward democratizing AI-powered insights, making sure that every employee has access to the tools they need to do their best work.

How would you describe Restructured to the uninitiated developer or AI team?

Restructured is an AI-powered assistant designed to work within complex unstructured data environments, at a large scale, helping users find answers across massive troves of documents, contracts, or complex data structures like audio, video, scans, and images. It combines LLM capabilities with search and retrieval methods, meaning it doesn’t just index information—it understands it.

For developers, think of Restructured as a replacement for retrieval-augmented generation (RAG) platform that is much simpler to build and maintain and provides high accuracy levels approaching the 95-99% in most scenarios. RAG is limited with the data size and context window and there are way too many components that need to be managed, tuned, tested, and maintained. RAG is going away. Building relevant extractions to the query and feeding LLMs with structured format of the data blends search capabilities with LLM models to ensure accuracy is the way to go. 

Who are your users today? Who is finding the most value in Restructured?

Our main users today are law firms, consulting firms, financial, and insurance institutions. Teams rely on Restructured to quickly retrieve critical insights, create real-time analytics dashboard, or have grounded conversations about their data. From contracts and case files, to invoices and handwritten forms, all the way to complex invoices. 

We’ve also found traction with corporate strategy teams—they use Restructured to navigate internal knowledge repositories and competitive research.

Which existing use case for Restructured are you most excited about?

I’m particularly excited about how Restructured is being used as an mid-career AI assistant in legal research, discovery, and due diligence workflows. In the insurance domain for example, teams often need to sift through mountains of policies, regulations, and contracts (millions of pages!) in tight timeframes. Restructured helps analysts surface relevant clauses, precedents, or risk factors in seconds, saving them from weeks of manual work.

Any customer success stories you’d like to share? How are you measuring the impact it’s having?

One of our customers, a top-tier law firm, shared that with Restructured they reduced the time spent on case discovery by over 60%. This was particularly significant in high-pressure transactions, where missing a clause or risk could cost millions.

We measure our impact by the time saved per task and the accuracy of retrieved insights. Through these metrics, we’re able to demonstrate tangible value in terms of productivity and risk reduction. Another way we gauge success is through user retention and feedback. Users have told us that Restructured transforms how they approach research, making it feel “invisible”—which is exactly what we aim for: seamless, almost effortless interaction.

How does Restructured actually work under the hood?

Restructured combines vector search, NLP models, and LLM multi-modal models. When a user asks a question or enters a query, our system dynamically searches through unstructured documents—like PDFs, emails, or contracts—and surfaces the most relevant pieces of information.

The system uses vector embeddings to understand the semantic meaning behind queries and documents, ensuring it returns not just keyword matches but contextually relevant answers. We also employ custom fine-tuning to align our AI with the specific domain knowledge, whether it’s legal research, insurance, or financial analysis.

The product architecture emphasizes privacy and security. We enable users to deploy Restructured either as a SaaS solution or on-premise, keeping sensitive data within their control.

Given the excitement around new trends in AI such as Agents and Multimodal AI, how does this factor into your product vision for Restructured?

Agents and multimodal AI open up exciting possibilities for Restructured. We envision Restructured evolving into a knowledge agent that not only answers questions but also proactively suggests relevant references, tracks evolving topics, and helps users form better strategies and analysis reports.

Multimodal AI is a main component in our product architecture — Restructured users are now able to analyze customer support audio calls, videos, images, and PPT slides. Imagine being able to dumb all your data with no prior knowledge on its contents, and instantly be able to analyze it and extract useful information for your task.

What has been the hardest technical challenge around building Restructured into the product it is today?

One of the hardest challenges has been balancing between the 3 pillars of AI Quality: accuracy, reliability, and privacy for vast amounts of data. 

We’ve also spent a lot of time perfecting the retrieval models. Every document is different —contracts don’t look like insurance claims, and case law behaves differently than consulting memos. Creating a generalizable model that handles these differences well has required a lot of iteration.

How do you see Restructured progressing in the next 6-12 months? How do you see the product evolving?

In the next 6-12 months, we plan to introduce more automation and proactive intelligence. Right now, Restructured excels in responsive search, retrieval, and information analysis, but the future lies in predictive insights. We want to put AI into the hands of an average worker in a simple form factor that enables them to run AI tasks just by talking to them. Tasks like: anomaly detection, fraud, predictive maintenance, market and weather predictions, you name it. All the, what i like to call “old school AI” that required specialized products for each task should be now available and ready with high quality to everyone to utilize in their day to day tasks.

Anything else you want people to know or understand about Restructured?

Restructured isn’t just an AI chat tool — it’s an AI-powered partner that evolves with your business. Whether you’re in legal, consulting, insurance, or finance, our mission is to make knowledge work simpler, faster, and produce higher quality work that wasn’t accessible before.

The exciting part is that we’re just getting started. As we continue to iterate, our vision is to build a system that can understand every nuance of your work environment and transform complex information into actionable insights—so you can focus on thinking strategically rather than searching manually.

Conclusion

To stay up to date on the latest with Kolena, learn more about them here.

Read our past few Deep Dives below:

• Our chat with Datastax's Chief Product Officer, Ed Anuff ❇️

• TensorStax is building Autonomous AI for MLOps 🌐

• Dashworks, your AI-powered internal search tool 🔍

• Delphi - the verified digital cloning platform 👥

• CodeGPT is intertwining human language with code 🌏

If you would like us to ‘Deep Dive’ a founder, team or product launch, please reply to this email ([email protected]) or DM us on Twitter or LinkedIn.

Join Slack | All Events | Jobs