Classic RAG Was Definitely Not Enough

Classic RAG Was Definitely Not Enough

When we initially worked with classic Retrieval-Augmented Generation (RAG), it was clear that it was simply not sufficient for our needs. As we built a textbook architecture, we encountered many obstacles and failures along the way.

Now, we have something that will soon be our internal perplexity engine. It will work with private data and provide superpowers to users.

Today, let's dive a little into the architecture. The image below gives a bird's-eye view of the result of many decisions, but as always, the devil is in the details.

Key Features of the Solution:

1. Combining Linguistic Intelligence with Precise Mathematical Recommendations – We merge language understanding with data-driven, precise outcomes.
2. Scalability – The system is capable of handling hundreds of thousands of documents.
3. Efficiency – It supports thousands of concurrent conversations.
4. Personalization – The system understands and adapts to each user individually.
5. Cloud or On-Premise – Flexible deployment options to suit different needs.
6. Wide Access to LLMs from Various Providers – While we have access to a wide range of large language models (LLMs), we primarily use our curated shortlist.

Author: Mariusz Jażdżyk

Lecturer at Kozminski University, author of the book “Chief Data Officer,” specializing in building data-driven organizations. He supports startups in the practical implementation of data strategies and AI solutions.