RAG
Published Updated
What is RAG?
RAG (retrieval-augmented generation) improves an AI model’s answers by first searching an external source for relevant facts and inserting them into the prompt, so the model answers from current data rather than memory alone.
An architecture that improves AI responses by searching an external database for relevant facts and inserting them into the prompt before the model answers.
Retrieval-Augmented Generation (RAG) grounds an LLM in specific, private data it wasn't trained on. When a user asks a question, the system first queries a database (often using embeddings) to find relevant documents. It then bundles those documents with the user's question, forcing the model to generate its answer based strictly on the retrieved context.
How it relates to Orbit
Apollo works on the same principle: Orbit injects live project state into its context, so you never paste status into the chat yourself. See Apollo, the Co-pilot for details.
When you would use it
You use RAG when you need an AI model to answer questions about proprietary, real-time, or highly specialized data without going through the expensive process of fine-tuning the model itself.
Common operations
- Building a chatbot that answers questions based purely on internal company wikis.
- Reducing LLM hallucinations by forcing citations to specific retrieved documents.
Frequently asked questions
Why use RAG instead of fine-tuning?
RAG keeps knowledge up to date without retraining: change the documents and the next answer changes too.
Does Orbit use retrieval?
Apollo works on the same idea: Orbit injects live project state into its context so you never paste it in, and Memtrace offers semantic code search. See Apollo, the Co-pilot.