Why most RAG implementations underperform
A basic RAG pipeline is easy to assemble in an afternoon: chunk the documents, embed them, retrieve the top matches, paste them into a prompt. It demos well and then disappoints, because retrieval quality collapses the moment real users ask real questions against real document sets.
The failures are consistent and fixable. Chunks split mid-argument so retrieved passages lack context. Pure vector search misses exact terms like part numbers and clause references. The top result is relevant but the genuinely correct passage sits at rank eleven. And nobody built an evaluation set, so quality is measured by whether the last demo went well.
- Chunking strategy matched to document structure, not a fixed token count
- Hybrid retrieval combining semantic search, keyword search and metadata filters
- Cross-encoder re-ranking to promote the genuinely correct passage
- Citations linked to source documents so answers are verifiable
- Evaluation pipelines that catch regressions before users do
The parts that decide whether it works
Retrieval quality — not model quality — determines whether a RAG system is useful. A strong model given the wrong passage produces a confident, well-written, wrong answer. That is worse than no answer, because it is harder to detect.
We tune retrieval against a golden question set drawn from your real users. Every configuration change is scored against it, so improvements are measured rather than assumed, and regressions surface immediately.
Grounding, citations and refusal
A production RAG system must be able to say "I do not know." Systems that always answer will always answer, including when the retrieved context contains nothing relevant. We build explicit refusal behaviour and tune the threshold to your risk tolerance.
Every answer carries citations to the source passages it drew on, so a user can verify a claim in a click. In regulated environments this is not a nice-to-have — it is the difference between a deployable system and one legal will not approve.
What you receive
- Production RAG pipeline deployed to your infrastructure
- Golden question set and automated evaluation harness
- Hybrid retrieval with re-ranking, tuned to your corpus
- Citation UI linking every answer to source documents
- Ingestion pipeline for ongoing document updates
Technologies we use
RAG System Development — frequently asked questions
What does a RAG system cost to build?
A focused RAG assistant over a single document corpus typically runs $18K to $35K. Enterprise systems spanning multiple sources with access control and compliance review range from $45K to $110K. The main cost drivers are source count, access-control complexity and the required accuracy bar.
How do you stop a RAG system hallucinating?
Grounding, refusal and evaluation together. Answers are constrained to retrieved context, every claim carries a citation, and the system is tuned to decline when retrieval returns nothing relevant. The evaluation harness then scores grounding on every change, so regressions are caught before release rather than reported by users.
RAG or fine-tuning — which do we need?
RAG for knowledge that changes or must be cited; fine-tuning for behaviour, format and tone. If you need the system to answer from documents that update, RAG is the answer, because retraining on every document change is impractical. Many production systems use both — fine-tuning for how it responds, retrieval for what it knows.
How much data do we need?
RAG works with whatever you have — there is no training-data threshold, because nothing is being trained. It works over a hundred documents or a hundred thousand. What matters more than volume is whether the documents actually contain the answers users will ask for.
Can RAG respect our existing permissions?
Yes, and it must. Access control is enforced at the retrieval layer, so a user physically cannot retrieve a passage from a document they are not entitled to read. Filtering at the answer layer instead is a data leak waiting to happen.
Talk through your rag system development project
A free 30-minute call with an engineer — not a salesperson. You leave with a scope, a cost range and an honest view of the risks.
Book a free strategy call
