RAG Systems & AI Chatbots
I build chatbots and assistants over your own documents that answer accurately and cite their sources. Most hallucination problems are retrieval problems, so I fix retrieval first, add evals, and ship something users trust.
Sound familiar?
- ✕The chatbot confidently invents answers that aren't in your docs
- ✕It retrieves the wrong passages, so the answer is off-topic
- ✕Users don't trust it because there are no citations to check
- ✕It works on a few test questions and falls apart on real ones
- ✕You have thousands of documents and no idea if retrieval is any good
- ✕Nobody can tell if a change made the bot better or worse
What you get
- ✓A chatbot that answers from your data and cites the source for each claim
- ✓A retrieval pipeline measured for accuracy, not eyeballed once
- ✓An eval set so quality is a number you watch on every change
- ✓Guardrails that make the bot say "I don't know" instead of guessing
- ✓A system your team can extend and operate after I hand it over
Most chatbot hallucination is a retrieval problem wearing a model costume
A chatbot that answers from your own documents lives or dies on retrieval. If the right passage reaches the prompt, a decent model answers well. If it doesn’t, the model guesses, and you get a confident wrong answer that erodes trust fast. I spend most days running Kubernetes and shipping AI infra, and I’ve seen plenty of demo chatbots that never survived contact with real users. The fix is rarely a fancier model. It’s better retrieval, honest evals, and guardrails.
What I help with
- Chatbots over your documents. Assistants over your wiki, docs, tickets, PDFs, or database, answering from your content rather than the model’s training data.
- Retrieval quality. Chunking, embeddings, hybrid search, and rerankers tuned to your data so the right passage actually gets fetched.
- Hallucination control. Grounding, citations, and a path for the bot to say “I don’t know” instead of inventing an answer.
- Evals. A real question set and scoring for retrieval and generation, so quality is a measured number on every change.
- Customer support and internal assistants. Support bots that deflect tickets and internal knowledge assistants that answer from the docs your team keeps re-asking about.
Why me for this
AI infrastructure is my day job at TrueFoundry. I work on the serving, gateway, and retrieval layers that sit under chatbots at production scale. You get someone who understands both the retrieval pipeline and the system that has to run it once real traffic arrives.
How an engagement works
- Scope. What the bot needs to answer, where the content lives, and the accuracy bar you need to clear.
- Build retrieval. Get chunking, embeddings, and search right first, then measure recall on a real question set.
- Generate and guardrail. Add grounded answers with citations, plus an honest “I don’t know” path, and score it end to end.
- Hand off. Eval harness, runbooks, and a system your team can extend without me.
Frequently asked questions
Why does my RAG chatbot keep hallucinating?+
In most cases the model is fine and retrieval is the problem. If the right passage never reaches the prompt, the model fills the gap by guessing. I look at chunking (chunks that split a fact in half wreck retrieval), embedding choice, and whether you need hybrid keyword plus vector search or a reranker. Then I add a step that lets the bot say it doesn't know when nothing relevant comes back. That combination removes the large majority of made-up answers.
How do I know if my chatbot's answers are actually accurate?+
You build an eval set, which is 50 to 200 real questions with known-good answers, and you score retrieval and generation separately on every change. Retrieval metrics like recall at k tell you whether the right chunk was even fetched. Answer scoring, often an LLM judge plus a few human spot checks, tells you whether the response was correct and grounded. Without this you are flying blind and shipping on vibes.
How long does it take to build a production RAG chatbot?+
A working prototype over your documents takes days. Production-ready, meaning measured accuracy, citations, guardrails, latency under control, and observability, is usually two to six weeks depending on how messy your data is. The data cleanup and the eval loop take the real time, not the LLM call.
Can the chatbot show where each answer came from?+
Yes, and it should. Every answer can link back to the source chunks it used, so a user can click through and verify. Citations do two things. They build trust, and they give you a fast way to debug wrong answers because you can see exactly what the model was handed.
related work
Where I’ve done this
Built a customer-support AI agent that deflected ~60% of tickets
A support team was drowning in repetitive tickets that the help docs already answered. An AI agent over those docs and past tickets, with order-lookup tools and a clean human handoff, took ~60% of volume off the queue.
Answer-finding time cut from ~15 minutes to under 1Built a RAG assistant over 12,000 internal docs with citations
Employees wasted hours hunting through 12,000 scattered internal documents for answers that existed somewhere. A RAG assistant with inline citations turned that into a one-line question with a sourced answer.
Answer accuracy raised from 72% to 96%Took a support bot from 72% to 96% answer accuracy
A support bot was already live and quietly wrong about a quarter of the time, with no one measuring it. An eval harness, fixed retrieval and grounding, and guardrails took accuracy from 72% to 96%.