Built a RAG assistant over 12,000 internal docs with citations
Employees wasted hours hunting through 12,000 scattered internal documents for answers that existed somewhere. A RAG assistant with inline citations turned that into a one-line question with a sourced answer.
The problem
A company had 12,000 documents spread across a wiki, a shared drive, and an old knowledge base. The information existed, but finding it meant knowing the right keyword or the right person to ask. New hires were the worst hit, and senior people lost time answering questions a search box should have handled.
The approach
- ▸Built an ingestion pipeline that pulled from the wiki, drive, and knowledge base, chunked by document structure, and kept the source link on every chunk for citations.
- ▸Used hybrid search (keyword plus vector in pgvector) with a re-ranking pass, because pure semantic search missed exact product names and error codes people actually searched for.
- ▸Made every answer cite its sources inline, so people could click through and trust the answer instead of taking the model at its word.
- ▸Added a nightly re-index and a stale-content flag, so the assistant stayed current as docs changed instead of confidently quoting last year's policy.
The result
- ✓Average answer-finding time dropped from ~15 minutes to under 1 minute
- ✓~12,000 documents searchable from one chat box with sourced answers
- ✓Every answer carried inline citations, so trust did not depend on faith in the model
- ✓New-hire ramp questions to senior staff fell by roughly 40%
stack: RAG · pgvector · OpenAI · hybrid search · re-ranking
Citations are what make people trust it
Internal RAG lives or dies on trust. The first time the assistant confidently quotes a policy that was rescinded last quarter, people stop using it and go back to asking a colleague. So two things mattered more than the model choice. First, citations on every answer, so someone could verify in one click instead of taking a paragraph on faith. Second, hybrid search, because internal docs are full of exact tokens (error codes, product SKUs, ticket IDs) that semantic search alone fuzzes over. The unglamorous plumbing, ingestion and re-indexing, is what kept it accurate enough that people kept coming back.
Done by Harshit Luthra, an independent infrastructure and AI engineering consultant. Bring me a similar problem →
Questions about this work
How do you build a RAG assistant over thousands of internal documents?+
An ingestion pipeline pulled from the wiki, drive, and old knowledge base, chunked by document structure, and kept a source link on every chunk. Hybrid search (keyword plus vector in pgvector) with a re-ranking pass handled the exact tokens people actually search, like error codes and product names. A nightly re-index kept it current.
Why does the assistant need citations?+
Internal RAG lives or dies on trust. The first time it confidently quotes a rescinded policy, people stop using it. Inline citations let someone verify an answer in one click instead of taking a paragraph on faith, and they make wrong answers fast to debug because you see exactly what the model was handed.
Why hybrid search instead of pure vector search?+
Internal docs are full of exact tokens, error codes, SKUs, ticket IDs, that semantic search alone fuzzes over. Combining keyword and vector search with a re-ranker fetches the right passage for both natural-language questions and exact-string lookups.