RAG on internal documents, where the work really is

The demo takes a weekend: embed the wiki, wire a model, ask it about the vacation policy, applaud. The production version takes a quarter, and the difference is never the model. It is the three jobs the demo skipped.

The corpus is the product. Internal document bases are archaeological sites: four versions of the travel policy, two of them contradictory, one marked FINAL_v3_actually. Retrieval faithfully surfaces all of it, which means the assistant automates the confusion. Before any embedding happens, someone has to decide which documents are canonical, retire the rest, and put an owner on each source. Unglamorous, decisive, and the step every stalled deployment skipped. (I keep saying it: preparation is the actual adoption strategy.)

Retrieval quality is measurable, so measure it. Twenty real questions from real staff, with the passages that should answer them. Test retrieval separately from generation; techniques like contextual retrieval exist precisely because naive chunking quietly drops the connective tissue that makes passages findable. If the right passage is not in the top results, no prompt downstream will save you.

Answers need permissions and provenance. The assistant must not tell an intern what the executive compensation memo says, which means retrieval has to respect the access model, not the other way around. And every answer cites its sources, so a wrong answer is a correctable document problem instead of a trust collapse. The first hallucinated policy answer that reaches a manager undoes a quarter of goodwill; citations are how you make errors cheap.

Budget for the librarian work, because that is what the corpus stage actually is. Somebody has to decide which of the four conflicting travel policies is current, mark the rest superseded, and own that call going forward. The model cannot do this; it can only summarize the argument your file shares have been having with themselves for a decade. Companies keep discovering that the assistant project was secretly a document hygiene project wearing a chat interface, and the ones that embrace the discovery early ship something trustworthy a quarter sooner than the ones who argue with it.

Sequence it corpus, retrieval, permissions, then interface. Every failed internal RAG project I have autopsied ran the sequence in reverse, led by the chat window, because the chat window demos so very well.