RAG vs. the LLM Wiki: Two Ways to Give an AI Memory
I'm building an airgapped research agent, and one question kept coming back: should it retrieve raw chunks (RAG) or read a curated wiki? They sound like rivals. They aren't — and here's how I actually think about it.
More in Working with AI
- How to Prove a Fine-Tune Actually Reduced Hallucination — Not Just Memorized Your Test Set
- How to Fine-Tune a Local LLM on an Apple Silicon Mac (MLX + LoRA, the Actual Commands)
- Why one benchmark won't tell you the best coding LLM in 2026 — and which three together actually do
- Designing Frontends Claude Can Actually Use — A 7-Step Field Guide
- A Build-Pipeline Checklist for Catching AI-Fabricated Research Citations
I've been building a local, airgapped agent that works on top of a research database — no internet, everything running on the box. Early on, one question kept resurfacing and quietly shaping the whole architecture: when the agent needs to know something, should it pull raw fragments out of a vector store (RAG), or read from a curated wiki that a human and an AI maintain together?
They're usually pitched as competitors. After living with both, I don't think that framing survives contact with a real system. Here's the mental model I landed on — starting from the ground up.
RAGvsthe LLM Wiki
Both are ways to let an AI lean on your documents. But they treat memory in opposite ways — one grabs fragments on demand, the other curates them in advance.
Fetch fragments when needed
A question comes in, and the system pulls similar-looking chunks from a big pile of raw documents on the spot and hands them to the model. Automatic, and it scales no matter how much you throw at it.
Curate it ahead of time
A human and an AI keep a set of distilled wiki pages. The model reads the index first, then reads whole relevant pages. It takes effort, but the answers come out tidy and trustworthy.
A box of photocopied scraps vs. a well-made encyclopedia
Picture the same library, used two different ways.
RAGShred thousands of pages into a box, and when a question arrives, fish out the few scraps that look related and drop them on the AI's desk. Fast and automatic — but they're fragments stripped of context, so an off-topic scrap sneaks in, or the page that actually mattered never surfaces.
LLM WikiA librarian summarizes the essentials into pages and wires them together with an index and links. The AI skims the index and reads the right page in full. Prep takes work — but by the time it reads, the knowledge is already verified and connected.
Same question, different route
The core difference is what you retrieve. RAG returns a chunk; the wiki returns a whole page found via the index. And RAG's knowledge only updates when the sources change — while the wiki's knowledge gets tidier the more you use it.
Laid out next to each other
| Dimension | RAG | LLM Wiki |
|---|---|---|
| Unit retrieved | Document chunk — context clipped | Whole page — context intact |
| How it finds | Semantic similarity (vector distance) | Explicit index + link navigation |
| Curation | Almost none — raw text goes in | Human + AI distill, resolve, verify |
| How it grows | Add & re-embed sources | Answers filed back as pages — self-accumulating |
| Contradictions | Conflicting chunks can arrive together | Resolved or flagged during curation |
| Setup cost | Low — drop it in, it's searchable | High — needs ongoing curation discipline |
| Scale | Millions of docs, automatically | As much as humans can shepherd (the core) |
| Sweet spot | Broad recall over vast, messy sources | Curated domain knowledge, trusted answers, reasoning |
There's no free lunch
RAG
- Drop it in and it's searchable — light setup
- Scales to however many sources you have
- Runs automatically, no human upkeep
- Context-less chunks cause wrong citations
- Answer quality rides on chunking & embeddings
- Doesn't filter out conflicts or duplicates
LLM Wiki
- Reads verified knowledge with context intact
- Index & links pin the exactly relevant pages
- Gets tidier as you use it — it grows itself
- Dies without curation discipline
- Only scales to what humans can maintain
- Goes stale and polluted if neglected
You don't pick one — you stack them
They aren't rivals; they play different roles. A good system doesn't drop either — it layers them: trusted answers come from the wiki first, broad raw coverage is backed by RAG. (The airgapped agent I'm building runs exactly this two-tier setup.)
Human + AI curated pages. Read the index first to frame the answer you can trust.
The vast, un-curated originals. Whatever the wiki hasn't covered — the corners, the latest, the details — swept up by hybrid vector + keyword search.
Add a layer that computes numbers in code (not RAG, not the wiki) and one that checks every answer against its sources, and you get an agent that works without making things up — and shows its receipts.
RAG = Retrieval-Augmented Generation · LLM Wiki = Andrej Karpathy's index-first knowledge base that humans and LLMs read and write together.
The one-liner — RAG is on-the-fly retrieval that grabs fragments; the LLM Wiki is curation you read whole. In practice, you stack them.
That last layer — compute-in-code plus a verification gate — is the part I care about most, but it's a story for another post. For now: if you're wiring up retrieval for an agent and find yourself arguing "RAG or a wiki," try reframing it as "wiki first, RAG underneath." The question mostly dissolves.
Written by
Jay Lee
Korea-Licensed Pharmacist (#68652) · Senior Researcher
Korea University, College of Pharmacy (B.S. + M.S., drug delivery systems & industrial pharmacy). Building production-grade AI tools across medicine, finance, and productivity — without a CS degree. Domain expertise first, code second.
About the author →Related posts