Why You Might Only Need a Folder Structure and a Claude Subscription

Most small companies do not need a vector database. They need a folder of markdown files with predictable names and a Claude subscription. Anthropic itself dropped the vector index from Claude Code because plain file tools beat it. For a team under 200 people, the same logic holds for your contracts, notes and SOPs.

The default AI infrastructure pitch is built for someone else

The standard enterprise AI stack assumes a problem most small companies do not have.

Walk into a conversation about "doing AI" and the first question is usually about RAG, embeddings, a vector store, maybe a knowledge graph on top. There is a pipeline, a chunker, a re-ranker and an evaluation harness. Six months of engineering before anything ships.

That stack was designed for corpora measured in millions of documents and dozens of teams. A 20 person consultancy, a Danish SMV, a solo founder with a Drive folder: none of them have that corpus. They have a few thousand files and one person who knows where things live.

The infrastructure was designed for a different shape of problem. Buying it anyway is how small teams spend the year that should have produced output building scaffolding for output.

Anthropic's own evidence that the index is optional

The team that built Claude Code tried RAG and removed it.

Boris Cherny, the engineer behind Claude Code, said it directly in an interview with The Pragmatic Engineer: "Early versions of Claude Code used RAG plus a local vector db, but we found pretty quickly that agentic search generally works better. It is also simpler and doesn't have the same issues around security, privacy, staleness, and reliability."

The replacement is three tools. Glob finds files by name pattern. Grep searches inside files. Read opens a specific file. The model uses them the way a person who knew the codebase would, before they opened anything. Vadim Geshel's teardown of Claude Code's architecture is the clearest write-up of how this actually behaves at runtime.

Anthropic ran the experiment on codebases with hundreds of thousands of files. They got the same answer the LlamaIndex team reached in January 2026 in "Files Are All You Need": for focused document sets, filesystem agents are more accurate than RAG. RAG wins on speed at large scale. Folders win on accuracy where most small companies actually live.

What this looks like for a non-engineering business

The setup that ships in an afternoon for most SMEs.

You already have a folder of documents. The single change that matters is naming. 2026-Q1-contract-acme-renewal.md is a file the agent can find from a description. final_final_v2.pdf is not. The cost of renaming is one focused afternoon. The cost of not renaming is paying for a vector pipeline to compensate for the bad names.

The second change is writing one index file at the root: a short markdown page that explains what each folder holds and what the naming convention is. The agent reads it on every session. It is the same one page brief you would give a new hire on their first day.

The third change is the access point. Either Claude Code running against the folder, or the Claude desktop app with the filesystem MCP server, or a custom interface that exposes Glob, Grep, Read. For most teams under 200 people, this is the entire infrastructure. Pricing for the Claude Max tier sits at 100 USD per month for individuals, with team seats starting around 100 USD per seat per month for Claude Code access, per Anthropic's published pricing page in May 2026. The vector pipeline it replaces costs more than that just to evaluate.

Why long context made this work

The shape of the model changed and the stack should have changed with it.

When context windows held a few thousand tokens, you needed retrieval because the model could not hold a document. The context window is simply how much text the model can keep in front of it at once. You chunked, embedded, ranked and stitched fragments back together at query time. That was the whole reason RAG existed.

Context windows in 2026 sit at one million tokens and counting on the major frontier models. The constraint that made chunking rational has loosened. Anthropic's September 2025 context engineering post made the discipline explicit: the work is finding the smallest set of high signal tokens, not the largest set of vaguely related ones. A folder navigator reads the three files that actually matter. A vector store dumps the top twenty chunks and hopes the model sorts it out.

Simon Willison's February 2026 piece on file native agentic systems pushed the same direction, citing 9,649 experiments across structured context formats where frontier models gained measurable accuracy from filesystem style retrieval. The pattern is converging across labs, not coming from one vendor.

Where the approach stops working

Here is the honest list. Folders are not magic.

True scale breaks the pattern. If your corpus is millions of documents (legal discovery, scientific literature, customer support archives at a large company), the agent cannot enumerate the directory and a vector index becomes the right tool. The LlamaIndex benchmark put the crossover roughly between 100 and 1,000 documents, with RAG pulling ahead beyond that on speed.

Real time data falls outside this entirely. A folder is a snapshot. Stock prices, live inventory, IoT telemetry: the agent needs a tool that queries a system, not a file that mirrors one. That is what MCP servers are for, and it is the topic of a different piece.

Multi tenant compliance is the other ceiling. The moment three departments need different views of the same documents with audit logging and per row access control, you are no longer in folder territory. You are in a permissions engine, and folders cannot model what you need.

The pattern also fails when the documents themselves are bad. A folder of inconsistent, undated, half versioned files does not become coherent because an agent reads it. The discipline of naming and indexing is the work. The agent multiplies that work. It does not replace it.

In our work with Danish SMEs at Ellypsis, the default opening move is the folder. We add RAG when document volume genuinely demands it, not before. Most of the time it does not demand it, which is the uncomfortable part of the pitch most vendors will not say.

If you want the underlying comparison between vector RAG, knowledge graphs and the folder approach, the long form version is here: vector RAG, knowledge graphs or structured folders.

The connectivity layer that lets Claude read your folders, or any other system, safely is covered in what MCP is and why it matters for your business.