Is Grep All You Need?
If you are still thinking of “rag = Vector Database” as the default answer, then I recommend that you read this paper.
It is a bit counterintuitive: the author found that in many real Agent scenarios, the most simple grep is actually better than vector retrieval.
This is not to say that vector retrieval is worthless, but it pokes a question that many people have not seriously thought about: is it really the retrieval itself that we usually optimize, or is it just the illusion of indulging in some "advanced component"?
One: Many teams today do rag, the first reaction is embedding, rerank, vector library, recall link, it looks very advanced, but what really affects the results is how the Agent searches, how to read, and how to feed the results to the model.
That is, the problem is not just in retrieval, but in the engineering closed loop of the entire agentic search.
Two: In the inline way, that is, when the tool results are directly crammed into the model context, grep basically leads the way.
Behind this is actually very easy to understand - many questions are not for you to "find similar semantic content", but for you to accurately hit that word, that sentence, that point in time, that name.
At this time, the "local method" of lexical search is more reliable.
Many people lose not to the model, but to "retrieving too smartly, and as a result, not detecting what really needs to be retrieved".
Three: The retrieval effect is good, not equal to the effect of the Agent system.
A key finding in the paper is that inline performs well, but once it is replaced with file-based delivery, that is, the results are written into the file and then read by the model, the effect may be significantly reduced.
This shows that the real difficulty is whether the model can steadily consume these results.
Many systems do not die from recall, but from the loss of tool call chains, context routing, and intermediate steps.
Four: If your task is more like finding evidence, finding original words, and finding certain factual fragments, then things like grep/BM25 should really be valued again.
If your task is more like finding similar semantics, finding relevant context, and doing open induction, vector retrieval is certainly valuable.
The really mature approach is to do routing by task type, mixed retrieval by scenario, and evaluation by system link.
Don't just look at the searcher itself, look at whether the entire Agent has finally handed over the correct answer steadily.
Rag is a complete Agent engineering issue.
Many times, the so-called "advanced solutions" are not necessarily stronger; the really strong systems are often those who use simple methods to get the details of the link right.
Rag should be made into an observable, routable, and evaluable retrieval action system.
The value of this paper is not to get everyone back to grep, but to remind us not to think of the retrieval layer in the Agent as an isolated API; true system capabilities are hidden in harness, tool interfaces, contextual delivery, and failure recovery.