Working Notes on Agent Systems/Brad Zhang

@teach_fireworks / X longform

RAG Technology Toolbox Layered Memory Retrieval: The simplest way to load all documents directly into the context window of LLM.

RAG Technology Toolbox Layered Memory Retrieval: The simplest way to load all documents directly into the context window of LLM. It is suitable for scenarios where the amount of...

August 10, 2025 · 2 min read

RAG Technology Toolbox Layered Memory Retrieval: The simplest way to load all documents directly into the context window of LLM.

It is suitable for scenarios where the amount of documents is small and the query is simple.

Term-based retrieval: considering term frequency, document length and term rarity, it is suitable for keyword retrieval, is easy to implement, and has significant effects.

Relevance embedding (vector retrieval): used for complex queries that cannot be covered by keywords.

When pre-trained embeddings are not sufficient, especially in professional fields such as law, it is recommended to customize embeddings to better model domain vocabulary.

Reranker (cross-encoder): performs a more refined ranking of candidate documents, taking into account both the query and the document, improving the ranking quality, but the computational cost is high.

Domain-specific ranking signals (vertical semantics): Relevance alone is not enough, structured signals must be introduced, such as prices in e-commerce, listening frequency in podcasts, etc.

User preference signals: Behavioral data such as clicks and likes are crucial to ranking and can reflect real user needs.

Query orchestration (Fan Out): Split complex queries into multiple subqueries to improve recall rate.

For example, Google's AI model will generate 15-20 subqueries for a user input.

Supplementary search: For fuzzy queries, it is recommended to search on multiple backends at the same time to improve coverage.

Distillation and system optimization: When the system becomes slow or the cost is too high, fine-tune smaller, more efficient models (such as Perplexity’s quick Q&A) to maintain quality while reducing resource consumption.

Graceful degradation: When the engineering limit is reached, the experience is adjusted through product design.

For example, "high potential UI" is displayed for queries that are well understood, and basic UI is displayed for queries that are not well understood.

Visual summary

Article argument map

Generated from the post's content graph

FORMATTOPICCAPABILITYMARKETcoverscoverscoverscoverscoverssignalssignalssignalsFORMATlongform noteTOPICtechnical distributionTOPICmemoryTOPICretrievalTOPICevaluationTOPICtoolingCAPABILITYdeveloper toolingCAPABILITYevaluationCAPABILITYproduct surfaceMARKETopen-source builders
Mermaid outline
flowchart LR
  format-long_post["longform note"]
  topic-technical-distribution["technical distribution"]
  topic-memory["memory"]
  topic-retrieval["retrieval"]
  topic-evaluation["evaluation"]
  topic-tooling["tooling"]
  capability-developer-tooling["developer tooling"]
  capability-evaluation["evaluation"]
  capability-product-surface["product surface"]
  market-open-source-builders["open-source builders"]
  format-long_post -->|covers| topic-technical-distribution
  format-long_post -->|covers| topic-memory
  format-long_post -->|covers| topic-retrieval
  format-long_post -->|covers| topic-evaluation
  format-long_post -->|covers| topic-tooling
  format-long_post -->|signals| capability-developer-tooling
  format-long_post -->|signals| capability-evaluation
  format-long_post -->|signals| capability-product-surface

Visual structure

Essay structure map

Built from summary and key paragraph positions

RAG Technology Toolbox Layered Memory Retrieval: The simplest way to load all documen...THESISRAG Technology ToolboxLayered MemoryRetrieval: Thesimplest way to loadSIGNALRAG Technology ToolboxLayered MemoryRetrieval: Thesimplest way to loadOPERATORUser preferencesignals: Behavioraldata such as clicksand likes are crucialIMPLICATIONFor example, "highpotential UI" isdisplayed for queriesthat are well
Mermaid outline
flowchart LR
  thesis["RAG Technology Toolbox Layered Memory Retrieval: The simplest way to load all documents directly into the c..."]
  signal["RAG Technology Toolbox Layered Memory Retrieval: The simplest way to load all documents directly into the c..."]
  operator["User preference signals: Behavioral data such as clicks and likes are crucial to ranking and can reflect re..."]
  implication["For example, \"high potential UI\" is displayed for queries that are well understood, and basic UI is display..."]
  thesis -->|frames| signal
  signal -->|develops| operator
  operator -->|lands in| implication

Source: View the original post