RAG Technology Toolbox Layered Memory Retrieval: The simplest way to load all documents directly into the context window of LLM.
It is suitable for scenarios where the amount of documents is small and the query is simple.
Term-based retrieval: considering term frequency, document length and term rarity, it is suitable for keyword retrieval, is easy to implement, and has significant effects.
Relevance embedding (vector retrieval): used for complex queries that cannot be covered by keywords.
When pre-trained embeddings are not sufficient, especially in professional fields such as law, it is recommended to customize embeddings to better model domain vocabulary.
Reranker (cross-encoder): performs a more refined ranking of candidate documents, taking into account both the query and the document, improving the ranking quality, but the computational cost is high.
Domain-specific ranking signals (vertical semantics): Relevance alone is not enough, structured signals must be introduced, such as prices in e-commerce, listening frequency in podcasts, etc.
User preference signals: Behavioral data such as clicks and likes are crucial to ranking and can reflect real user needs.
Query orchestration (Fan Out): Split complex queries into multiple subqueries to improve recall rate.
For example, Google's AI model will generate 15-20 subqueries for a user input.
Supplementary search: For fuzzy queries, it is recommended to search on multiple backends at the same time to improve coverage.
Distillation and system optimization: When the system becomes slow or the cost is too high, fine-tune smaller, more efficient models (such as Perplexity’s quick Q&A) to maintain quality while reducing resource consumption.
Graceful degradation: When the engineering limit is reached, the experience is adjusted through product design.
For example, "high potential UI" is displayed for queries that are well understood, and basic UI is displayed for queries that are not well understood.