Based on the recommendation of the boss, I combed through a list of high-quality AI Engineer learning materials, which is worth collecting and learning! Too dry, too dry! 🥳🥳🥳 A total of 11 parts are too long to put down, and the remaining 6 parts are placed in the comment area.
- Harness engineering, not just prompt engineering article | Martin Fowler: Harness Engineering for Coding Agent Users — Understanding "agent = model + harness", that is, context assembly outside the model, tool interface, status, execution loop, error handling, evaluation and observation layer. Link: Article | Anthropic: Building Effective AI Agents — Learn engineering principles such as agentic workflow, tool use, agent-computer interface, transparent planning, and simplified design. Link: Article | OpenAI: U nrolling the Codex Agent Loop — See how real coding agent harness organizes models, tools, prompts, performs loops, and performance design. Link: YouTube | How We Build Effective Agents: Barry Zhang, Anthropic — Video supplement to the Anthropic agent architecture article. Link:
- Prompt caching vs. semantic caching tradeoffs Official Documentation | OpenAI Prompt Caching — Learn the provider-side mechanism of prompt caching, applicable conditions, and cached token statistics. Link: Official Documentation | Anthropic Prompt Caching — Learn the difference between automatic caching and explicit cache breakpoints. Link: Article | Redis: Prompt caching vs semantic caching — Build tradeoff: prompt caching for multiplexing fixed long contexts; semantic caching for multiplexing “semantically similar” question answers; production system Often the two are combined. Link: PDF | GPTCache: An Open-Source Semantic Cache for LLM Applications — Thesis-level implementation of learning semantic cache: embedding, similarity search, cache hit, error hit risk, cost/delay benefit. Link:
- KV cache management at scale PDF | vLLM/PagedAttention: Efficient Memory Management for LLM Serving — Core paper, focusing on how PagedAttention divides KV cache into blocks to reduce debris and improve serving throughput. Link: Documentation | vLLM Automatic Prefix Caching Implementation — See Engineering Implementation: KV cache is divided into KV blocks and allows discontinuous physical memory storage. Link: PDF | LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference — Interdisciplinary Request, KV cache multiplexing across engines, offloading, prefill-decode disaggregation. Link: YouTube | Fast LLM Serving with vLLM and PagedAttention — Understand the intuition of vLLM serving, KV cache, PagedAttention with the paper. Link:
- Speculative decoding vs. quantization PDF | Fast Inference from Transformers via Speculative Decoding — speculative decoding A classic paper that understands the mechanism of parallel validation of the draft model by first guessing the token and the target model. Link: PDF | QLoRA: Efficient Finetuning of Quantized LLMs — Understanding quantization fundamentals: 4-bit quantized model, LoRA adapter, NF4, double quantization, paged optimizer. Link: PDF | QSPEC: Speculative Decoding with Complementary Quantization Schemes — Specializing in speculative research how decoding is combined with quantization. Link: Article | Google Cloud: Five techniques to reach the efficient frontier of LLM inference — See continuous batching, paged attention, routing, speculative decoding, quantization in an inference optimization framework. Link: Getting Started Video for YouTube | Faster LLMs: Accelerate Inference with Speculative Decoding — speculative decoding. Link:
- Structured output failures & fallback chains Official Document | OpenAI Structured Outputs — Learn the basic abilities and limitations of JSON Schema, strict schema, and structured response. Link: Article | OpenAI: Introducing Structured Outputs in the API — Understand the difference between JSON mode and Structured Outputs: JSON mode is not equal to schema must be correct. Link: Document | Instructor: Structured LLM Outputs + Validation/Reasking — Automatic retry after learning Pydantic schema and validation failure/ the mode of re-ask. Link: Document | Pydantic AI Output Validation — Why do I need to apply layer validation and retry budget in addition to the native structured output of the mathematical model. Link: Document | Guardrails AI — Learn how to handle raw output, validated output, validation success/failure as system state. Link: YouTube | Validate & Standardize LLM Output with Guardrails-AI — Output Validation and Standardization Hands-on Video. Link: