Working Notes on Agent Systems/Brad Zhang

@teach_fireworks / X longform

Good in-depth article: AI's "deep water zone": What kind of tipping point are we waiting for?

Good in-depth article: AI's "deep water zone": What kind of tipping point are we waiting for? Recently I looked at two seemingly unrelated things side by side. One...

December 26, 2025 · 7 min read

Good in-depth article: AI's "deep water zone": What kind of tipping point are we waiting for? Recently I looked at two seemingly unrelated things side by side. One is McKinsey’s latest AI Agent application chart. The cold data is all the current “reality”. The other is an in-depth interview with Sebastian Borgeaud, who is responsible for Gemini 3 pre-training at Google DeepMind, talking about how the model can think like a human being. This is about the "future". Putting these two things together, I suddenly understood what kind of awkward and fascinating stage we are in now. To be honest, the word "Agent" has been too hot in the past six months. It’s so hot that it feels like when I wake up tomorrow morning, my computer will automatically help me write my weekly report, order coffee, and make a strategic plan for next year. But the data gave a basin of cold water. 📷 The reality is: only “your own people” can use it best. Look carefully at that McKinsey chart, it’s particularly interesting. Among all industries, the technology industry is the only “all-around player”. If you look at its data on "software engineering", 24% of the respondents said that they have achieved large-scale application. This is amazing. This means that within one of every four technology companies, AI is no longer a toy, but an infrastructure like water and electricity, helping programmers write code and correct bugs. But it's also ironic. People who do AI make the best use of AI. It's like the chef knows how to eat his own food. This is both an advantage and a limitation. Let’s look at other industries. IT and knowledge management are the only two fields that have a slight presence in all walks of life. What does this mean? It shows that for most non-tech companies, AI is currently just an "efficiency tool" used to repair computers and check documents. It hasn't really penetrated into the core heart of the business. With one exception: the insurance industry. The insurance industry’s application proportions in “marketing and sales” and “risk control and legal affairs” are alarmingly high, at 20% and 16% respectively. Even more than financial institutions. Why insurance? I thought about it and found that the logic is actually very simple. The essence of the insurance industry is actually rule-based text processing and probability calculation. This perfectly coincides with the circle of competence of LLM (Large Language Model). Insurance does not need to tighten screws like manufacturing, nor does it have a very low error tolerance rate like medical care where life is at stake. It's about dealing with massive terms and assessing risks. Therefore, rather than saying that AI has changed insurance, it is better to say that the insurance industry is inherently a training ground for AI. Where is the bottleneck? Why hasn’t it “exploded” yet? Since both the technology industry and the insurance industry can make good use of it, why haven’t we seen the explosion of AI Agents in the whole society? This brings us to the information revealed by Sebastian Borgeaud in the interview. The current models, even the Gemini 1.5 Pro or the GPT-4 generation, although they are very strong, they are more of a "gut reaction". You ask, it answers. But real-world tasks are complex. For example, you want AI to "help me plan a company annual meeting." This is more than just generating a document. It requires checking venues, comparing prices, sending emails to inquire about schedules, and coordinating staff time. There are countless decision-making chains here. Just one wrong step and the entire mission falls apart. This is why current Agents look a bit "dumb" or can only do very simple things. Because they lack an ability. Sebastian calls this ability Deep Think. One of the core directions of Gemini 3 is to let the model learn to "slow down". Before answering you, perform multi-step reasoning in your mind (or in the background of the system). Think about plan A first, and then think about plan B, and then think about plan B, and finally give you an optimal solution. This is not just a larger model, this is a qualitative change in the way of thinking. If you compare today's AI to an intern who responds very quickly, he will respond instantly to whatever you say, but he often fails to think. Then the AI ​​powered by Deep Think is like a calm expert. He will hold a pipe and meditate for two minutes, and then give you a watertight plan. This is the key for Agent to truly step out of the "technology circle" and enter the complex business world. Systems Engineering: The Neglected Moat There is another point in the interview that particularly touched me. We always stare at the "model" and think that whoever's model scores higher will win. But Sebastian told the truth: we are no longer just building a model, we are building a complex system. The power of Gemini 3 is not due to the discovery of any earth-shattering mathematical formula, but the superposition of hundreds or thousands of small improvements. It's like building a car. Just because you can build the most powerful engine (model) doesn't mean you can build the fastest racing car. You need the perfect coordination of gearbox, chassis, and aerodynamics. In the field of AI, this is the engineering art of native multi-modal architecture (the same brain processes image, text and sound) and MoE (mixed expert model). This also explains why in McKinsey’s chart, the technology industry is far ahead. Because only technology companies have this kind of systems engineering capability. They are not buying an API and using it, they are breaking up AI and integrating it into their own infrastructure. The "Dusk" and "Dawn" of Data Regarding the future, there is another signal that is both exciting and worrying. We are entering a Data Limited Regime. Simply put, the Internet is running out of high-quality human writing. We used to think that data was infinite, but now we find that that is an illusion. Does this sound desperate? Not really. This has instead forced a new evolutionary path: synthetic data. Let the AI ​​generate high-quality data to train itself. This sounds a bit like "left foot on the right foot and go to heaven", but it is mathematically possible. As long as there is a strong enough "teacher model" to judge, AI can evolve in self-game. This is how AlphaGo defeated human chess players back then. Now it’s the language model’s turn. This also means that the competition in the future will no longer be who crawls more web pages, but who can design a better "curriculum" so that AI can learn on its own. After reading these two materials, my anxiety was relieved. The "difficulty in implementing AI Agent" we see now is not because the technology is no longer good, but because we are in a period of shifting gears. Shifting gears from “intuitive AI” to “reasoning AI”. Shift from "fighting for data quantity" to "fighting for data quality." Shift from "playing with models" to "working on the system". For us ordinary people, or practitioners with non-technical backgrounds, here are a few practical suggestions: First, don’t rush to hand over all business to Agent. Let’s look at McKinsey’s data. Let’s start with IT operations and knowledge base management, which have high fault tolerance and high degree of standardization. Don’t expect AI to help you handle complex customer negotiations right from the start. Second, focus on the “thinking process.” When using AI, use the Chain of Thought prompt word technique. Now that the model is developing towards Deep Think, our current interaction method must also keep up and guide it to think step by step instead of just seeking one result. Third, systems thinking. Whether you are doing products or operations, don’t treat AI as a plug-in. Think about how to restructure your workflow. Just like what technology companies do in software engineering, AI is not an auxiliary, it is the new foundation. In this era, the most dangerous thing is not not knowing technology, but using old maps to find new continents. Technology is evolving, and our knowledge must also iterate. Don't be fooled by short-term bubbles, and don't leave the market because of temporary bottlenecks.

Visual summary

Article argument map

Generated from the post's content graph

FORMATTOPICCAPABILITYMARKETcoverscoverssignalssignalssignalssignalssignalsFORMATarticle featureTOPICagentsTOPICtoolingCAPABILITYagent workflowCAPABILITYdeveloper toolingCAPABILITYevaluationCAPABILITYproduct surfaceMARKETopen-source builders
Mermaid outline
flowchart LR
  format-article["article feature"]
  topic-agents["agents"]
  topic-tooling["tooling"]
  capability-agent-workflow["agent workflow"]
  capability-developer-tooling["developer tooling"]
  capability-evaluation["evaluation"]
  capability-product-surface["product surface"]
  market-open-source-builders["open-source builders"]
  format-article -->|covers| topic-agents
  format-article -->|covers| topic-tooling
  format-article -->|signals| capability-agent-workflow
  format-article -->|signals| capability-developer-tooling
  format-article -->|signals| capability-evaluation
  format-article -->|signals| capability-product-surface
  format-article -->|signals| market-open-source-builders

Visual structure

Essay structure map

Built from summary and key paragraph positions

Good in-depth article: AI's "deep water zone": What kind of tipping point are we wait...THESISGood in-depth article:AI's "deep waterzone": What kind oftipping point are weSIGNALGood in-depth article:AI's "deep waterzone": What kind oftipping point are weOPERATORGood in-depth article:AI's "deep waterzone": What kind oftipping point are weIMPLICATIONGood in-depth article:AI's "deep waterzone": What kind oftipping point are we
Mermaid outline
flowchart LR
  thesis["Good in-depth article: AI's \"deep water zone\": What kind of tipping point are we waiting for? Recently I lo..."]
  signal["Good in-depth article: AI's \"deep water zone\": What kind of tipping point are we waiting for? Recently I lo..."]
  operator["Good in-depth article: AI's \"deep water zone\": What kind of tipping point are we waiting for? Recently I lo..."]
  implication["Good in-depth article: AI's \"deep water zone\": What kind of tipping point are we waiting for? Recently I lo..."]
  thesis -->|frames| signal
  signal -->|develops| operator
  operator -->|lands in| implication

Source: View the original post