Working Notes on Agent Systems/Brad Zhang

@teach_fireworks / X longform

The model benchmark is very high. Why does it still happen that "the code has been chan...

The model benchmark is very high. Why does it still happen that "the code has been changed and the task failed" after connecting to the real Agent? This issue only explains one...

July 18, 2026 · 1 min read

The model benchmark is very high.

Why does it still happen that "the code has been changed and the task failed" after connecting to the real Agent?

This issue only explains one mechanism: the model provides capabilities, and Harness determines how these capabilities pass through tools, Retry Budget, Verifier, failure loops, and readback, and finally become verifiable delivery.

Visual summary

Article argument map

Generated from the post's content graph

FORMATTOPICCAPABILITYMARKETcoverscoverscoverscoverssignalssignalssignalsFORMATshort signalTOPICagentsTOPICharness engineeringTOPICevaluationTOPICtoolingCAPABILITYagent workflowCAPABILITYharness engineeringCAPABILITYevaluation
Mermaid outline
flowchart LR
  format-short_note["short signal"]
  topic-agents["agents"]
  topic-harness-engineering["harness engineering"]
  topic-evaluation["evaluation"]
  topic-tooling["tooling"]
  capability-agent-workflow["agent workflow"]
  capability-harness-engineering["harness engineering"]
  capability-evaluation["evaluation"]
  format-short_note -->|covers| topic-agents
  format-short_note -->|covers| topic-harness-engineering
  format-short_note -->|covers| topic-evaluation
  format-short_note -->|covers| topic-tooling
  format-short_note -->|signals| capability-agent-workflow
  format-short_note -->|signals| capability-harness-engineering
  format-short_note -->|signals| capability-evaluation

Visual structure

Essay structure map

Built from summary and key paragraph positions

The model benchmark is very high. Why does it still happen that "the code has been ch...THESISThe model benchmark isvery high. Why does itstill happen that "thecode has been changedSIGNALThe model benchmark isvery high.OPERATORWhy does it stillhappen that "the codehas been changed andthe task failed" afterIMPLICATIONThis issue onlyexplains onemechanism: the modelprovides capabilities,
Mermaid outline
flowchart LR
  thesis["The model benchmark is very high. Why does it still happen that \"the code has been changed and the task fai..."]
  signal["The model benchmark is very high."]
  operator["Why does it still happen that \"the code has been changed and the task failed\" after connecting to the real..."]
  implication["This issue only explains one mechanism: the model provides capabilities, and Harness determines how these c..."]
  thesis -->|frames| signal
  signal -->|develops| operator
  operator -->|lands in| implication

Source: View the original post