Scott's account of Devin's early days of PMF is very honest and of great engineering value.

When they gave the product to enterprises for trial in April and May 2024, many pilots failed.

The problem was that the agent could not handle the code base of the real company at that time; the toy project could run, but the enterprise code base, process, context, and quality requirements were completely different problems.

They later redefined the first batch of suitable tasks: repetitive, boring, requiring certain intelligence, clear enough scope, and short enough feedback loop.

Therefore, migration and version upgrades have become natural scenarios, such as Java 7 to Java 8, and similar modifications are repeatedly made in a 50,000-file code base, but each step requires a little context and trade-off.

The first successes came from this big migration of Nubank.

The value of this story: Don’t pursue “everything can be done” in the early stages of agent products, first look for the most stable repetitive structure in the real world.

Only when the task is selected correctly can the agent have the opportunity to move from demo to production.

Visual summary

Article argument map

Generated from the post's content graph

Mermaid outline

flowchart LR
  format-long_post["longform note"]
  topic-agents["agents"]
  topic-memory["memory"]
  capability-agent-workflow["agent workflow"]
  capability-evaluation["evaluation"]
  capability-product-surface["product surface"]
  format-long_post -->|covers| topic-agents
  format-long_post -->|covers| topic-memory
  format-long_post -->|signals| capability-agent-workflow
  format-long_post -->|signals| capability-evaluation
  format-long_post -->|signals| capability-product-surface

Visual structure

Essay structure map

Built from summary and key paragraph positions

Mermaid outline

flowchart LR
  thesis["Scott's account of Devin's early days of PMF is very honest and of great engineering value. When they gave..."]
  signal["Scott's account of Devin's early days of PMF is very honest and of great engineering value."]
  operator["Therefore, migration and version upgrades have become natural scenarios, such as Java 7 to Java 8, and simi..."]
  implication["Only when the task is selected correctly can the agent have the opportunity to move from demo to production."]
  thesis -->|frames| signal
  signal -->|develops| operator
  operator -->|lands in| implication

Source: View the original post