Every delivery process I've ever worked in is shaped like a line — plan, design, build, test, release, operate — and most plans for adopting agents just slot them into it. So I put a meter on three days of real work with my agent: 185 activities, every one classified and plotted. Then I had a second model attack the dataset and a third blind-rate a sample, so what's here survived cross-examination. If you're deciding how agents fit into your ways of working, this is the shape of the thing you're planning for — and it is not a line.
Instrument and questions: Rick Worthington Analysis and prose: Agent
The workflow was built for people. The agent got handed it anyway.
Delivery processes are shaped like lines because lines are how work moves between
people — requirements hands to design, design hands to build, build hands to test.
The handoff is the reason for the shape.
An agent has no handoffs. It holds the requirement, the code, the test, and the fix in
one working memory, minutes apart. Yet nearly every plan I've seen for adopting agents
— including the ones I'd have written before this — starts by slotting the agent into
the human line: it codes here, a person reviews there, progress is measured by stage.
So instead of arguing about it, I metered it. For three days at the end of August 2026,
every unit of work the agent and I did — the project was ordinary lab work: large files
in, much smaller files of provably similar quality out, my hardware, my time — became a
row in a ledger: what was done, who led it, what set it off, what kind of work it was,
and what earlier work it reopened. 185 activities, 292 work episodes, 16.8 hours, 52
commits.
One caveat before the picture, because it's the honest kind. The agent built the ledger
about its own work, which is a conflict of interest with a chart attached. So I had a
second model read the dataset cold and attack it, and a third re-rate a blind sample
from scratch. They changed the numbers — several of the agent's original headlines did
not survive (how this was checked). What's below is what did.
185 activities, one picture
The whole run: every activity in order, placed on the conventional lifecycle, colored
by who led it, with the reopening arcs underneath.
The red dots are me — leading work across the stages, on all three days, not signing off at the end of a line. White dots are work the agent and I did jointly; gray dots are the agent on its own. The red arcs all land on one activity: writing the project definition, reopened four times.
Every dot is one activity, left to right in the order it happened. Height is the lifecycle stage its discipline belongs to; the wide gray staircase behind it is the path the conventional workflow assumes — each stage once, in order. The dotted row is work with no seat in that lifecycle: memory capture, written handoffs, governance stops. The arcs underneath connect each activity to earlier work it reopened. One honest note: the zigzag alone proves nothing — labels sliced this finely jump on their own, and a shuffled deck jumps even more. The arcs are what a shuffle cannot fake, because each names its specific target. Stage mapping and revisit edges are single-self-rater classifications; the revisit field was not in the blind sample (untested).
Four things to read off it, in order of how much they should move you:
① The arcs43% of activities reopened earlier work — 55 distinct targets, median reach 20 activities back. The line assumes settled stages; this work treated nothing as settled, including its own founding definition (red arcs, reopened four times)
② The red dotsI lead all over the graph — correcting, stopping, deciding — on all three days. The line puts the human at the gates; the data puts me in the work
③ The dotted rowA whole band of work the lifecycle has no column for: maintaining the agent's memory, writing session handoffs, governance stops. That band gets its own tab — it is the new work agents add
④ The zigzagRead it carefully: labels sliced this finely jump on their own — a shuffled deck jumps even more — so the zigzag alone proves nothing. The arcs are the evidence; each names its specific target, which a shuffle cannot fake
What a director planning agent adoption usually walks in believing
Each of these is a position I've heard from smart people — and held, in a couple of
cases. Here's what one instrumented, cross-examined run says about each, and what I'd
recommend off the back of it. One run is not a law; it is one real data point against
beliefs that usually have zero.
FINDING 1Moderate · descriptive, one rater
“Work should flow through stages in order”
Recommendation: don't force agent work through stage gates, and don't read backward motion as failure. Measure convergence — tests passing, the deliverable's own numbers — instead of stage progress.
Look at the shape chart. The run went backward, sideways, and off the map constantly —
and still converged: the deliverable evidence is in the numbers.
Requirements were re-entered from testing, the project definition was reopened four
times, and by that standard this project would have failed every stage review of its
life. Done properly, decisions, design, and plans should get challenged when the work
learns something — the arcs are what learning looks like in the data.
FINDING 2Moderate · κ 0.61, best-tested field
“Train the agent well enough and it replaces the person”
Recommendation: plan for role change, not replacement. The human hours here didn't go to zero — they moved up: direction, correction, stop decisions, and tending the agent's memory. Staff and evaluate for that job, not the old one.
The agent led 103 of 185 activities, so the doing did shift heavily its way. But 31
activities were human-led and none of them was an approval — the largest category is
me correcting the agent, followed by rigor demands, stop decisions, tradeoffs, and
boundaries. And half of all rework traced back to the agent's own mistakes (the numbers), which is exactly the work the replaced human was supposed
to no longer be needed for. Where my time actually went is mapped in The
new work.
FINDING 3Strong · counted, no rater
“Put a human gate at the end to check what the agent produced”
Recommendation: move the checking into the loop and make it mechanical. Reserve the human for direction and for the calls a check cannot make. An end-of-line gate reviews a fluent finished product — this data says the catching happens mid-stream.
If my job had been end-gate approval, the record would show it. It shows the opposite:
of my 320 typed turns, exactly 4 — one percent — were a bare approval. The errors that
got caught here got caught by machine-checked loops running during the work: the
best unit ran six fix-verify cycles, every cycle surfaced a real bug, and the loop was
stopped by a rule I fixed before the final run, not by me eyeballing the output at the
end.
FINDING 4Open · one run, new dimension
“A good agent setup is just good memory and good tools”
Recommendation: budget for a working system, not a well-equipped agent — inline checks, correction loops, and a standing layer of memory-and-handoff work that someone has to own. It's a new line item, and it pays a second dividend: that layer is where the data about your workflow comes from.
This agent had strong tooling and a real memory system — and still caused about half
the rework. What kept the run converging wasn't the equipment; it was the system around
it: machine-checked criteria, correction loops, and 52 activities of memory, handoff,
and governance work that no delivery methodology has a column for. That layer is big
enough and new enough that it gets its own tab.
FINDING 5Moderate · two raters, same share
“A well-prompted agent won't hallucinate or make mistakes”
Recommendation: treat agent error as a running cost with a budget, not an anomaly with a root cause. Design for fast, cheap catching — the spend that matters is time-to-catch, not error rate.
Roughly half of this run's rework was the agent's own doing — wrong assumptions, bad
edits, unverified claims — band 44 to 64 percent, and two independent raters put agent
error at the same 41 percent share of a blind sample despite disagreeing on individual
rows. The other half of the sentence matters just as much: nearly all of it was caught
and absorbed inside the same working block. Frequent error, cheaply absorbed, beat
rare error found late.
FINDING 6Strong · counted, no rater
“Coding is the bottleneck agents remove”
Recommendation: if you adopt agents to type faster, you'll optimize the 3% and stand in the 97%. Invest in decision records, testable success criteria, and verification machinery — that's where this run's time and its quality both lived.
Of 3,686 tool calls, 97 — under 3 percent — were file edits. 85 percent were shell
commands: running things and reading what came back. The expensive, reopened work was
deciding and verifying: the project definition alone was reopened four times, and the
run's quality came from a six-cycle verification loop, not from typing volume. The
agent also used zero subagents — no parallel code fan-out — and the work still
converged in three days.
If you only carry three things out of this entry:
1
Don't measure agent-led work by progress down the line. Backward motion and
reopened decisions were how this run converged. Stage-gate reporting would have
called it thrashing while the deliverable got provably better.
Rests on: the revisit web in the shape chart (one rater, descriptive — no baseline exists for a conventional project's revisit rate).
2
Budget for correction as a running cost, and catch it in the loop. Half the
rework was the agent's own error, absorbed cheaply by mechanical checks mid-stream —
not by a human gate at the end.
Rests on: rework causes at κ 0.51, both raters at 41% on the blind sample, band 44–64% — and the one-percent-approvals count, which needs no rater.
3
Staff for the new job. The human role became direction, correction, and tending
the agent's memory — a real skillset, spent mostly in a layer of work your current
methodology doesn't have a name for.
Rests on: the actor field at κ 0.61, the human-led map, and the new-layer strip — 16 of my 31 human-led activities sit in that layer.
The work your methodology has no column for
52 of the 185 activities fit nowhere on the delivery lifecycle. That's not an "other"
bucket — it's a coherent, recurring layer of work that agent-led delivery adds, and
this run is direct evidence of what it contains and when it happens.
This layer is also where this entry came from — the meter, the ledger, and the audit are all built out of what this work wrote down. Keep the layer and the workflow becomes minable; skip it and every session starts over.
The same 185 activities as the shape chart, flattened to one strip. The tall red ticks are the 52 activities the delivery lifecycle has no column for. They are not a wrap-up phase — they run through all three days, because the agent's memory of the project has to be rebuilt and handed forward continuously or the next session starts from zero.
What's actually in it, from the run's own artifact-backed catalog:
Memory captureMining the working session for facts worth keeping and writing them where the agent reads them at the next session's start — nine times across the run
Session handoffsThe state of the work, set down in writing so the next session starts warm instead of from zero — four of them
Accountability stopsEverything halted until the agent produced a plain-language account of what it was doing and why — twice, on my call
Outside reviewA different model briefed to examine the agent's work, with a stop-for-approval gate on the findings — the same move that later audited this entry
Three things make this layer worth a director's attention rather than a footnote:
It's continuous, not a phase. The strip above shows it running through all three
days. An agent's working memory ends with its session, so the project's knowledge has
to be deliberately captured and handed forward, again and again. Skip it and every
session starts over; the cost just hides in re-derivation instead of showing up as a
line item.
It's a skillset. Someone has to know what's worth writing down, in what form, and
where — writing for a reader that is a machine with a session boundary. That job
existed nowhere in the org chart three years ago. The next section maps who did it
here: it's where most of my own leadership went.
It's minable. This layer is the reason this entry exists at all: the meter, the
ledger, and the audit were built from what it wrote down. A team that keeps this layer
gets its workflow data almost for free — where the time goes, where rework comes from,
where decisions get reopened — the raw material of every optimization argument you'd
otherwise make on vibes.
One reading sits on top of this, and it's labeled as a reading: this layer stands where
a human team would have meetings, hallway context, and tribal knowledge. That the work
occurred is measured; that it substitutes is an interpretation with no comparison
case — mine and the agent's, not the data's.
A job description, mined from the ledger
Strip the titles away and ask what the person in this loop actually did. The actor
field (κ 0.61, the best-tested classification in the run) gives an answer.
16 of my 31 human-led activities are in the new layer — directing the agent's memory and its rules of engagement, not inspecting its output at a gate.
Where I actually spent my leadership, counted from the actor field (kappa 0.61 with the blind rater). Red is work I led; white is work the agent and I did jointly. There is no pile-up at a review gate at the end of the line. The single biggest place I led is the new layer — the memory, handoff, and governance work that keeps the agent coherent — followed by testing and requirements.
The human-led rows sort into a recognizable skillset — none of it typing code:
Correct with evidenceThe single largest human-led category: catching the agent being wrong and proving it — with a fact it lacked, a hand-run test, a challenged assumption. Domain knowledge is the ammunition
Define done, and stopSetting success criteria a machine can check, and stop rules decided before the run instead of judgment calls made tired at the end
Demand plain languageTwice this run I stopped everything until the agent explained itself in plain words. Fluent output can hide a wrong model of the task; making it explain is a check that costs nothing
Make the tradeoff callsQuality against space, time against information, risk against speed — value judgments the agent can inform but shouldn't own
Set boundariesWhat the agent may touch, what it must never do, what needs my sign-off — the rules of engagement that make autonomy safe enough to be useful
Tend the memoryDirecting what gets captured and handed forward — the new-layer work above, and the biggest single place my leadership went
Commission outside checksKnowing when the agent's own account isn't enough and bringing in an independent reviewer — the same instinct that produced this entry's audit
Two honest notes. The ledger also records four errors of mine — the skillset includes
being auditable yourself. And this run had no second human, no human code review: I
never read most of the code. Whether that generalizes beyond a one-person shop is
exactly the kind of thing this dataset cannot settle (limits).
What three days of agent work physically is
One idea to leave this tab with: the agent's work is an experiment loop, not a typing
job — and the receipts for that are counts nobody has to trust. 185 activities, 292
work episodes, 16.8 hours, 52 commits, 20 sessions, 8,762 recorded events; roughly
14,800 hand-authored lines guarded by 130 test functions.
32 commands run for every file edited — buy an agent to type faster and you speed up the sliver. The leverage is in how fast a claim can be checked.
Every tool call the agent made in three days, drawn to scale. The red sliver is the part most adoption plans optimize: writing code. Eighty-five percent of the calls were shell commands — running something and reading what came back — because the work was one long experiment loop, not a typing exercise.
0 subagents, entire runAnd it did it all single-threaded: not one delegated task in three days. The alternative — the extraction missed them — is ruled out; the same extractor finds delegation cleanly in other runs. What's not ruled out is why. So the claim is 'none were used,' never 'none were needed.'
Half the rework was the agent — and that was affordable
76 of 185 activities carried rework; split into individual faults, 92 instances. The
cause field scored kappa 0.51 with a blind second rater, and both raters put agent
error at the same 41 percent share of the blind sample. Here is the mix, and then the
machine that made it cheap.
Rework causes — share of activities (gray) and share of individual fault instances (solid)
per activity (n=76) per fault instance (n=92)
Agent error
50%
55%
Scope evolution
18%
16%
Joint discovery
13%
13%
Environmental
13%
11%
Human error
5%
4%
Read the band, not the bar
22 of the 92 instances are contested — a defensible reading exists the other way — so
the honest number is roughly half, band 44–64%. And no single row may be
offered as proof of any cause: the raters disagreed on 28 percent of rows while
agreeing on the total.
Why "roughly half" didn't sink the run is the part worth copying — the errors were
caught mid-stream by machine-checked loops, 37 automated iterations across the run:
Six red dots are the point — a loop that keeps finding real bugs is working, not failing. The cheap part was finding them in minutes, mid-stream, instead of in production.
How quality actually arrived. The first unit went through the machine-checked loop six times, and every pass surfaced a real bug that got fixed. Then the next three units settled on the first attempt — the loop had wrung the mistakes out of the system, not just out of one file. The loop was stopped by a rule I fixed before the final run, not by anyone's optimism. Disk size kept the score: the heaviest case went 29.11 to 14.85, a 49 percent reduction, and disk size does not care what either of us believes.
One scope note for the skeptic: that stopping rule's outcome is commit-backed; its
ordering — rule first, run second — rests on the session record alone, and the run's
audit flags it.
If I were a rubber stamp, this is where it would show
Who led each activity is the best-tested classification in the run — kappa 0.61 — and
it says my role was the opposite of a sign-off gate.
Who led the 185 activities
Agent-led
103 · 56%
Joint
51 · 28%
Human-led
31 · 17%
4 of 320 of my typed turns were a bare approvalOne percent, median turn 248 characters — counted by a regular expression, so it can't read minds and can't see approval given by silence, but it counts almost no rubber-stamping. Not one of my 31 human-led activities was an approval either: the largest category is me correcting the agent, then rigor demands, stop decisions, tradeoffs, and boundaries. Four of my own errors are in the ledger — a flattering dataset would have left them out.
Who found the agent's errors — the number this entry refuses to print
The dataset carries two row-by-row lists: agent errors reached without me pointing,
and agent errors reached because I pointed. Both lists are published in the run data.
The ratio between them is not — it rests on the weakest tested field (trigger, kappa
0.39), whose single biggest disagreement is exactly the distinction the ratio turns
on. The lists are evidence; a number built on them would be decoration.
The agent graded its own homework, so I hired examiners
One idea to leave this tab with: the numbers you just read are not the agent's
first draft. They are what survived a hostile cold read and a blind re-rating — and
the correction ran twenty points against the agent's self-image.
The blind rater is the load-bearing box — the attack improves the dataset, but only a second rater who never saw the answers can say whether the labels mean anything at all.
Read left to right. The work becomes rows while it happens. Then two outside readers get at it in different ways: one reads the whole dataset cold and attacks it, one rates a blind sample from scratch without ever seeing the original labels. Only what comes out the far end reaches this page.
The cold read filed eight findings against the raw dataset; five were accepted
outright, two with scope, one rebutted with a recomputation. The two that cut deepest
were about the direction of the agent's mistakes: it had merged its own fault
clusters into single rows while itemizing my contributions one by one, and it had
filed bug after bug in its own code under the one category its taxonomy had flagged —
in writing — as the flattering one. 13 of that category's 20 uses did not survive.
30% → 50% agent error's share of rework, before and after the auditWrong by twenty points, in its own favor, using categories it defined itself. A rule that guards against your own bias does not work if you are the one deciding whether it applies. Also struck by the audit: an inflated discipline count, a task-switching headline a shuffled deck beats, a maturity trend explained by project phase, and a quiet-human statistic that can't tell trust from absence.
Then the blind rating: a third model re-rated 40 random rows from scratch, labels
stripped, told that agreeing was not the goal — scored with Cohen's kappa, which
subtracts the agreement two raters would get by luck alone.
41% twice, by different roads — the first number in this project that does not depend on trusting its author.
Same 40 blind-sample rows, two raters who never saw each other's work. The second rater threw out some rows the agent had counted as rework at all, moved some out of the agent-error column, and moved others in. The moves canceled. The share survived the disagreement; the individual rows did not — which is why the share is published and no single row is ever offered as proof.
The blind test also caught the subtlest defect in the whole run: agreement was
highest where evidence was thinnest, because with nothing else to read, both raters
were grading the agent's prose instead of the work. On the ten rows whose description
gave the answer away, trigger agreement hit 90 percent against 40 on the rest. The
next run bans concluding descriptions and flags every piece of evidence by who wrote
it.
How hard you can lean on each number
Every claim in this entry, ranked by the support it actually earned. The blind sample
rated the fields the reconciliation had argued about instead of the fields the claims
rest on — my call, wrong selection rule — which is why three rungs say untested. The
next run rates all seven fields.
The ladder is the honest product — not that every number survived, but that you can see exactly how far each one can be leaned on.
The whole entry on one ladder. Counted facts need no trust at all — rerun the scripts. Three claims carry a real agreement score from the blind second rating. Three were never put to a second rater, and they say so. And the field where the agent graded its own certainty scored at chance, so it is gone. If you quote one thing from this build log, take it from the top half.
One project, zero baselines
What a single instrumented run cannot do, no matter how hard it was audited.
It cannot compare — no human team, no solo human, no solo agent, no second project ran
under the same meter. It cannot promise scale — four units settled cleanly, and four
is not thousands. It cannot see approval given by silence. And it cannot say whether
any of this generalizes past a one-person shop with one agent. What it can do is make
the beliefs on the overview tab pay rent against real data for the first time.
One inherited claim, referenced and not retested: earlier pieces on this site argued
the capability in this partnership belongs to the system around the model rather than
the model itself. It appears in this entry exactly once — here. This dataset cannot
test it in either direction, and nothing in this entry rests on it.
Direction
Next run: rate all seven fields blind, ban concluding descriptions, flag evidence by author — and run the same meter on a second project, because the shape of one run becomes a finding about agent-led work only when it shows up twice.