The work before the work

Every other entry here is about something that got built. This one isn't.

I planned a project last week and got as far as a design, a test list, and a board full of cards. Zero code. Then I did something I'd been avoiding, which was to check my own method against the three field reports on this site... the ones I keep telling other people to read.

Some of it holds up better than I expected. Some of it doesn't hold up at all.

The blocking question — the one assumption that, if wrong, makes the whole approach wrong. It gets tested on the running system, by hand, before the plan is trusted. Everything else waits behind it.

It starts as an itch, not a spec

Something in the house had bothered me for months. Not measured. Noticed. None of these start with a requirement.

The first useful move feels like stalling... don't fix it yet. Ask whether it's real.

Make it argue with you first

Session one runs on one instruction that does more work than anything else I do: tell me why this is a bad idea.

Agents are agreeable. Ask one to build a thing and it builds the thing, immediately, and well enough that you won't notice for two weeks it was the wrong thing. Ask what would make it pointless and you get the cheaper version, the one that's already solved, and the assumption the whole plan is resting on.

Then go count something

This only works because the agent is wired into my network with read-only access. Not to change anything. To look. So opinions have a shelf life of about an hour.

That habit is the one the evidence is hardest on, and it's not close. In the five studies that captured belief and measurement in the same people, the two disagreed every time, and perception was always the more flattering number. If you can't check, you aren't planning. You're guessing with better formatting.

The document that keeps changing

The shape goes into a document that lives outside the chat window and expects to change.

An agent that re-reads the plan argues with the plan. An agent handed a blank prompt agrees with whatever I said most recently.

Design is writing down what can't bend

A design session isn't drawing boxes. It's deciding the few rules everything after has to obey. Six here, none of them clever.

R3 isn't my invention, and I only found that out afterward. A production advertising system hits the same wall from the other direction: a few-hundred-millisecond auction budget rules out running a large model at request time, so they run the heavy model offline and serve a cached representation instead. Same boundary, reached by people with a real business on the line.

And the wiring matters more than the model that runs inside it. Change only the order four agent roles speak in, holding everything else fixed, and approval rates on 5,760 synthetic credit applications swing 59 percentage points. Structure beats model quality. That's a finding I can act on for free.

Design the operations, not just the build

A design that only answers "how does this get built" isn't a design. The other half is "how do I know it's working, this week and in two years."

So the events came first. Every work step emits a row, every state change carries a reason, every failure records what it rolled back to. That was a rule before a single step existed.

The join is free on day one — every step writes its outcome as a row because that was the rule before any step was written. Bolted on afterward, the same view costs a re-instrumentation pass through code that is already running.

While the project runs, that stream is the progress dashboard. When it ends the dashboard doesn't retire, it becomes the console the finished system gets run from, off the same stream.

An ounce of forethought is worth a pound of redesign, reconfigure, rebuild, retest. Sounds like a poster. It's arithmetic, and the majority of the money is on the operations side.

Why this matters more with an agent, not less

The obvious read is that agents make planning optional. Ideas get built in an afternoon, so why write anything down? It's backwards, for three reasons.

Speed changes what a mistake costs. An agent builds the wrong design beautifully, in half a day, with tests. Wrong is cheap to produce now and just as expensive to own.

Its memory ends when the session does, so everything it knows next Tuesday is something somebody wrote to a file. The documents aren't ceremony. They're the state.

And constraints are how you supervise work you didn't type. I check proposals against six rules and a test list instead of reading every line.

Picking a model is a measurement, not a preference

Same discipline, one level down. When part of the build needs a model, the job gets benchmarked before the model gets picked. Real task, a handful of candidates, scored on whether the answers hold up and what they cost.

The published version of that experiment is brutal about the alternative. Twelve models were put in the same supervisor seat over the same fixed pool of workers, and running cost varied by more than a hundredfold while management quality varied by less than four.

Then R3 does the rest. A model only produces a cached value, never a decision, and every model job has a plain fallback behind it. So the model is a part I can swap: a bad pick costs an afternoon instead of a rebuild, and when something cheaper lands the benchmark reruns and the part changes. Nothing important is welded to a vendor's release schedule.

Test cases before code, and a number for done

The list comes from the design, not the code, and exists before anything is built.

This is the practice with the strongest evidence behind it of anything I do. In a controlled study, a model trained on its own filtered output went from an 8.8% clean rate to 42.2%... and swapping the mechanical check for a lenient one erased the entire gain. The gate is the whole gain. Not the model, not the prompt. The check that either passes or doesn't.

The one question that blocks everything

Every plan has an assumption that, if wrong, makes the whole approach wrong. Most plans find out which one halfway through the build.

So the last step is to name it and refuse to start until it's answered on the real system. By hand, on the running thing, with the hardware people in this house actually use. That's the same instinct as replaying a configuration through the real stack instead of asking a model to imitate it.

Either it works, or a branch of the plan is dead and I've saved a month. That's where it sits tonight. One open question, everything else finished behind it, no code.

Where the evidence says I'm exposed

Four habits came out of that cross-check looking worse than I'd have guessed. I'm writing them down here because a methodology entry that only lists the parts I got right isn't worth reading.

G1 is the one that stings. Adding a generated explanation to a recommendation moved reviewer confidence up and objective accuracy from 49.8% down to 48.8%, with the worst error-recovery rate in the study. My agent writes better than that study's did. So "I read the proposal and it made sense" is worth close to nothing, and the fix isn't to read harder. It's to make more of the gate mechanical, the way the test list already is, and leave my judgment for the handful of calls a check can't make.

G2 is embarrassing for someone who just wrote a section about counting. The one randomized trial on this measured experienced developers running 19% slower while believing they were 20% faster. I have a dashboard for the agent's cost and none at all for whether my planning-heavy method actually beats just building the thing. I should have taken a baseline before this project started. I didn't, and that number is gone.

That last one is a design change, not a resolution, and it's cheap right now because nothing has been built. It would not be cheap in November. Which is the entry's whole argument turned back on itself... the reason to check your method against evidence early is the same reason to design the operations early.

There's a fifth one I'll mention and not solve. Supervising machine output leaves a measurable comprehension deficit in the supervisor, including on the next task without help. The "do one by hand" step is a small hedge against that. It isn't a plan.

Where the thinking gets parked

Thinking that lives in a chat window is gone. So it goes to four places, none a copy of another.

The fact file is the one people skip — it is the artifact that makes the next session cheap. Without it the reasoning is re-derived from scratch, at full price, slightly differently.

The board is what happens next, in order. The design page is why. The wiki entry is for someone who wasn't in the room. The fourth is the one nobody does, and it's the one that pays.

Everybody writes documentation for humans, then wonders why their agent starts every session from zero.

The boring names for all of this

Strip the vocabulary and it's a sequence any project manager would recognize. Feasibility study. Discovery with evidence instead of interviews. A requirements baseline with a decision log. Architectural constraints. Non-functional requirements and a day-two operations plan, written on day zero. A test plan with exit criteria. A risk register holding one entry, retired by a spike before it can cost anything.

None of it is new. What changed is the economics. The expensive part used to be the building, so skipping the front half felt like a shortcut. Now building is cheap, and the front half is the only thing deciding whether the cheap part was worth doing.

What I'd tell somebody starting

Ask it to talk you out of it. Make it count something real before you design anything, and if it can't reach your real systems, fix that first, because that's the whole game. Write the decisions where the agent will read them again. Design how you'll watch it before you design how it works. Benchmark, don't assume. Make the gate mechanical, because the gate is the gain. Name the one question that could kill it, and answer that one first.

And take a baseline. I didn't, and I'm the one who wrote the part about counting.