The orchestrator was supposed to be the expensive part

That distinction sounds academic until it starts costing money — because a company that believes AI is the strategy will buy the most impressive version of it, and the most impressive version is not the one that pays.

The vision currently being sold to executives is a team of agents: one "orchestrator" directing several "workers," each sitting on top of company tools and data, doing the routine work of a department. An agent, plainly, is a program that uses a language model to do a multi-step job — read a ticket, look something up, make a change, check its own work — rather than answering one question and stopping. The orchestrator is the agent that decides which other agent or tool handles each step, and when the job is finished: the shift supervisor, not the smartest person on the floor.

This report does not argue against that picture. The evidence broadly supports it. The question it asks is narrower and more useful: if teams of agents really are the coming shape of work, what is the least expensive way to get there, and how would a company know whether it was working?

Most executives have been shown a version of this slide, or drawn one:

Nothing about it is wrong. The difficulty is that the picture is silent on four decisions, and those four decisions are where essentially all of the cost, all of the compounding, and all of the exposure are determined. Same diagram, with the silences marked:

A company can implement that first diagram four different ways and see a twenty-fold difference in running cost, with one version improving every month and another staying flat forever. The rest of this report is about those four decisions — the architecture the evidence points to is drawn in Conclusions. Four positions come out of the evidence.

Hypotheses were written down before they were checked, together with the method that would settle them; derived claims are stored with the exact query behind them, so any number can be re-run by anyone with the corpus.^[1] Of the 22 claims on the ledger, 11 were the author's own hypotheses — and most did not survive intact: 14 claims settled as mixed, one was outright refuted and became this report's headline finding, one proved untestable and says so.

• Adversarially re-analyzed, second pass — a fresh session sent to destroy this reportBefore publication, a red team with no stake in the draft attacked every finding: all 18 cited papers re-read at the methods level, every quoted number re-derived or traced to its source, the classifier re-judged with a third model family, the collection censoring tested for time correlation. No finding died. Five were reframed: the orchestrator comparison recut around the same-toolbox baseline (F7), the social-corpus levels re-anchored to the reference judge (F3), the decision-path cost ground narrowed to the frontier tier (F19), the bandit finding scoped to offline experiments (F20), and one finding's numbers replaced because their original query had never been persisted (F18).^[6]
• Caught by that red team: the trust tile itself was wrongThe "sources read at full length" count was rendering the instrument's whole library — including 3,154 video transcripts belonging to a different sensor — instead of the ~65 documents actually read for this report. Provenance inflation of roughly fifty-fold, on the page whose only job is trust, live until the red team read the query behind the tile. Fixed by scoping the count; kept here because a trust page that has been wrong should say so.^[6]
• Caught by that red team: the AI disclosure was falseAn earlier revision of this tab said "no model scores, ranks or classifies any evidence." Untrue: the social-corpus composition is produced by a model classifying posts. The disclosure below now states what actually happens and how the classifier's error was measured, instead of denying it exists.^[6]
• Killed: the flagship experimentThe plan was to rent a GPU and test whether filtering training data through a verification gate beats training on everything. A corpus search found the decisive version already published, with controls that would not have been designed here — including one that swapped only the gate and watched the entire gain vanish. The experiment was canceled rather than run, and the finding now cites someone else.^[14]
• Killed: a novelty claim"Provisioning an agent as a scoped bundle of capability and permission" was held as the novel half of the argument. A sweep found a July 2026 paper opening with the same problem statement, with a released dataset and a stronger principle. The novelty claim was dropped.^[19]
• Reversed: a design rule the author believed"Weights hold behavior, files hold facts" was proposed as a clean split. A controlled comparison found the opposite on accuracy — tuning beat retrieval by nearly seven points while retrieval added nothing. The rule survives as a risk argument, not a capability one.^[18]
• Reversed: the architectural assumptionThat a loop should be supervised by a frontier model was contradicted by a measured result in the other direction. Recorded as refuted on the ledger, and it became the headline of this report.^[10]
• Null check: is routing research just tracking the archive's growth?Routing paper counts rose steeply — but the archive itself nearly tripled over the same window. Normalizing to share of corpus was built so that "no real trend" was an available answer. The share rose anyway, roughly fifteen-fold. The keyword remains uncleaned, so the shape is published and the level is not.^[2]
• Null check: is the filings trend just a change in who files?The rise in control-layer language was re-run on a balanced panel — only companies filing in all four years — so that composition change could produce a flat line if it were the true cause. The trend got stronger, not weaker.^[3]
• Failed, then rebuilt: the practitioner-sentiment sensorA social corpus of 4,328 posts was expected to show what developers say about these tools. It contained zero matching posts — it had been collected scoped to named projects, so the silence was an instrument gap, not a result. A topic-scoped collector was written and run: 18,620 posts, 15 terms, no errors.^[4]
• Caught: keyword precision was NOT flatThe new corpus was audited by month before any finding rested on it, on the assumption that flat precision would license the trend. Precision was not flat — it rose from 67.5% to 75.5% as the vocabulary settled, which would have manufactured part of any trend measured on raw counts. Every composition figure was recomputed within confirmed on-topic posts.^[5]
• Failed: a trend read off a sample too small to carry itAn early pass called advocacy "flat" from the 60-per-month reference sample. At that size the noise band is about ±6 points — wider than every trend in the finding. Re-measured at 400–800 posts per month, advocacy rises modestly. The error was in reading a trend off a sample sized for accuracy, not for power.^[5]
• Measured the judge before trusting it — then got caught quoting the wrong judge anywayThe reference model was run twice over the same 60 posts at temperature zero to establish a ceiling — 98% self-agreement on topic, 97% on who was speaking. Candidates were scored against that ceiling, and the cheap winner was weak on identifying the speaker (68% against a 97% ceiling). Revision 2 then published the cheap model's speaker levels anyway, while its provenance text claimed the reference model's. The red team caught it: the cheap model runs 6–10 points hot on the practitioner share in both years. This revision publishes the reference model's stratified-sample levels, which is what this paragraph should have described the first time.^[5][6]
• Failed: the assumption that a prior bake-off transfersThe model that won an earlier classification job on corporate filings came fourth here, and the winner on this task was a different one entirely. Neither price tier nor past performance predicted accuracy. Whole-corpus classification cost $0.31.^[5]
• Defeated by its own cost control: the share measurementFive control terms were collected specifically so topic volume could be read as a share of conversation rather than a raw count. Collection was then capped at four queries per month to fit a time budget — and that cap pins the busiest terms at a ceiling of roughly 160 posts per month. Every control term and the three busiest topic terms sit at that ceiling, so both sides of the ratio are clipped and the share cannot be computed. The controls were collected and are unusable for their stated purpose. Composition measures, being ratios within a month, are unaffected — and the red team verified the 22 censored windows split 12/10 across the two years, too small and too balanced to manufacture the composition trend.^[4][6]
• Killed by the prior red team — and nearly missed twiceA within-company panel finding (F22) was carried in from earlier work, re-run, and confirmed to reproduce numerically. It was already dead: an adversarial re-analysis had shown the qualifying companies are mostly sellers of the software in question, that two are outright false positives, and that a placebo panel puts the persistence rate between on-premise and blockchain. Re-running the query verified the arithmetic and missed the critique entirely. The finding is kept in its broken state rather than deleted — and the second red team then found the same dead reading reused, un-caveated, in a different claim's evidence on the ledger. It is annotated there now.^[27][6]
• Excluded: a widely quoted industry statisticA frequently cited figure on vendor purchases outperforming internal builds was ruled inadmissible by the prior red team — a small interview-and-survey base, publicly criticized methodology, and unpublished underlying data. It is not cited anywhere in this report, and F23 says the evidence base is absent rather than quoting it.^[27]
• Reversed: the claim this sensor was built to test"Practitioner sentiment is turning toward local-first" did not survive its own data. Posts arguing against the practice sit at 1–3% all window, so there was no opposition to turn from — the corpus is self-selected and structurally cannot answer the question asked. It answers a better one: who is doing the talking.^[4]
• Found in the collector: silent incompletenessThe first version of the new collector resumed per search term rather than per term-and-month. A run stopped part-way left a term looking finished, and the missing months would have read downstream as "nobody posted about this after November." Caught by checking window coverage rather than row counts, and fixed before any finding rested on it.^[4]

Data lives in a column-store database on a home server and is never pruned; history is the point of the instrument. Sources are fetched by purpose-built collectors that record every fetch, including the ones that failed, so a gap is visible as a gap. Full-text sources are stored as immutable originals with a hash, and the extracted text is regenerable from them.^[1]

Effort is counted across the whole project, not the drafting. It covers choosing the questions, standing up the collectors, the validation passes, the analyses that were killed, and the adversarial reviews — quoting only the writing sessions would describe a research program as though it were a blog post. Counts are programmatic from session transcripts, human prompts only, tool output and sub-agent turns excluded; transcript retention covers the instrument's whole life, so the count is complete rather than sampled.^[29]

What it cost, precisely. The classifier ceiling runs, the model bake-off, the whole-corpus classification and the stratified reference sample: $5.52. The adversarial re-judge of 672 posts by a third model family: $1.90. The corporate-filings classification behind F2 and F22, run for the prior report and reused here: $10.61. Every figure is from the gateway's spend log, not estimated.^[29] Hardware is a home server already owned; its marginal cost is the electricity — which, per Finding 4, is the cheap and misleading number.

Platforms. Python collectors feeding a ClickHouse analytical database, queried in SQL. Drafting and both adversarial passes ran as Claude Code sessions against that database. The report is hand-authored MDX.

Models, by role. Claude Opus 5 and Claude Fable 5 (Anthropic) did research direction, drafting, and the red-team passes, under human instruction. Claude Opus 5 also served as the classification reference judge — measured against itself before use. deepseek-v4-flash (open-weight, via API) classified the 18,620-post corpus for $0.31, with its published levels replaced by the reference judge's where its error ran hot. gemini-3.5-flash re-judged the calibration sample during the adversarial pass, blind to the other two. No model generated any number cited as a finding without a measured error bar attached.

The prose of this report was written by an AI (Claude, Anthropic), working from the ledger and the sources, under human direction. The human owns the questions, the instrument, and every editorial judgment, and ordered both adversarial passes.

The numbers come from three places, and the report says which everywhere it matters. Most are deterministic database queries over a stored corpus. Some are quoted from named source documents with their own methodology. One finding's numbers (F3) are produced by a model classifying text — disclosed as method: the classifier was measured against a reference judge before use, its published levels are the reference model's, and an adversarial pass re-judged the same sample with a third model family and got the same answer. No model summarized evidence into a finding or graded its own work.

What that means for the reader: the framing and the sentences are machine-drafted and human-directed, while the quantities are auditable and re-runnable — and where a number could not be produced honestly, the report says it is missing instead of supplying one. An earlier revision of this disclosure denied the classifier existed; the red team caught it, and the correction is listed above rather than papered over.

• Rev 1 · Aug 15, 2026Organized around the instrument's own question ("is local-first cheaper?"), presented as a sequence of hypotheses checked. Staged privately, never published.
• Rev 2 · Aug 15, 2026Reframed to start from the business problem, with every term explained before use and one narrative through the evidence; the claim-by-claim accounting moved to Provenance. Two claims settled since Revision 1, both as mixed, and a third reframed from a sentiment turn — which its own data refuted — to a shift in who is talking. Staged privately, never published.
• Rev 3 · Aug 15, 2026 — the adversarial pass, and publicationA fresh red-team session attacked every finding before release. No finding died; five were reframed (F3's levels re-anchored to the reference judge, F7 recut around the same-toolbox comparison, F18's numbers replaced with a persisted query, F19's cost ground narrowed to the frontier tier, F20 scoped to offline experiments). Two provenance defects on this tab — an inflated source count and a false AI-disclosure sentence — were found, fixed, and documented above. Effort and cost figures were instrumented and added. This is the first published revision.

A decision engine, plainly: the system that picks what to show, offer or approve for a specific customer in a specific moment — which options were eligible, which were withheld and why, what was recommended, in what order. It runs inside a page load or an ad auction, and it logs every decision.

One machine, many specialists. The instinct behind "ten agents" is ten deployments, and that instinct is what makes the idea sound unaffordable. It is not how the serving layer works. A single base model stays loaded, and each team's specialization is a small adapter file selected per request — measured at eight running in parallel on one base (F9). The frontier model stays in the picture as the exception rather than the default (F5, F7).

This is available as a hosted product as well as something to run on owned hardware — the adapter-serving mechanism is the same either way, which is what makes Step 1 below possible without buying anything (F6).

The loop that makes it compound. The previous diagram is a cost structure. It does not, by itself, improve. What turns a deployment into an asset is the harness around it: routing decided per step rather than per request, a mechanical check on whether the work functioned, and the verified results kept as training data for the next adapter.

Two things about this diagram carry the weight of the report. The gate is not a quality-assurance nicety — remove it and the loop does not merely weaken; training on unfiltered output measured worse than not training at all (F10). And the trace log is the compounding asset, not the model. It exists only if the harness belongs to the company, which is also why the shrinking exportability of frontier reasoning traces is a supply risk worth watching (F11, F12).

Step 1 · Take the free third before buying anything. Move routine, high-volume work to hosted open models and route the exceptions upward. This needs no hardware, no capital request and no data-center conversation. The measured saving is about a third at matched quality (F5), and the price gap between tiers is roughly twelvefold at the median (F6). A company that does only this step has captured the part of the argument that is real, bankable and reversible.

Step 2 · Find the gate before designing the loop. The single highest-leverage question in the whole program: for the work in question, what is the automatic test that says it functioned? Tests pass or fail. A ticket is resolved or reopened. A record validates or does not. Where such a test exists, a self-improving loop has a mechanism and the measured gains are large. Where it does not, the loop has nothing to filter on, and the evidence says training on unfiltered output makes things actively worse than doing nothing (F10). Choose the pilot domain by where the gate is strongest — not by where the enthusiasm is.

Step 3 · Run the pilot with one team, and instrument it before it starts. A workable first pilot is a single team with a repetitive, checkable workload, and four things captured from day one: a baseline taken before anything is deployed (productivity is unmeasured in the literature, F16 — the only credible number a company will ever have is its own, and it cannot be reconstructed after the fact); cost per completed task, not cost per million tokens; the gate's pass rate over time (F10); and the escalation rate to expensive models (F5). Record every step's decision and outcome, not just inputs and final answers — that record is the training data for everything that comes later (F11), and a system that did not log it has thrown the compounding away.

Step 4 · Scope permissions per task, from the first day. Grant an agent access to what the task in front of it needs, not to what its role might ever need (F15). This is far cheaper to build in at pilot scale than to retrofit across a company, and the standing over-privilege — not the model's behavior — is the exposure.

Step 5 · Only then consider owning hardware. The large cost multiples require roughly $45,000 of equipment kept genuinely busy, plus the engineering to run it (F4) — and the depreciation rate that would justify or sink that purchase is not yet measurable (F17). A company with a measured, high, steady volume from Steps 1–3 can evaluate this on its own numbers. A company without that evidence is buying a thesis.

Step 6 · Take this into a platform renewal as a narrower question. If an existing decisioning platform contract is up and the number causes sticker shock, the evidence here does not support treating agents as a replacement bid. Such a contract buys at least two things — an auditable, low-latency runtime, and the tooling to optimize it — and everything in this report bears on the second (F19, F23). The useful question at renewal is narrower and more answerable: how much of what is being paid for is optimization tooling, and does that half still earn its share now that agentic analysis costs cents? If the engine is not being actively optimized today, that half of the contract's value is already unrealized, and the comparison is against zero rather than against the vendor.

• Any promise of frontier quality from cheap models via smart routing. Twenty-one methods converge well below a perfect router and the gap does not close with effort (F13). Routing buys cost at acceptable quality, never parity.
• Any productivity multiple. Nothing in the evidence base supports one (F16).
• Any claim that the supervisor must be the most capable model. The measured result runs the other way, and this default is expensive (F7).
• Any plan to teach the model company policies by training it on them. That is the configuration that produces fluent, confident, invented specifics (F14).
• Vendor-authored sufficiency arguments, including the one this report cites and grades as a position rather than a result (F8).
• Any architecture whose critical input is another company's optional output (F12).
• Any proposal to have agents make the decisions at request time. Latency and reproducibility each rule it out on their own; at volume, frontier-tier cost does too (F19).
• Any claim that building beats buying here, or the reverse. No outcome record has been published for this category (F23).
• Any plan to test changes on model-simulated customers. Use replay against real logged decisions instead (F21).

• Replaced · Rev 3F18's numbers. The original bars came from a drafting-session query whose term list was never persisted — a violation of this report's own provenance promise, caught by the red team. The finding survives with re-derived numbers whose query is stored on the ledger; the shape is unchanged, the levels moved.
• Reframed · Rev 3F3's published levels (they were the cheap classifier's, 6–10 points hot — now the reference judge's), F7's headline comparison (now the same-toolbox baseline), F19's cost ground (now scoped to the frontier tier), F20's scope (now offline experiments, explicitly). No finding was withdrawn; each reframe is described inside its finding.
• Removed · Rev 2The claim-by-claim structure of Revision 1. No finding was withdrawn; the accounting moved to Provenance, where the ledger belongs.
• Standing limit · the social corpus is self-selectedF3 measures who is talking, never whether anyone changed their mind. Collection capped bisection at four queries per month: 532 windows hit the cap and 22 remained censored at single-day granularity, which loses a window's oldest posts and therefore biases against finding an early rise. The red team verified the censored windows are balanced across years.
• Standing limit · volume trends are not usableTwo terms had their earliest months collected before the per-month cap was introduced, so those months are denser than later ones by collection method rather than by activity. Nothing in this report rests on a count of posts over time; F3 uses only ratios computed within a month, which that difference does not affect.
• Standing limit · keyword measurementsThe corpus trends (F1, F2, F18) rest on uncleaned keyword matching with no hand-labeled precision audit. Shapes are published; levels are not.
• Standing limit · single-source findingsF7, F9 and F10 each rest on one published result. They are strong results with real controls; they have not been independently replicated; and each finding now names its domain, its authors' stake, and what does not travel beyond it.
• Untestable · the skill barrierWhether this pipeline is accessible to ordinary IT staff cannot be settled from literature — it is a claim about how hard work feels to a non-specialist. The adjacent evidence cuts against easy optimism: the measured adapter configuration needed compiler-level debugging (F9).

What this report is not claiming. Not that capital is flowing to this layer — only that disclosure language is. Not that hosted models beat owned hardware — the depreciation rate that would settle it is unmeasured. Not that small models are sufficient for most work — that remains a vendor's argued position. Not that any of this raises productivity, which nobody has shown. And not that the cost figures here are capability figures; price is not quality, and no benchmark in this report says otherwise.

1. DataThe claim ledger: 22 claims on the topic, each stored with its origin (external / the author's hypothesis / derived from reading), the method recorded with the claim, findings and conclusion kept as separate fields, refuted claims retained. The staging page renders its counts live from this ledger. Full-text reading shelf: 65 sources (40 papers, 25 web documents) stored as immutable originals with content hashes.
2. DataResearch corpus: 290,240 arXiv abstracts, January 2023 – August 2026, share-of-corpus routing measurement with the archive-growth null check (11,303 → 32,206 papers/quarter). Keyword uncleaned; shape published, level not.
3. DataCorporate filings corpus: 29,287 extracted passages, 2023-Q1 – 2026-Q3, balanced-panel restriction (companies filing in all four years), like-quarter comparison. Q1 2023 0.6% → Q1 2026 27.0%.
4. DataSocial corpus: 18,620 posts from 12,608 accounts, Sep 2025 – Aug 2026, 15 search terms (10 topic + 5 control), 1,081 queries, zero errors, resumable per term-and-month. Cap-censoring accounting: 532 windows capped, 22 censored at day granularity.
5. DataClassifier calibration: reference judge measured against itself first (98% on-topic / 97% speaker self-agreement, n=60, temperature 0); candidate bake-off scored against that ceiling; whole-corpus classification $0.31; monthly precision audit (67.5% → 75.5%, drift removed by conditioning on confirmed on-topic posts); published composition levels from the reference judge's 672-post stratified sample.
6. DataAdversarial re-analysis, second pass, Aug 15, 2026: a fresh session with no stake in the draft. Methods-level re-read of all 18 cited papers; every quoted number re-derived or traced to source; third-model re-judge of the 672-post calibration sample ($1.90, results within 1–3 points of the reference); censoring-by-month test; two provenance defects found on the trust page itself. Verdict file and re-judge artifacts stored with the analysis (REDTEAM.md, rejudge_x.py, rejudge_gemini.json).
7. DataInference price sensor: daily snapshots of a 413-model hosted catalog (prices, capability flags), begun mid-August 2026. Vendors overwrite yesterday's prices, so the series exists only because collection started before the question needed it. Cost projections in F19 are arithmetic over this catalog's published prices.
8. ExtPractitioner cost write-up (public forum, 2026): the $0.015/M-token marginal electricity figure with the author's own statement that it excludes hardware; ~$45,000 build cost from the public discussion beneath it. Graded: honest primary source, narrow scope.
9. ExtLLMRouterBench (arXiv:2601.07206): 400K+ instances, 21 datasets, 33 models. Up to 31.7% cost reduction at matched best-single-model performance; ~4% accuracy gain; named binary routers fail to trade cost for savings; pool-size diminishing returns. Graded: strong for the cost figure. Verified at methods level by the red team.
10. ExtToolOrchestra (arXiv:2511.21689, vendor research lab): 8B orchestrator, RL-trained on outcome/efficiency/preference rewards. 37.1% vs 21.2% (frontier model prompted to orchestrate the same expanded toolbox) and 35.1% (frontier model with basic tools only) on the text-only subset of a hard reasoning exam; +2.3/+2.5 points on two further benchmarks at ~30% cost. Graded: strong result, vendor-authored, judge circularity noted (the frontier model also scores the outcome reward); same research group as c11 — counted as one voice with it.
11. ExtSmall-model sufficiency position paper (arXiv:2506.02153, vendor research, v2 Sep 2025): explicitly a position, not a benchmark. Graded: contested; used only as framing, never as evidence of sufficiency. Same research group as c10.
12. ExtMulti-adapter serving report (inference project with a cloud provider, Feb 2026): GPT-OSS 20B base, 8 adapters in parallel, LoRA rank 32, 1,600 input / 600 output tokens, vLLM 0.15.0 — 144 OTPS / 135 ms TTFT, 171 / 124 tuned; the 10× TTFT compiler-recompilation regression and its one-hint fix. Graded: strong for the configuration, vendor-published. All numbers verified verbatim by the red team.
13. ExtAdapter-store serving documentation (a second inference stack): the same per-request adapter selection mechanism, documented independently of c12.
14. Ext"The Verifier is the Curriculum" (arXiv:2607.09709, academic — Science Tokyo / Zhejiang / NUS): execution-gated self-distillation on video-game project generation, Qwen3-14B + LoRA. 8.8% → 42.2% over three gated rounds (all significant); gold-duplication control regresses to 5.6% (p=0.019); gate-swap control (lenient check passing 99.9%) erases the entire gain (p=1e-3); coverage 18/25 → 25/25. Graded: strong; the decisive control is what earns it; single model, single benchmark, N=25 — the design rule travels, the effect size does not.
15. ExtHarness-native routing architecture (arXiv:2607.11399, open-sourced Jul 2026): architectural argument, not a benchmark. Graded: moderate; used for framing.
16. ExtPublic issue tracker of a frontier provider's tooling, counted Aug 15, 2026: 18 open issues on reasoning summaries, one documenting encrypted-by-design reasoning content. Graded: observational; establishes the pattern, not the cause; count not independently re-verified.
17. Ext"The Routing Plateau" (arXiv:2606.07587, university + industry): 21 methods, 5 benchmarks, one unified setup. Best router trails oracle by 10–30 points; top-15 spread 0.23 points on one benchmark; kNN top-2 on all five; plateau persists cost-aware; authors' remedy closes 14.6% of the gap. Graded: strong; the central negative result of this report. Verified at methods level.
18. ExtFine-tuning and hallucination mechanism study (arXiv:2604.15574 — interference among overlapping representations; freezing mitigation, which suppresses new-fact learning and is scoped by its authors to settings where that is acceptable); controlled tuning-vs-retrieval comparison (arXiv:2604.23801 — +6.8 points at 4B scale, retrieval null); LoRA configuration ablation (arXiv:2605.28222). Graded: mixed — they disagree, and the disagreement is reported rather than resolved.
19. ExtDynamic capability scoping (arXiv:2607.22445, independent researcher, workshop): 600-prompt dataset over a 15-permission taxonomy, κ = 0.917/0.967 on a reviewed sample; ceiling violations 46 → 3 by iterating dataset against policy. Graded: strong prior art on the framing; the architecture is proposed, not implemented.
20. DataDecision-support framing share: re-runnable corpus query stored on the ledger (claim decision-support-framing-share) with its full term lists. 2023: 0.68% of all papers / 10.3% of agent papers; 2026: 1.46% / 7.5%. Replaces Revision 2's unpersisted drafting-session numbers.
21. ExtLong-history user transformers for real-time ad ranking (arXiv:2607.14331, production advertising system): the few-hundred-millisecond constraint quoted verbatim; offline encoder + cached representation + lightweight runtime model; 72–80% offline quality retention of an unservable full-history model; deployed via live A/B. Graded: strong, production. Shares an organization and an author with c23 — counted as one production culture with it.
22. ExtVariance-components decomposition of LLM non-determinism (arXiv:2607.13304): 12,933 responses, fully crossed; four sources — resampling, prompt phrasing, model identity, language. Graded: moderate for this use — the task is brand-sentiment measurement, so it is cited for the structure of the variance, not the magnitudes.
23. ExtGryphon-v2 generate-and-rank recommender (arXiv:2608.06213): replaces a 15-generator production cascade; distills a training-only teacher into the serving module explicitly to avoid a second model on the serving path; deployed via online A/B. Graded: strong, production. Shares an organization and an author with c21.
24. ExtRecHarness bandit-routed agentic harness (arXiv:2607.29241): Thompson-style routing over offline experiment directions; ablation vs model-chosen directions (improving-trial hit rate 47.9% vs 21.7%); one live element tested a single already-chosen candidate. Graded: strong for the offline split; not evidence about live-traffic exploration, and cited accordingly.
25. ExtOff-policy evaluation literature (964 papers, 2023–2026, from the research corpus); logging-policy design and the reward–coverage tradeoff (arXiv:2605.15108); replay-to-launch-readiness framework (arXiv:2605.12840 — offline improvement is evidence for a live test, not a substitute; benchmark-log study). Graded: mature method literature.
26. ExtFour studies refuting LLM-simulated survey respondents: cross-domain benchmark vs non-LLM baselines (arXiv:2607.26348 — no model beats the strongest baseline at the individual level; its own discussion names offer personalization as the decision at risk); distribution-first population simulation (arXiv:2607.18310 — 85% collapse on 2,414 respondents; a measured mitigation exists but over-disperses; includes a small booking-task choice probe showing the same collapse); urban-publics behavioral replication (arXiv:2607.27100 — 1 of 8 models met the authors' prespecified criterion against 843 respondents); belief-update simulation (arXiv:2607.28347 — all 6 models fail from self-generated starting points; 4 of 6 pass given true starting points, so the failure is the cold-start regime this use case requires). All four verified at methods level by the red team.
27. DataPrior adversarial re-analysis (Aug 11, 2026, of the preceding report): seller-cohort and false-positive audit of the 16-company decisioning panel; placebo persistence panel (mainframe 100%, on-premise 91.7%, decisioning 87.5%, blockchain 77.3%, digital transformation 72.0%, metaverse 42.9%); exclusion of a widely quoted vendor-outcome statistic as inadmissible. This report inherits those kills; the second red team verified the inherited rewrite reproduces the prior verdict exactly and annotated the one place on the ledger where the dead reading had been reused without it.
28. ExtBuild-vs-buy decision framework (arXiv:2606.29816): cited from its abstract, where the quoted cold-start phrase appears verbatim; not on this report's full-text shelf. Graded: honest framework, concedes the missing evidence base.
29. DataEffort and cost accounting: human prompts counted programmatically from Claude Code session transcripts (tool results and sub-agent turns excluded), 425 prompts across 30 sessions, Aug 3–15, 2026; transcript retention covers the instrument's entire life, so the count is complete rather than sampled. Model spend from the LiteLLM gateway's per-request log: $5.52 report evidence work, $1.90 adversarial re-judge, $10.61 prior-report filings classification, $19.10 instrument lifetime.