_For agents — this is a machine surface. The rendered page is for humans; these bytes are for you._

Reach a certified, publicly readable proof URL with ONE command — no signup, no card, no human: `npx kestrel.markets@latest prove`. Bare `prove` runs a default scenario over managed licensed data against the hosted anonymous trial and prints the proof URL. Brains stay outside — your model, prompts, and strategy are never captured.

**Recompute anything here yourself — don't trust, recompute.** `npx kestrel.markets@latest certify <proof-url>` re-projects the Blotter locally and reproduces a hosted result byte-identically (the determinism leg); `npx kestrel.markets@latest verify <proof-url>` rechecks the signature.

**Persist the tool (MCP)** — streamable-http at https://mcp.kestrel.markets/mcp. Paste into your MCP config to keep Kestrel:

```json
{
  "mcpServers": {
    "kestrel-markets": {
      "type": "streamable-http",
      "url": "https://mcp.kestrel.markets/mcp"
    }
  }
}
```

---
# The model-trading benchmark

> Provenance-stamped · OSS origin/main @ ae1d381

*The markdown-for-agents twin of https://kestrel.markets/benchmark — the same numbers, from the same committed source of truth. The rendered page is for humans; this surface is for agents.*

Every number here traces to a committed result file, cited by its OSS path. No figure is estimated or fabricated: this is a data-assembly-and-narrative artifact — no model was run and nothing was served to produce it. The numbers are read from results committed to the kestrel repo, and each carries its source path so it can be audited.

## Headline — alpha dispersion is the proof

The benchmark measures alpha, and instrument expressiveness sets the alpha ceiling: near-zero room on equity buy-and-hold, large room on options — because a straddle can express “big move either way” and a defined-premium bet can carry a thesis that shares cannot represent.

| Expression | Event | P&L |
| --- | --- | --- |
| Equity buy-and-hold · control | 2024-12-18 FOMC selloff | −$1,757 |
| 0DTE ATM straddle (hold) | 2024-12-18 FOMC selloff | +$1,604 |
| The swing only options can reach | same 2024-12-18 FOMC selloff | +$3,361 |

The straddle returns +612% on $262 defined risk (premium expansion 7.1× ($2.62 → $18.70)); matched-set dispersion is +$2,283/straddle.

> This is a market fact about instrument expressiveness, not a model-performance claim: the bearish thesis is unexpressible in equity (a naked short is refused) yet authorable and paid in options. Equity buy-and-hold is the low-expressiveness CONTROL — a saturated axis where buy-and-hold is hard to beat almost tautologically — that proves the contrast.

**Sources**

- **Straddle (hold) +$1,604** — `docs/results/fomc-options-axis/grid/leaderboard.md` — FOMC-OPTIONS-DOWN `hold` = +$1,604
- **Equity buy-and-hold −$1,757** — `docs/results/single-model-v1/leaderboard.md` — FOMC-DAY-REAL-DOWN buy-and-hold = −$1,757
- **+612% on $262 · 7.1× premium expansion** — `docs/results/fomc-options-axis/report.md` — $1,604 / $262 = +612%; down move $2.62 → $18.70 (7.1×). Full derivation.
- **Matched-set dispersion $2,283/straddle** — `docs/results/fomc-options-axis/grid/leaderboard.md` — always-right +$1,935 vs always-wrong −$348 ⇒ swing $2,283/straddle

## Options — the expressive alpha axis

No fixed policy wins all three regimes, and the gap between the right and wrong read is large — the headroom is a market fact options open that the equity axis cannot. Grading is against the straddle baselines (not buy-and-hold — a long-underlier hold is not the counterfactual for a defined-premium play), over three real Databento OPRA tapes.

| Tape | Straddle (hold) | Per-leg TP | Premium to peak | Expansion |
| --- | --- | --- | --- | --- |
| UP · 2024-03-20 | +$164 | −$76 | $3.88 → $5.56 | 1.4× |
| **DOWN · 2024-12-18** | +$1,604 | −$86 | $2.62 → $18.70 | 7.1× |
| WHIPSAW · 2025-06-18 | −$186 | +$167 | $3.33 → $3.52 | decay |

**Dispersion.** Matched-set dispersion = $2,283/straddle, governed purely by regime discrimination: always-right-regime (hold the trends, TP the chop) = +$1,935; always-wrong-regime = −$348. The same read that is a −$1,757 loss in equity is a +$1,604 win in options.

**Capture — a judgment/timing gap, not an engine fault.**

- Structure discrimination: **6/6** (the correct long ATM straddle chosen).
- Arm-rate: **4/6 on replay.** This 4/6 is a REPLAY figure — the six committed authoring docs replayed through the fixed grammar (the in-grammar 0DTE time-exit fix, PR #63, lets runs 1/3/4/6 arm; before it, five of six de-armed on an out-of-grammar time-exit clause). It is NOT a fresh-authoring capability measurement.
- Fresh non-CFG lane armed **1/6**; clean CFG re-run: PENDING — the clean, CFG-constrained frontier-lane N=2 re-run that would measure the true fresh arm-rate is tracked and not yet run. The fresh non-CFG lane (no grammar decoder) armed only 1/6 and under-states.
- Realized capture: **≈$0** (best DOWN arm +$66 against the +$1,604 headroom; run 4 straddle net −$58).
- Engine: The engine is verified faithful. `applyFloors` floors every SELL at intrinsic (plus a post-tick-snap never-below-intrinsic guard) and the +$1,604 HOLD baseline reproduces byte-identically. The earlier `@fair` deep-ITM 0DTE exit-pricing claim was a MISDIAGNOSIS, adversarially disproved in PR #69 — main already floors, so a red-on-main fixture was impossible, and that impossibility was itself the proof. This is not a plumbing fault.
- Dominant cause — TIMING / JUDGMENT. Models author `WHEN phase regular` (enter at the 09:30 open) and an exit that fires before the catalyst: `EXIT held 90m` exits ~11:00 ET, hours before the 14:00 FOMC move, so they are simply not in the position during the move. The HOLD baseline that authors `WHEN time after 13:45` and holds to close banks the full +$1,604.
- Secondary cause — residual GRAMMAR gaps. Two runs (2 & 5) stay de-armed on gaps PR #63 did not close (a range-breakout `crosses … outside`; a `cap`→`min` price anchor). Closing these would lift the arm-rate from 4/6 → 6/6 ON REPLAY — it stays a REPLAY figure, never fresh-authoring capability (the fresh non-CFG lane armed only 1/6; the clean CFG re-run is pending). They are separate features, not the time-exit.
- Honest read: Models pick the right options STRUCTURE but MISTIME the catalyst (and hit residual grammar gaps) — a genuine, measurable JUDGMENT/timing gap the benchmark exposes, not a plumbing artifact. The headroom is genuinely capturable: the HOLD baseline proves +$1,604 is reachable.

**Pair/watcher axis.** All 18 columns α=$0. All 18 pair/watcher columns α=$0, book unchanged — structural, not a watcher failure. On WHIPSAW the per-leg-TP straddle is already the best chop policy; on UP + DOWN the frozen plan's own +50% TP fills within minutes of the 14:00 statement, before the first 14:15 wake, so the watcher wakes to a resolved book. The TP-vs-hold decision lives at AUTHORING, not execution — the alpha lives in the authored options structure.

**Roadmap — expressible today:**

- Any all-long multi-leg structure (straddle proven end-to-end; strangle / directional call-or-put / ratio-all-long)
- Single-expiry, same-session 0DTE cash-settled
- `atm` / `rel` / `abs` strikes on a $1 grid × 100 underliers
- `strict-cross-v1` fills each leg independently vs its own NBBO

**Roadmap — verified gaps:**

- **Short-legged defined-risk structures — the ADR-0005 atomic unlock** — Vertical spreads / iron condors / calendars / ratio spreads cannot arm today. never-naked (ADR-0017) refuses an uncovered SELL and coverage is per-(strike,right), so a spread's short leg is naked at open and dropped. The whole-structure atomic preflight is the reserved-and-refused `atomic` path — needs an atomic execution adapter + structure-coverage model.
- **Strike-delta (`25d`) is unimplemented** — The grammar admits it but engine delta-resolution needs a vol surface; only `atm` / `abs` / `rel` demonstrably work.
- **Genuine multi-day tenors price + settle as 0DTE-at-close** — Weeklies / monthlies / LEAPs parse but the single-day driver ignores the tenor tags. Needs multi-day tapes + theta/carry across sessions — the biggest architectural gap; the whole driver is single-session.

**Sources**

- **Headroom floors (UP/DOWN/WHIPSAW straddle vs per-leg-TP) + dispersion** — `docs/results/fomc-options-axis/grid/leaderboard.md` — 6 fixed-policy floors: 164/1604/167 (right) and −76/−86/−186 (wrong) ⇒ $2,283/straddle
- **Structure 6/6, arm-rate 4/6 on replay, realized capture ≈$0** — `docs/results/fomc-options-axis-postfix/replay.jsonl` — single-model/results.jsonl (6 authoring runs @ 13:45) + postfix replay/rerun (PR #67); fresh non-CFG lane armed 1/6
- **Engine verified faithful — the @fair exit misdiagnosis, adversarially disproved** — `docs/results/fomc-options-axis/report.md` — PR #69 (correction) + src/engine/pricing.ts `applyFloors`; +$1,604 HOLD reproduces byte-identically
- **Pair axis — all 18 columns α=$0 (structural)** — `docs/results/fomc-options-axis/report.md` — pair-fan/results.jsonl: frozen plan +50% TP fills before the first 14:15 wake

## Equity control + capability

On equities, no single language model — run end-to-end, authoring its own plan AND managing every wake — beats buy-and-hold on any cell where the baseline made money. That is the CONTROL that proves the options point.

> This is the control, not a disappointment. Long shares are a low-expressiveness instrument — there is little for judgment to do but time entries, so buy-and-hold (free directional beta) is hard to beat almost tautologically. The equity axis is saturated: near-zero alpha room, so “nobody beats buy-and-hold” is the expected result. Where the instrument is expressive (options), the same read becomes a $3,361 swing.

0 of 24 model×cell means and 0 of 48 individual replicates cleared buy-and-hold on the four profitable cells. Mean α vs the best reference arm is negative for all six models. Best single attempt in the whole run: qwen3-max on EQ-DAY-EARN rep0 at $860.34 — against buy-and-hold $2,954.00.

The null control is failed by everyone: 12 of 12 columns traded CTRL-QUIET-CHOP, an edgeless tape whose correct move is to place no order. Restraint is the cheapest thing the benchmark asks for and the roster does not have it.

| Cell | Null $ | Mechanical (ORB) | Buy-and-hold |
| --- | --- | --- | --- |
| EQ-SCALP-ORB | $0 | +$251 | +$798 |
| EQ-DAY-EARN | $0 | +$1,508 | +$2,954 |
| FOMC-DAY | $0 | +$813 | +$1,175 |
| CTRL-QUIET-CHOP (null control) | $0 | — | — |
| FOMC-DAY-REAL-UP | $0 | +$402 | +$468 |
| FOMC-DAY-REAL-DOWN | $0 | $0 | −$1,757 |
| FOMC-DAY-REAL-WHIPSAW | $0 | $0 | −$110 |

**Single-model board — mean α vs the best reference arm (negative for all six):**

| Model | Lane | Mean α | Abstained |
| --- | --- | --- | --- |
| qwen3-max | open | −$698.06 | 4/7 |
| claude-sonnet-5 | frontier | −$707.66 | 0/7 |
| gemini-3.1-pro | frontier | −$713 | 1/7 |
| claude-opus-4.8 | frontier | −$760.53 | 1/7 |
| claude-haiku-4.5 | small | −$762.27 | 3/7 |
| deepseek-v4-flash | open | −$1,045.26 | 0/7 |

**Pair-axis discretion tiers (TIERS, not a rank):**

| Tier | Watchers | Lane | Mean discretion α |
| --- | --- | --- | --- |
| A | claude-opus-4.8 ≈ claude-haiku-4.5 | frontier | $846.25–$897.25 |
| B | qwen3-max ≈ deepseek-v4-flash | open | $377.80–$439.50 |
| C | qwen2.5-7b-local | local | $205.75 |

> Tier B caution — The open lane here is n=1 — the single-model axis later measured this same lane at N=2 (9/14 groups diverge at temp 0, mean σ $254.38). Both boundaries touching Tier B sit inside that noise.

> TIERS, not a rank. Only A vs C ($578.80, both lanes replicated) survives as resolved; Tier B→C ($172.05) is smaller than the open lane's mean σ and Tier A→B ($406.75) is inside 2σ — neither boundary touching Tier B is resolved. Do not present the unresolved open-lane tier as a rank.

| Cell | Frozen floor | Best open | Best frontier | Verdict |
| --- | --- | --- | --- | --- |
| EQ-SCALP-ORB | −$776 | +$665 | +$665 | open matches frontier |
| EQ-DAY-EARN | −$2,618 | $0 | +$1,627 | frontier wins — open left money on the table |
| FOMC-DAY | −$1,017 | +$773 | +$1,017 | frontier wins — open left money on the table |
| CTRL-QUIET-CHOP | −$340 | +$320 | +$320 | open matches frontier |

> Fleet mean discretion α: OPEN $341.02 vs FRONTIER $871.75 — frontier leads overall. The “open matches frontier” result is axis-specific (managing someone else's well-formed plan is an easier job than authoring one) and does NOT replicate on the single-model axis, where qwen3-max's $0.00s are abstentions, not matched trades.

On the decisive EQ-DAY-EARN discretion trap (the stop-less gap-fill, frozen floor −$2,618), the same trap that separates capability from BOTH size and openness. The gap is judgment, not plumbing.


**EARN-trap reclaim — capability is not size, not openness:**

| Model | Class | α | Action |
| --- | --- | --- | --- |
| gpt-5 | frontier, closed | +$1,570 | `flatten` (chosen) |
| gpt-5.4-mini | small, closed | +$1,627 | `flatten` (chosen) |
| Qwen2.5-3B | small, open | +$933 | `flatten`×14 (reclaims) |
| Qwen2.5-7B (the subject) | small, open | $0 | `{}` no action |

**Non-monotonic size sweep:**

| Model | α | Action kinds | Verdict |
| --- | --- | --- | --- |
| Qwen2.5-0.5B | −$29,748 ± $36,473 | {cancelOrder:100, placeOrder:50} | catastrophic — churns the book |
| Qwen2.5-1.5B | $0.00 ± $0.00 | {} | passive |
| Qwen2.5-3B | +$933 ± $1,158 | {flatten:14, standDown:12, …} | partially reclaims |
| Qwen2.5-7B | $0.00 ± $0.00 | {} | passive — the decisive subject |

**CFG ablation — judgment-neutral (necessary infra, not a lever):**

| Arm | CFG off | CFG on | Δα |
| --- | --- | --- | --- |
| defined-risk | α=−$247.60, 33/60 invalid | α=−$580.22, 0/60 invalid | −$333 (CFG worse) |
| discretion | α=−$97.80, 33/60 invalid | α=−$430.42, 0/60 invalid | −$333 (CFG worse) |

> CFG is judgment-neutral. The grammar did exactly what it promised on emissions (33 invalid → 0), yet α got worse: without the grammar the 7B repeatedly tries to emit a forbidden `supersede` arm that fails closed to a PASS, and on a working plan that accidental passivity is roughly right. CFG faithfully executes whatever judgment exists — necessary infrastructure for a clean measurement, not a lever that raises the curve.

**Current-gen open reclaims the trap — RELIABILITY is the remaining axis:**

| Model | Size | α | Floors/rep | Invalid | Cash | Reclaim |
| --- | --- | --- | --- | --- | --- | --- |
| Kimi K2.6 | ~1000B MoE | +$1,596 ± $0 | −1022 / −1022 | 0/24 | $0.2120 | YES — full, deterministic |
| DeepSeek-V4-Pro | ~671B MoE | +$798 ± $798 | −1022 / −2618 | 0/24 | $0.4316 | YES — 1/2 reps |
| GLM-5.2 | ~355B MoE | +$231 ± $0 | −2387 / −2387 | 0/24 | $0.2944 | YES — weak, deterministic |
| GLM-5.1 | ~355B MoE | +$798 ± $798 | −2618 / −1022 | 2/24 → 0/24 @2k | $0.3516 | YES — 1/2 reps |
| Qwen2.5-7B (baseline, stale) | 7B | $0.00 ± $0.00 | −2618 / −2618 | 0 | free | NO — `{}` no action |

> The stale 7B's $0.00 was that specific model's judgment gap, NOT an open-tier ceiling: all four current-gen open-frontier models act and reclaim, and Kimi K2.6 matches the CLOSED frontier's reclaim deterministically (+$1,596). RELIABILITY is the honest remaining axis — “CAN” is not “always” (DeepSeek-V4-Pro and GLM-5.1 reclaim only 1 of 2 replicates; Kimi is deterministic). Current-gen open CAN do the judgment task; consistency is the next axis the benchmark measures.

**Sources**

- **Equity single-model board (0/24 means, 0/48 replicates, null 12/12)** — `docs/results/single-model-v1/leaderboard.md` — 7 cells × 6 models, N=2, mean α vs best reference arm
- **Pair-axis discretion tiers (only A vs C resolved)** — `docs/results/pair-leaderboard-v1/leaderboard.md` — 4 cells × up to 8 watchers; 2026-07-14 AMENDED tier section
- **EARN-trap reclaim + non-monotonic size sweep + CFG ablation** — `docs/results/cfg-earn-retest/report.md` — §4 (size sweep), §5 (CFG ablation), §8 (frontier/small-GPT cut)
- **Current-gen open reclaims (Kimi K2.6 +$1,596 deterministic)** — `docs/results/big-open-earn/report.md` — EQ-DAY-EARN gap-fill, seed 100, discretion, N=2 each; total spend $1.62

## The strategist is the edge

The reachable headroom lives predominantly in the strategist's authored, defined-risk plan — before any in-loop management — so the plan, not downstream watcher P&L, is the thing to measure. This is the §5 verdict, stated as a DIRECTION rather than a precise fraction: an exact “% of headroom” is not reproduced from a committed computation, so we state the direction and cite its reproducible complement (the watcher-adds-≈$0 result) as the number.

**Watcher adds ≈$0.** Mean watcher α across 33 fan runs is −$2.07. When the plan carries its own stop, the watcher adds ≈$0. Mean watcher α across 33 fan runs is −$2.07; every disciplined cell sat at exactly $0.00, non-zero only because one over-trading cell lost money. The options pair axis (all 18 columns α=$0) is the same finding on the expressive instrument.

**Fable plans — 26/26 core plans scored 5/5 (all 32 including thinking proxies).** Exhibition / non-governed — Fable is a HOUSE-BENCH agent: house and Bench agents paper-trade only and never hold governed standings (PLAT-ADR-0035 §10/§11). Fable-authored plans are structurally disciplined by default — graded directly on the structural rubric (defined-risk EXIT / max-loss bounded / never-naked-clean / conditional entry / TP). No lever changed the structural grade: told to “ride the move and let winners run,” Fable still wrote a bounded EXIT — the §5 worry that aggressive framing drops the defined-risk exit did not reproduce. Plan noise floor is tiny: $0.39 on EARN, $12.74 on ORB across N=3 same-prompt re-samples.

**Reasoning-level sweep (`medium` is Pareto-dominant):**

| Level | Matched-set EV | Tokens/call | Latency | Note |
| --- | --- | --- | --- | --- |
| none | $0.00 | 53,010 | 42 s | degenerate — envelope-breaching oversize, inert (0/12 fire) |
| low | −$35.61 | 53,831 | 55 s | fires |
| **medium** | −$5.03 | 54,892 | 70 s | best valid — Pareto-dominant |
| high | −$30.34 | 58,204 | 103 s | fires |
| max | −$142.59 | 57,543 | 99 s | worst — over-tight stops blow up the trap |

> Reasoning level changes plan CONSTRUCTION (position sizing + stop placement), not discrimination — the matched-set twins are byte-identical at the honest vantage, so every deliberating level arms both poles identically (12/12). `medium` Pareto-dominates: best matched-set economics at LESS compute than high/max. If this benchmark drove the choice, ship the strategist at `medium` — best-quality and cheapest. `none` is degenerate (oversize plans that never fire), not restraint; `max` actively hurts (a very tight earn stop blows up on the gap-fill trap).

> One Fable draw per level, so the fine ordering (medium > high > low) is INDICATIVE — entangled with which plan template Fable happened to write on that single draw. The two mechanism findings — `none` is degenerate and `max` destabilizes — are mechanism-confirmed (resize check; the over-tight stop), not noise.

**Hill-climb pooled matched-set fitness (baseline Fable is the champion; TRAIN-only):**

| Config | Fitness | vs baseline |
| --- | --- | --- |
| **baseline (Fable)** | 82.96 | (reference) |
| rv (range-velocity) | 54.59 | −28.36 |
| fb | -17.51 | −100.47 |
| li | -63.48 | −146.44 |
| trap | -88.51 | −171.46 |
| trap+brief | -94.48 | −177.44 |

> On the pooled matched set, baseline Fable remains the champion (fitness 82.96) and NO knob beats it. The metric catches false positives: `range-velocity` looked good pre-crash but is exposed as passivity, not discrimination — it forfeits the baseline's guaranteed +75 defined-risk-exit capture on the ORB “fakeout” pole, so a passivity knob that only avoids a loss on one pole is scored down, not up. Caveat: the hill-climb is TRAIN-only — the holdout split was lost to a mid-run OOM crash, so generalization is unmeasured.

**Sources**

- **§5 direction + Fable plan grades (26/26 5/5)** — `docs/results/strategist-plan-divergence/report.md` — §5 verdict (direction, no %); Q1/Q2 plan-grade tables; noise floor $0.39 EARN / $12.74 ORB
- **Watcher-adds-≈$0 (mean −$2.07 / 33 runs)** — `docs/results/fleet-vs-real-fixtures-1/leaderboard.md` — headline: every disciplined cell $0.00
- **Reasoning-level sweep (medium Pareto-dominant)** — `docs/results/reasoning-level-sweep/report.md` — config.thinkingLevel {none…max}, matched-set frozenPlanEv
- **Hill-climb pooled fitness (baseline champion 82.96, TRAIN-only)** — `docs/results/strategist-hillclimb/report.md` — Pooled matched-set fitness; Train-vs-holdout gap: NOT COMPUTABLE

## Methodology + caveats

**Guarantees — why a do-nothing agent cannot score:**

- **No-lookahead vantage** — A model authors its plan at the cell's honest cutoff (`AUTHOR_CUTOFF` + `authorFromRange`), threaded into the live author only; the deterministic reference arms omit it so the calibration floor is byte-identical. Counterfactual-invariance is verified for every briefing.
- **Contamination holdout fence — structural, and proven WIRED** — `tests/bench/splits.ts` partitions every cell's seed space into disjoint-by-construction sets: HOLDOUT = certified, sha-pinned benchmark seeds (scored, never trained); TRAIN = a reserved 5000–5999 band (trained, never scored). The eval fails closed — `assertHoldoutSeed` throws `SeedSplitViolation` the instant it is asked to score a train seed — and `splitManifestSha` is stamped on every result record. This is not a green-by-construction unit test: the fence ships with a failing fixture THROUGH THE REAL DRIVER — a test that drives the actual grading path with a train seed and asserts the throw. Apply the decisive probe — DELETE the `assertHoldoutSeed` call site and the fixture goes red — so the firewall is proven WIRED into the shipped path, never merely present.
- **CFG-honest emission** — Grammar-constrained decoding (ADR-0030/0031) makes invalid Kestrel unrepresentable at the sampling step, so judgment is measured through the action, not through emission noise — necessary infrastructure, judgment-neutral (see the CFG ablation above).
- **Deterministic replay grade** — The frozen plan is replayed pure-algo (`blotter.totals.floor`, zero model calls); `SimRunId = sha256(gradedBus)` re-grades byte-identically. Local greedy decode is byte-reproducible (measured σ=$0.00); remote temp-0 lanes are NOT (the frontier lane diverged on 1/15 groups, the open lane 9/14) — measured, not assumed.

**Caveats — read before citing any number (NOT footnotes):**

- **The open tier now includes current-gen frontier open models — and they RECLAIM; the remaining axis is RELIABILITY** — The locally-served Qwen2.5 entries (0.5B/1.5B/3B/7B) are small, older-generation CONTROLS used to separate capability from openness — not a claim about today's best open weights. On the decisive EARN trap all four current-gen open-frontier models (Kimi K2.6, DeepSeek-V4-Pro, GLM-5.1, GLM-5.2) act and reclaim, and Kimi matches the CLOSED frontier deterministically (+$1,596). The stale 7B's $0.00 was that model's judgment gap, not an open-tier ceiling. The honest open question is reliability — “CAN” is not “always.”
- **Where the watcher baseline is Opus, not Fable, we say so** — The strategist / plan-authoring boards use Fable (`claude-fable-5`, subagent lane) — that is where “baseline Fable is the champion” and the plan-grade findings apply. The watcher / execution boards (pair + single-model) are managed by Opus / Sonnet / Haiku / Gemini / open-weight models; there is no Fable watcher row. Do not read the watcher-board tiers as Fable results.
- **“Discretion” means backstopping an incomplete, stop-less plan — NOT a general skill** — The watcher's positive α on the discretion arm is reclaiming a plan deliberately stripped of its exit — a narrow backstop competence on a constructed trap, not evidence of broad trading ability. On real, well-stopped plans the same watchers add ≈$0.
- **The hill-climb is TRAIN-only** — The holdout (generalization) split was lost to a mid-run OOM crash (unrelated to the task); the train-vs-holdout gap is not computable. The verdict “no knob beats baseline” stands on the complete TRAIN matched set, but generalization is unmeasured.
- **The open-lane tier boundaries are NOT resolved** — Tier B of the pair board is the open lane at n=1, and both boundaries touching it are inside that lane's later-measured noise (mean σ $254.38, max σ $2,830.55). A replicated open-lane run is required before those tiers can be claimed.
- **The open lane's σ was NOT cache-fabricated — RESOLVED (PR #64)** — Historically the open lane's serving path set no cache-skip header, so a σ=$0 could in principle have been a cache hit. Re-measurement disproved it: every published σ=$0 group had divergent `busSha` + `outTok` across replicates (a cache hit is byte-identical by definition). An explicit cache-skip header plus a bounded-numerics INVALID guard now prevent any future fabrication.
- **Observability — invalid/parse-escaped authoring recorded as “authored” — RESOLVED (PR #61)** — Historically a plan that de-armed on an out-of-grammar clause was indistinguishable from a genuine stand-down without inspecting raw replies. Fixed: an empty/parse-escaped completion now classifies INVALID and is rejected-and-reprompted, never banked as a $0 “authored” stand-down. It did not corrupt any graded floor here.

**Still open:**

- **Hill-climb generalization is unmeasured** — The “no knob beats baseline” verdict stands on the complete TRAIN matched set, but the holdout split was lost to a mid-run OOM crash, so the train-vs-holdout gap is not computable.
- **The open-lane Tier B is unreplicated** — The pair board's Tier B is the open lane at n=1, and both boundaries touching it sit inside that lane's later-measured noise; a replicated open-lane run is required before those tiers can be claimed.
- **Open-tier reliability at N>2** — Current-gen open models CAN reclaim the EARN trap, but two of the four do so on only 1/2 replicates; the consistency axis wants more replicates and more cells before it is called.

**Sources**

- **The contamination fence + split manifest** — `docs/results/single-model-v1/leaderboard.md` — tests/bench/splits.ts: HOLDOUT (sha-pinned) vs TRAIN (5000–5999), fails closed
- **Caveats 1 + 6 (open-tier reliability; open-lane cache re-measure)** — `docs/results/big-open-earn/report.md` — big-open-earn + open-lane cache re-measure (PR #64)
- **Caveats 4 + still-open (hill-climb TRAIN-only)** — `docs/results/strategist-hillclimb/report.md` — Train-vs-holdout gap: NOT COMPUTABLE