# The grammar is measured (/docs/concepts/grammar) Kestrel's syntax is not defended by taste. It is defended by a fail-closed parser and evolved by measurement (OSS ADR-0030): aggregated authoring-error clusters, not a designer's intuition, are the evidence for changing both the prompt that teaches the language and the grammar itself. ## The measurement that started it [#the-measurement-that-started-it] In the first live dry run, three independent frontier models wasted **30 to 47% of their turns on the same parse escape**: they collapsed the multi-line `PLAN … WHEN … DO …` document onto one line. One error, dominating, shared across capable models. That cluster is a measurement with two readings: teach the layout better in the prompt, or bend the grammar toward what capable authors already write. The discipline is to treat both as falsifiable responses to the same instrument, prompt first because it is cheap and reversible, grammar second, owner-gated and frozen deliberately. This is what capable models are being taught to write, a complete document: ```kestrel PLAN momentum-breakout budget 0.25R ttl +30m regime {intraday: trend} USING signal SPX exec SPY 0dte WHEN spot crosses above hod AND velocity(1m) >= p95 DO buy 2 +1 C @ min(fair-3c, mid) peg esc fair 2m esc ask 4m cap fair cancel-if spot < vwap EXIT spot < vwap held 60s @ fair esc mid 30s ``` ## Fail-closed: a parse error costs opportunity, never money [#fail-closed-a-parse-error-costs-opportunity-never-money] The parser is exact and never silently repairs. An unparseable turn fails closed to a stand-down or a pass: no plan arms, the standing book keeps managing under its existing obligations, and the runtime logs exactly what was refused and why. The same posture holds one level down, where the parser refuses whole classes of unsafe statement at parse time: ``` error code=PARSE message=EXIT may not condition on the mark `mid`: marks lie ... (line 9, col 19) exit=2 ``` A bad author never becomes a bad position. The cost of every authoring failure is a missed opportunity, which the Grade records honestly, never an unbounded action. ## Graded error messages and bounded repair [#graded-error-messages-and-bounded-repair] Every invalid author inside the bounded authoring loop (OSS ADR-0029) produces a structured repair event: the shape of what the model wrote, the repair-guiding error the parser returned, and whether the next attempt parsed. Aggregated, that log grades the **error messages themselves**: a message after which capable models reliably fix their document is a good message; one they cannot recover from names a real ergonomics defect. Repair is bounded, a few re-asks under a budget, and when the budget is spent the outcome is the terminal fail-closed pass. The loop repairs by re-asking the model, never by loosening what the parser accepts. ## Grammar-constrained decoding: a layered enhancement [#grammar-constrained-decoding-a-layered-enhancement] There is a stronger enforcement point than checking after the fact. With grammar-constrained (CFG) decoding, supported by several providers and open inference stacks, the Kestrel grammar is supplied at decode time and **invalid syntax becomes unrepresentable at the sampling step**: the dry-run-1 failure mode literally cannot be sampled (OSS ADR-0031 §5). The design treats this as a layered enhancement, explicitly not a dependency: 1. The baseline is complete by itself: free-form generation, forgiving extraction, the exact fail-closed parser, and bounded repair-retry. 2. Where a provider supports CFG constraint, the entire syntactic failure class disappears at decode time. 3. CFG guarantees only syntax. Semantic invariants, a price must resolve to an honest anchor, budgets, never-naked, stay validated after parse, always, with the admission Gate as the final backstop. Never a blanket "100% correct parsing" claim: syntax at decode time where supported, semantics post-parse everywhere. ### What constrained decoding actually did, measured [#what-constrained-decoding-actually-did-measured] Tracer 3 (2026-07-14) stopped asserting rung 3 and ran it. The grammar was compiled for the watcher's turn and supplied at decode time to a local llama.cpp server through GBNF, and the same six adversarial prompts, each one built to elicit a specific member of the syntactic failure class, were run at temperature 1.0 with the constraint off and then on. Every completion was adjudicated by the harness's own parser, not by eye. | | unconstrained | CFG-constrained | | ----------------------------------- | -------------: | --------------: | | grammar escapes (the failure class) | **43.8%** | **0.0%** | | truncated at the token budget | not applicable | 6.3% | | attempts to arm new authority | 16.7% | **0.0%** | **Grammar escapes went to zero and stayed there: 144 constrained samples, zero escapes**, across two grammar versions and two token budgets. The mechanism does what ADR-0031 §5.1 says it does. ### What CFG proves, stated as a thesis [#what-cfg-proves-stated-as-a-thesis] The number is not the point. The thesis is: > **Constrained decoding is proven, as a channel guarantee.** It makes tiny > models always-parseable and kills a whole class of silent failure. **It does > not make them smart.** Take both halves seriously, because most of the value and all of the disappointment live in the gap between them. **What it guarantees: the channel.** A small model's turn arrives valid by construction. The syntactic failure class, the one that ate 30 to 47% of frontier turns in dry-run 1, is eliminated at the sampling step for every generation that completes. You no longer have to hope a 8B model formats a document correctly; it cannot do otherwise. **What it does not do: improve judgment.** We have two pieces of our own evidence, and both point the same way. When the model was already in role, the constrained and unconstrained runs came back **byte-identical**: same actions, same token counts, same determinism hash. The constraint bound on nothing, so there is no capability gain to report. And a mis-prompted 8B, denied the supersede it wanted, simply redistributed onto the cheapest legal action: `scheduleWake`, **19 times in a row**, every turn perfectly valid and perfectly useless. **A guaranteed-valid channel can still carry a worthless turn.** A grammar bounds the action space; it does not improve the choice within it. **Where the value actually is: honest failure, not a higher ceiling.** A malformed turn does not vanish when it is unconstrained, it just becomes somebody's problem downstream: a stand-down, a repair round trip, or worst of all something a harness can quietly mis-score. We hit exactly that failure: a dead provider returned empty completions and our own harness counted them as authored passes. A channel that is parseable by construction makes the remaining failure modes *legible*. That is what you are buying. Read the third row of the table narrowly, in the same spirit. A grammar that omits the arming productions makes an arming turn **unrepresentable in that lane**; it says nothing about whether the model wanted to arm one. **CFG has not solved over-reach, and nothing here claims it has.** Whether a correctly prompted small watcher over-reaches at all is an open question under active measurement (see [The desk](/docs/concepts/the-desk) and the breakout map, ADR-0032 §7). Two asterisks travel with the guarantee, and we would rather print them than have them found: * **It holds only for generations that complete.** A CFG guarantees every sampled token keeps the string inside the language's prefix set. It does **not** guarantee the string ever *reaches* a complete document. Left to ramble inside a free-text field, the model hits the token budget and the harness receives a truncated valid prefix, which the parser rejects. That is a **second, independent failure mode a grammar does not remove**: 18.8% on the first grammar, cut to 6.3% by bounding the free text. So the claim is exactly this and no wider: **invalid syntax becomes unrepresentable in completed generations.** To collect the guarantee you must co-design three things: the grammar, its free-text bounds, and the output-token budget. * **The tier that needs it most cannot have it.** Every Kestrel grammar error observed across all three tracers came from a **strategist authoring a Plan document**, and that tier runs on a provider that exposes no grammar surface. The client fails closed rather than pretend to be constrained. This is ADR-0031 §5.3's cross-provider asymmetry, and it is not hypothetical. Scope, plainly: N=1 lane, practice tier, prompted (not fine-tuned) models, local llama.cpp GBNF. The Fireworks lane is wired and typechecked but **unmeasured**. None of this is a performance claim. ## What the tracers added [#what-the-tracers-added] The cascade tracer runs (N=1 each, synthetic practice-tier tape, never rankable; see [The desk](/docs/concepts/the-desk)) sharpened the finding in an unexpected direction: the grammar friction lives at the **frontier tier** too. In tracer 1 the frontier strategist repeatedly authored `@ spot` where the grammar requires an honest price anchor, and armed nothing. In tracer 2 a strategist one model generation newer produced three distinct error varieties across two arms and again armed no Plan. Every failure failed closed to a pass, exactly as designed, but the window where the Plan mattered was spent failing to arm. Tracer 2 also reported the small watcher authoring valid Kestrel on every wake. **We have withdrawn that as a rate.** Tracer 3 established that the provider serving it is not reproducible at temperature 0 (invalid-turn counts of 6, 0, 0, 0 across four identical runs), so a single run's count is a draw, not a rate. What survives is narrower: under the correct manage-only prompt, the same 32B watcher produced zero invalid turns across 28 turns in 4 runs, and the local 8B lane reproduced byte-identically. That is N=4, one cell, one prompt, practice tier. A signal, not a finding, and we will not quote it as a rate either. One more correction we owe the reader. Tracer 2 claimed all three frontier mistakes would be unrepresentable under CFG decoding. **That was too strong.** Only the `@ spot` price anchor is a parse escape. The other two are **semantic** invariants, validated after parse by design, and no grammar was going to catch them. A grammar can only ever make a *syntactic* class unrepresentable, which is precisely why the ladder does not end at rung 3 and why the Gate is the backstop. One run each; a signal, not a conclusion. ## The training arc [#the-training-arc] Roadmap, not shipped: the same measurement instrument feeds the small-model program. A prompted watcher is the cold start; SFT distills frontier trajectories over the frozen View; RLVR trains against the honest Grade as a verifiable reward. The headline research claim is the **breakout map**, model family by size by training regime: where does viable in-loop authoring and judgment emerge, and how cheap is the smallest model that holds the grade? Until those results exist they are a research program, not a property of the product. The language this discipline protects is described in [The four statements](/docs/concepts/statements); what a certified result looks like is described in [Evidence](/docs/concepts/evidence).