Is there a deterministic, reproducible backtester for AI and LLM trading agents?

The two failures every current tool forces

Rule-based backtesters (Pine, Backtrader, vectorbt and kin) run the same way twice, but they cannot read unstructured text — no news, no market gestalt, no judgment. Wire a raw LLM into a loop and you get perception but lose the second run: sampled tokens make every replay a different backtest, so nobody — including you — can regenerate your own result. A backtest another person cannot regenerate is a story, not evidence.

How an LLM stays in the loop without smuggling in nondeterminism

A Kestrel Session preserves three runtime invariants: no wall time, no nondeterminism, no silent defaults. The model reasons over the market — rendered as compact text Frames it can actually read — and writes its conditional strategy down as a deterministic Kestrel plan; the run executes that plan against the recorded session, so the replay is exact. Brains stay outside: the platform hosts the deterministic reflexes and the evidence, never the model, and an author-provenance guard refuses any certified record that credits strategy authorship to the platform. The strategy is yours; the runtime is infrastructure.

Byte-identical means byte-identical

certify pointed at any proof URL re-projects the run’s Blotter locally from the published npm CLI and reproduces the hosted result byte-for-byte — the determinism leg. It is exactly that claim and nothing more: it is not a judged-Grade claim, and it says nothing about whether the strategy is any good. What it establishes is that the numbers on the page are the numbers the runtime produced — and that a skeptic can establish that on their own machine, without trusting yours.

Scored against doing nothing

Every run is graded against The Perch — the null-policy do-nothing baseline — with losses kept on the record, and buy-and-hold beside the result as the control, never the headline. The platform makes no skill or alpha claim in its own voice; it is the referee. The certified record speaks for itself, and you read it out-of-sample.

Falsify this page

This page pins a replay of a market that already happened: the S&P 500 ETF through the 2020 pandemic volatility crash, 371 frames. Recompute it — if the Blotter does not reproduce byte-for-byte, the claim fails and the command tells you:

Or start from scratch: npx kestrel.markets@latest sim s-p-500-etf-pandemic-volatility-crash mints a fresh proof URL of your own — no signup, no account.