A Supervisor Field Guide to Agent Trading: What to Ask, Demand, and Verify

Answer card

When an agent trades, five questions decide whether it is supervised or merely unattended: who authorized it, its bounded worst case in dollars, how it is revoked, how a run is reconstructed, and who wrote the evidence. Demand each as an artifact, not an assurance. On Kestrel systems each maps to an inspectable primitive: the Envelope, the term-sheet worst-case, one-tap revocation, the certified Blotter, and a Grade the agent cannot author.

Why the old questions miss

Supervisory questions written for human desks assume a human is in the loop at the moment of action. Agent trading breaks that assumption in a specific, checkable way: the intelligence is a language model that is slow and stochastic, so it cannot sit in the execution path. On a well-built system it does not. The runtime fires a pre-authorized, bounded program in milliseconds and informs the agent in parallel: fire-then-inform, and the agent is never in the hot path.

That changes what "supervision" means. You are not supervising a decision made at 11:04:07; you are supervising the authority that made that decision fireable, the bounds on it, and the evidence of what happened. The questions below target those, not the model's reasoning.

A framing note before the checklist. The honest thesis of this design is that the two failures of LLM trading are interface (perception and latency), not intelligence. That matters to a supervisor because it tells you where to look: not at the cleverness of the model, but at the authorization and evidence plumbing around it. That plumbing is what you can actually inspect.

Full disclosure up front: the Kestrel language and runtime are open source, and the platform described here (kestrel.markets) is partly live and partly in build. As of mid-2026: anonymous trial sims, certified Grades, shareable proof URLs, and 402 Offers with Stripe settlement are live; always-on paper presence and the human-signed live path are in build. The free tier needs no signup. Treat this as a specification of what to demand, some of which is live and some of which is being built toward.

The five questions

1. Who authorized it?

Demand a single, named authorization primitive, not a pile of API keys, config flags, and IAM roles you have to reason about together. In Kestrel systems that primitive is the Envelope: a signed grant of {scope, budget, ceiling, expiry, revocation} attached to a node in the pod tree.

What to verify:

• Authority only narrows downward. A parent pod allocates a risk envelope to a child Book or Trader; the child can never hold more authority than the parent granted. Ask to see the tree and confirm no leaf exceeds its parent.
• Two-signer rule. A wallet may sign commerce-only, reversible scopes (data, sim, grade, paper). A human must sign anything legally irreversible: broker connection, LIVE authority. If a machine signature alone reached a real broker, that is a finding.
• Mandatory expiry. Every Envelope carries an expiry. A grant with no expiry is a standing liability; reject it.
• Risk can veto but never open. The risk seat (L0) may clamp or veto anyone, including the agent, but may never OPEN risk. Confirm risk authority is one-directional.

Ask: show me the Envelope that authorized this position, the signer, and the parent it narrowed from.

2. What is the bounded worst case, in dollars?

An agent that can size its own risk is unbounded until proven otherwise. Demand the worst case as a number before the fact, not a reconstruction after.

Two places to check it:

• The Envelope ceiling. budget and ceiling bound loss structurally. The term-sheet approval URL must show worst-case-in-dollars before a human signs, and its sliders may only tighten: a supervisor or principal can reduce exposure but the interface cannot widen it.
• The Plan bracket. Every Kestrel PLAN is a bounded-risk contingent program with an explicit budget, bracket, invalidation, and TTL. Risk is stated in the statement itself.

USING signal SPX exec SPY 0dte
PLAN momo-fade budget 2R ttl +45m regime {intraday: trend-up}
  WHEN spot crosses above HOD AND velocity(5m) >= p95 held 120s
  DO buy 1 atm C @ lean(bid, fair, 0.5)
  TP 2x frac 0.5 @ fair+3c
  EXIT velocity(5m) <= p50 @ bid
  INVALIDATE spot crosses below HOD

Read it as a supervisor: budget 2R caps the loss, ttl +45m caps the duration, INVALIDATE names the kill condition. The worst case is legible in the text. A play you cannot read a dollar bound out of is a play you cannot supervise.

Illustrative only: generic instruments, not a strategy recommendation, not advice. The point is the shape of a supervisable authorization, not this trade.

3. How is it revoked?

Authorization you cannot withdraw instantly is not authorization; it is a hostage situation. Demand:

• One-tap revocation on any Envelope, propagating to every child that narrowed from it.
• LIVE as a platform-enforced singleton (the live singleton), so there is exactly one live executor per pod lineage to revoke, never a fog of duplicates racing each other.
• Risk-seat veto independent of the agent, so a human can clamp exposure even while the agent keeps reasoning.

Ask: if I pull this grant right now, what stops firing, and how fast? The answer should be "everything downstream, immediately," not "we'd need to redeploy."

4. How is a run reconstructed?

After the fact, you must be able to rebuild exactly what happened from evidence, every fill, every trigger, every wake, without asking the agent to narrate. Demand:

• Certified Blotters: the immutable record of what executed.
• Grades: the honest, counterfactual result of a run (see question 5).
• Names are data. A recurring strategy name is one lineage with many instances, a leaderboard key. You can trace a name across runs and pods; the history is queryable, not anecdotal.

Reconstruction should be deterministic: the same inputs replay to the same Frames. Ask to replay a run and confirm the Blotter matches.

5. Who wrote the evidence?

This is the question that separates supervision from theater. If the agent can author its own grade, the grade is worthless. Demand evidence integrity as a structural property.

Kestrel's answer is the GRADE, the honest, counterfactual result of a run, with three protections a supervisor should verify:

• Contamination-fencing. LLM authors are graded only on post-training-cutoff, date-blinded days. A model cannot be credited for "predicting" a day that was in its training data. A Grade is never flattering here by construction, and weights leak what code fences can't stop is the reason the fence exists.
• Counterfactuals in the syntax. VS ungated and VS null are first-class: the grade shows what the judgment added over doing nothing and over the same plan with its regime gate removed. It grades judgment, not parameters.
• Support flag. The grade refuses to bank extrapolated fills: if a fill isn't supported by real liquidity, it is not counted.

GRADE plan momo-fade OVER 2026-04-01..2026-06-30 FILL conservative
  VS null
  BY regime

Ask: who signed the grade, and could the agent have influenced it? The correct answer is that the judge is independent and the agent cannot touch it: open judge, sell certification. The platform's regulatory posture follows from this: certification over custody. It never custodies funds, requires BYO-broker and BYO-plan for anything live, and is impersonal (it certifies runs; it does not advise).

The checklist, condensed

Where this design does not help you

Be honest about the boundaries, because a checklist that claims to cover everything covers nothing:

• It does not judge the strategy's wisdom. The bounds are structural; a well-bounded bad idea is still a bad idea. Supervision here checks authority and evidence, not alpha.
• It does not remove the human where the law requires one. LIVE and broker scope demand a human signature by design. If your regime wants a licensed human accountable, that accountability is a person, not the Envelope.
• It does not custody or advise. If your supervisory model assumes the platform holds funds or gives personalized recommendations, this is the wrong fit; it deliberately does neither.
• Not all of it is live yet. As of mid-2026 the trial sims, certified Grades, proof URLs, and 402 Offers with Stripe settlement are live, while always-on paper presence and the human-signed live path are in build. Verify what is shipping before you rely on it in an examination.

Comparison: what a supervisor can actually inspect

Radically fair: the columns favor no one, and Kestrel is not the answer to every row.

Where the alternatives win: a broker-API setup is faster to stand up for a throwaway experiment, and a human-desk OMS is the right tool when a licensed human is genuinely making each decision. Kestrel earns its ceremony only when an agent is acting and you must prove the authority and the evidence afterward.

What to do Monday

Pick one live or proposed agent strategy and demand the five artifacts: the Envelope, the term-sheet worst-case, the revocation path, a deterministic replay, and a contamination-fenced grade the agent didn't write. If any of the five comes back as prose instead of an artifact, you've found your gap.

If an agent trades and no one can name who authorized it, bound its worst case in dollars, revoke it in one tap, and reconstruct the run from evidence you didn't let the agent write, it is not supervised; it is unattended.