How do I wire a pilot sample into my agent harness for a first run?
Load and validate the JSONL, expose a few mirrored tools with structured gate errors, snapshot and reset state per task, drive time from a simulated clock and run two baselines.
A first integration can be small. Load the sample, wrap a few tool actions and run a baseline within your existing harness.
**Load.** Read the JSONL files into an in-memory store or a database of your choice. Validate foreign keys on load: `conditions.decision_id`, `evidence.condition_id` and `work_items.gated_by`. Reject the load if any link breaks and report it back.
**Tools.** Expose a handful of actions that mirror your real ones: `get_decision`, `list_conditions`, `read_evidence`, `set_condition_state`, `request_review` and `advance_work_item`. Return structured errors, such as `GATE_OPEN` with the list of open condition identifiers, so you can see whether the agent responds to refusals.
**Reset.** Snapshot state per task using `initial_state_ref`. Restore it before every run, and clear any agent memory or caches that could carry answers between runs.
**Clock.** Drive expiry from a simulated clock held in the environment, not the system clock.
**Scoring.** Write traces in the format your grader reads, and attach `task_id` and `run_id` to everything.
**Baselines.** Implement a status-only agent and a wait-always agent first. They take little code and show whether the pack separates behaviours.
**Hand-off.** Send MirrorFirm schema questions and mismatches as a short list, so the next release can adjust field names, states or volume.
A small, validated load with structured refusals and two naive baselines gives you a working first evaluation in your existing harness.
- Audience:
- Agent developers
- Content type:
- Integration
- Buyer question:
- Scope and getting started
Content release: r-a220eb6e099b243b
Related questions
- What does a first pilot review look like for a referral-processing agent team?
- How is generated conditional-approval material licensed, and what limits apply?
- What should a pilot request include so the first sample can be judged quickly?
- How should I scope a first pilot for conditional-approval material?
- What should I ask for in a first sample or pilot?
- What should I ask about licensing and permitted use before starting?