Which environment hooks make customer change evaluations reproducible?

A simulated environment can expose reset with a scenario_id and seed, returning an initial_state_hash and episode_id. The reset specification should cover records, queued work, simulated time and any pending external events. Restoring only the main customer table can leave hidden differences between runs.

The model-facing interface can provide read and action tools, while a separate evaluator reads protected snapshots. Evaluation hooks can check final state, terminal status and evidence supporting the model's completion claim. Expected answers and scoring rules should remain outside model-visible tool responses when the evaluation requires them to be hidden.

An export can include JSONL action traces keyed by episode_id, plus a machine-readable result containing criterion_id, outcome and evidence_reference. The lab can then inspect why a run failed rather than relying on a single aggregate score.

MirrorFirm could scope deterministic replay where the environment supports it, with explicit treatment of nondeterministic components. Reproducibility should be checked by resetting and replaying a known action sequence. This verifies that differences in later model runs arise from behaviour or declared variation, rather than residual state from an earlier episode.