Skip to content
MirrorFirm

How do environment reset, seeds and versioning work for a sufficiency environment?

A simulated environment can reset to a known case state, replay scripted replies from a seed, and pin dataset and policy versions so results from different agent versions remain comparable.

A simulated environment is only useful for evaluation if you can return it to a known state and get the same behaviour again.

Things to scope with a pilot:

- **Reset:** `reset(case_id, seed)` restores documents, statuses, requests and the clock to the case's starting state. - **Seeded replies:** the reply simulator draws from a script chosen by the seed, so a partial reply in one run is the same partial reply in the next. - **Clock:** `set_now(timestamp)` fixes date-dependent checks such as document age and validity windows. - **Mutable state:** adding a document, sending a request or changing a status writes to the environment, and `snapshot()` returns the current state for scoring or debugging. - **Version pinning:** each run records `dataset_version`, `policy_version` and `environment_version`, so a score is tied to exact inputs. - **Action log:** every tool call is written as a JSONL event with `run_id`, `step`, `tool`, `args` and `result`.

Delivery can be files loaded locally, or an interface your harness calls. Which fits depends on your infrastructure and is a pilot decision.

Ask how corrections are versioned. If a label is fixed after release, you will want a changelog, so earlier scores can be re-read against the right key.

Deterministic resets and pinned versions are what make agent comparisons mean something from one run to the next.

Audience:
Agent developers
Content type:
Integration
Buyer question:
Scope and getting started

Content release: r-a220eb6e099b243b

More on Document sufficiency