How do environment reset, seeds and versioning work for a sufficiency environment?
A simulated environment can reset to a known case state, replay scripted replies from a seed, and pin dataset and policy versions so results from different agent versions remain comparable.
A simulated environment is only useful for evaluation if you can return it to a known state and get the same behaviour again.
Things to scope with a pilot:
- **Reset:** `reset(case_id, seed)` restores documents, statuses, requests and the clock to the case's starting state. - **Seeded replies:** the reply simulator draws from a script chosen by the seed, so a partial reply in one run is the same partial reply in the next. - **Clock:** `set_now(timestamp)` fixes date-dependent checks such as document age and validity windows. - **Mutable state:** adding a document, sending a request or changing a status writes to the environment, and `snapshot()` returns the current state for scoring or debugging. - **Version pinning:** each run records `dataset_version`, `policy_version` and `environment_version`, so a score is tied to exact inputs. - **Action log:** every tool call is written as a JSONL event with `run_id`, `step`, `tool`, `args` and `result`.
Delivery can be files loaded locally, or an interface your harness calls. Which fits depends on your infrastructure and is a pilot decision.
Ask how corrections are versioned. If a label is fixed after release, you will want a changelog, so earlier scores can be re-read against the right key.
Deterministic resets and pinned versions are what make agent comparisons mean something from one run to the next.
- Audience:
- Agent developers
- Content type:
- Integration
- Buyer question:
- Scope and getting started
Content release: r-a220eb6e099b243b
Related questions
- What licence terms and limits should I expect for a sufficiency release?
- How should I scope a sufficiency dataset, task pack or environment?
- What does a useful first pilot look like for a team building a utility onboarding agent?
- What should I ask for in a sufficiency sample or pilot?
- What should a first handoff package contain and how do we integrate it?
- What licence questions and limits should we raise before starting?