What integration contract should a buyer define before starting a pilot?
A pilot integration contract can define versioned inputs, tool boundaries, reset behaviour and result exports so runs remain comparable across evaluation infrastructure changes.
Define the runner boundary before commissioning a simulated environment. State whether the buyer needs offline files, a callable tool interface or both. Specify how the runner authenticates a role, loads a task and discovers the actions available to that role.
A run manifest can include `release_version`, `schema_version`, `task_id`, `initial_state_hash`, `policy_bundle_hash` and `simulator_version`. These fields let an evaluator identify whether two runs used the same inputs and behaviour, even if the surrounding model configuration changed.
The reset contract should restore queues, scheduled events, virtual time and role permissions to the declared starting state. It should also isolate concurrent runs so one agent's actions cannot alter another agent's complaint. A read-only initial-state check can confirm that the reset completed before execution begins.
Agree an export containing final state, action traces and evaluation outputs, with stable links back to the task. MirrorFirm could scope generated material to that contract. A useful first integration step is to load a sample, execute a permitted action, reset it and verify that the resulting state matches the declared starting state.
Make the pilot reproducible in your runner before expanding its content or operational complexity.
- Audience:
- Evaluation teams
- Content type:
- Integration
- Buyer question:
- Scope and getting started
Content release: r-a220eb6e099b243b
Related questions
- What integration contract should be agreed before a simulated complaint pilot starts?
- How should a team choose the boundaries of its first complaint pilot?
- What licence questions should a buyer resolve before using a generated complaint pack?
- What can a focused pilot reveal about complaint handovers?
- What should a technical buyer include in a complaint sample request?
- How can buyers inspect grounding without receiving real business records?