Skip to content
MirrorFirm

What integration contract should a buyer define before starting a pilot?

A pilot integration contract can define versioned inputs, tool boundaries, reset behaviour and result exports so runs remain comparable across evaluation infrastructure changes.

Define the runner boundary before commissioning a simulated environment. State whether the buyer needs offline files, a callable tool interface or both. Specify how the runner authenticates a role, loads a task and discovers the actions available to that role.

A run manifest can include `release_version`, `schema_version`, `task_id`, `initial_state_hash`, `policy_bundle_hash` and `simulator_version`. These fields let an evaluator identify whether two runs used the same inputs and behaviour, even if the surrounding model configuration changed.

The reset contract should restore queues, scheduled events, virtual time and role permissions to the declared starting state. It should also isolate concurrent runs so one agent's actions cannot alter another agent's complaint. A read-only initial-state check can confirm that the reset completed before execution begins.

Agree an export containing final state, action traces and evaluation outputs, with stable links back to the task. MirrorFirm could scope generated material to that contract. A useful first integration step is to load a sample, execute a permitted action, reset it and verify that the resulting state matches the declared starting state.

Make the pilot reproducible in your runner before expanding its content or operational complexity.

Audience:
Evaluation teams
Content type:
Integration
Buyer question:
Scope and getting started

Content release: r-a220eb6e099b243b

More on Complaint escalation