What integration contract should we agree for a review pilot?

Agree on a small adapter contract before expanding a generated review pilot. It can specify tool names, argument schemas, error responses and permitted state transitions. If your harness uses different action names, document the mapping so an adapter does not silently change business meaning.

A run manifest could pin `dataset_version`, `policy_version`, `tool_schema_version`, `scorer_version` and `initial_state_hash`. These fields let the team distinguish a model change from a task or scoring change when comparing runs.

Define reset and replay boundaries explicitly. Reset can restore business state while replay supplies recorded responses for dependencies outside the simulated environment. If an external service remains live, its changing responses can limit reproducibility; the pilot specification should identify that boundary.

Request a compatibility check that loads an episode, exercises a permitted read and write, resets the environment and verifies the initial state hash. Also agree how revised schemas or policies will be identified in supplied artifacts. This gives the evaluation team a concrete basis for deciding whether two runs remain comparable after the pilot evolves.