What integration should a pilot prove before a larger evaluation release is scoped?

Choose a narrow task and run it through the intended evaluation system from import to report. The pilot should establish who supplies the task loader, who implements tool adapters and which component owns scoring. A file that parses successfully has not yet demonstrated that the runner interprets its actions correctly.

Agree an interface contract containing `task_id`, `release_id`, `runner_config_id`, `seed` and `episode_id`. Tool adapters should map business actions to explicit requests and return structured errors. Evaluator output can include `check_id`, `result`, `evidence_ref` and `scoring_version`, allowing the buyer to trace a reported failure to a specific state observation.

Keep credentials and deployment configuration outside the generated task bundle. Confirm how the runner creates a fresh episode and captures its configuration for replay. A replay check should establish whether the same controlled events produce the same starting conditions and scoring inputs.

MirrorFirm could scope a pilot around this integration contract without assuming support for every evaluation framework. The useful first step is to exchange a minimal schema and adapter specification, then inspect a complete execution trace before agreeing broader coverage.