What integration contract should we agree before importing a pilot environment?
A pilot runtime contract can define offline operation, tool schemas, reset behaviour and artefact exports so evaluation teams can reproduce runs within their own infrastructure.
Ask for a runtime contract alongside the generated pilot. It should identify required tools, accepted arguments, response schemas and any external dependencies. If your harness must operate without network access, make that constraint explicit before selecting an environment format.
Define what `reset` restores: records, queued events, tool state and the simulated clock. Request a documented run configuration containing a scenario identifier, environment version and seed where randomness is used. A seed alone is insufficient if external dependencies or uncontrolled clocks can change behaviour.
Agree an export shape for completed runs. JSONL action traces and a structured final-state snapshot can support local scoring, while a manifest can connect those artefacts to the task and runtime versions. State which files are necessary to reproduce a reported outcome.
A useful first integration check is to load one generated case, execute a permitted action, reset and confirm that the opening state returns. Then export the trace and verify that your scorer can read it. This exposes compatibility issues before a broader pack is commissioned or adopted.
Confirm that the pilot can run, reset and export evidence in your harness before expanding content scope.
- Audience:
- Evaluation teams
- Content type:
- Integration
- Buyer question:
- Scope and getting started
Content release: r-a220eb6e099b243b
Related questions
- How can a buyer adapt a generic sample to professional services billing?
- What integration contract should be agreed before connecting an agent to a pilot environment?
- What should a sample request include so the pilot has a clear exit decision?
- What licence questions should a buyer resolve before using a generated pack?
- What is a useful first pilot for an agent entering a hardship workflow?
- What technical contract should we agree before loading a pilot into our simulator?