What should a technical pilot include for importing and maintaining generated releases?
A technical pilot should define portable artefacts, schema versions and adapter responsibilities so generated follow-up tasks remain interpretable as evaluation infrastructure changes.
Begin with a release manifest describing `release_id`, `schema_version`, `policy_bundle_version`, `task_files`, `record_files`, `licence_reference` and file checksums. Keep agent-visible material and evaluator-only artefacts explicitly separated in the manifest so an import process does not expose answers accidentally.
JSONL can represent self-contained task envelopes, while Parquet can carry linked record tables for analytical inspection. Define timestamp encoding, decimal representation, null semantics and foreign-key rules before building an adapter. A missing field and a field explicitly marked unknown should not become indistinguishable during conversion.
MirrorFirm could scope exports against this contract. The buyer should identify who supplies tool adapters, state storage and evaluator integration, and which components the pilot is intended to validate. Compatibility with one harness should not be assumed to establish compatibility with every internal system.
A practical first import should validate checksums, resolve record links and confirm that an exported task retains its original completion conditions. If environments are included, document reset inputs alongside the release rather than relying on undocumented local defaults. Preserve migration notes when schemas change so longitudinal benchmark comparisons can distinguish model behaviour changes from changes introduced by the data pipeline.
Agree on an import contract that preserves task meaning, provenance and licence references across exports and internal tooling.
- Audience:
- Evaluation teams
- Content type:
- Integration
- Buyer question:
- Scope and getting started
Content release: r-a220eb6e099b243b
Related questions
- What integration contract should be agreed before connecting a pilot to an agent harness?
- Which licence questions matter when generated material enters an agent development pipeline?
- What should developers include in a request for a generated sample?
- What makes a useful first pilot for an overdue-handling agent?
- How can a sample review uncover a missing requirement before a pilot starts?
- What integration contract should we agree before a pilot is generated?