How could a pilot be delivered, versioned and wired into our CI?
A pilot can be delivered as versioned files with a manifest and seeds, described through MirrorFirm's MCP server, and run in CI with fixed seeds and structured grader output.
A pilot can be structured so that it behaves like any other versioned test asset in your pipeline.
**Delivery:** a downloadable export containing Parquet tables, JSONL tasks and answer keys, policy documents and a manifest. The manifest records the release version, schema version and the seed used to generate each task.
**Versioning:** treat each release as immutable. Changes to keys, cases or schemas arrive as a new version with change notes, so a result can always be tied to the exact material it ran on.
**Discovery:** MirrorFirm's public MCP server can be used by tools and AI systems to read descriptions of what a release contains. Whether a given environment or dataset is available is confirmed during scoping.
**CI wiring:**
- fetch a pinned release version - call `reset(seed, task_id)` before each task - run the agent with a fixed configuration - call `grade(task_id)` and write JSONL results - compare check-level results with the previous agent version
**Data handling:** store the material where your governance process expects it, and record who has access. Note the permitted uses from your licence in the repository README.
**Feedback loop:** log disputed keys and implausible cases as issues against the release version, so the next iteration can address them.
Versioned files, fixed seeds and structured results let a pilot slot into your existing test pipeline and be repeated.
- Audience:
- Agent developers
- Content type:
- Integration
- Buyer question:
- Scope and getting started
Content release: r-a220eb6e099b243b
Related questions
- What does a first pilot look like for an agent team in vendor finance?
- What limits apply, and will we receive any real customer records?
- How should a team scope and license a generated booking pack?
- What should we ask for in a sample or pilot of generated booking material?
- How should a sample be delivered, versioned and load-checked in my pipeline?
- What does a first sample review look like for an environment team?