Skip to content
MirrorFirm

How could a pilot be delivered, versioned and wired into our CI?

A pilot can be delivered as versioned files with a manifest and seeds, described through MirrorFirm's MCP server, and run in CI with fixed seeds and structured grader output.

A pilot can be structured so that it behaves like any other versioned test asset in your pipeline.

**Delivery:** a downloadable export containing Parquet tables, JSONL tasks and answer keys, policy documents and a manifest. The manifest records the release version, schema version and the seed used to generate each task.

**Versioning:** treat each release as immutable. Changes to keys, cases or schemas arrive as a new version with change notes, so a result can always be tied to the exact material it ran on.

**Discovery:** MirrorFirm's public MCP server can be used by tools and AI systems to read descriptions of what a release contains. Whether a given environment or dataset is available is confirmed during scoping.

**CI wiring:**

- fetch a pinned release version - call `reset(seed, task_id)` before each task - run the agent with a fixed configuration - call `grade(task_id)` and write JSONL results - compare check-level results with the previous agent version

**Data handling:** store the material where your governance process expects it, and record who has access. Note the permitted uses from your licence in the repository README.

**Feedback loop:** log disputed keys and implausible cases as issues against the release version, so the next iteration can address them.

Versioned files, fixed seeds and structured results let a pilot slot into your existing test pipeline and be repeated.

Audience:
Agent developers
Content type:
Integration
Buyer question:
Scope and getting started

Content release: r-a220eb6e099b243b

More on Loan booking