Skip to content
MirrorFirm

What delivery formats and environment setup should a pilot specify?

Lists the file formats, versioning, validation scripts, environment interface and reset expectations a pilot can specify so the output drops into an existing evaluation stack.

Put integration needs into the pilot brief so the first delivery runs in your stack without rework.

**Data files.** Choose JSONL for record-by-record loading, Parquet for analytics, or both. Ask for one file per record family, such as authority, delegations, requests, routing and gold, with documented primary and foreign keys.

**Versioning.** Request a release identifier in each file and a manifest listing files, schema versions and record counts, so you can detect drift between deliveries.

**Validation.** Ask for a schema definition, for instance JSON Schema, and a script or checklist to verify keys, date ranges and limit arithmetic. Run it in your own pipeline, not only theirs.

**Task pack layout.** Each task bundles a prompt, the record references it needs and a gold entry with condition tags.

**Environment option.** If you want a simulated environment, specify the tool interface (for example function-call signatures or an MCP-style tool list), whether write tools enforce authority or only log attempts, and the simulated clock behaviour.

**Reset and runs.** Agree on snapshot loading, seeded randomness and an append-only event log, so results are repeatable and can be compared across model versions.

**Output.** Define the results format your graders will emit, including per-condition fields.

The content is generated to your specification. Confirm formats and reset behaviour on the sample, along with grounding and disclosure risk.

Specifying formats, versioning and reset behaviour at the start prevents rework and lets the pilot output run in your stack on day one.

Audience:
Evaluation teams
Content type:
Integration
Buyer question:
Scope and getting started

Content release: r-a220eb6e099b243b

More on Delegated approval