Skip to content
MirrorFirm

How could a pilot delivery be organised so my team can load it quickly?

A pilot can ship as a self-describing bundle with a manifest, schemas, JSONL cases, Parquet tables, policies, tasks, environment seeds and grader files, plus split and feedback conventions.

A pilot delivery can be organised as a small, self-describing bundle so your team can load it without back-and-forth.

- `manifest.json`: version, domain, seed list, file hashes, schema versions and the agreed scope. - `schema/`: JSON Schema for each record type and for each tool action. - `data/cases.jsonl`: one nested case history per line, with submission, checks, actions and outcome. - `data/tables/*.parquet`: flat tables for submissions, cases, rules, queues and links, joined by identifier. - `policies/`: numbered rules with effective dates, in JSON and in readable text. - `tasks/tasks.jsonl`: `task_id`, `seed_id`, instruction, `allowed_tools`, `max_steps`, `reference`. - `env/`: seed snapshots and a reset entry point that takes a `seed_id` and optional perturbations. - `eval/`: grader code or specification, and a results schema.

**Loading.** Read JSONL line by line, validate against the schema, and check file hashes against the manifest before use.

**Splitting.** Ask for train, validation and held-out splits by seed or by rule, so evaluation tasks do not share cases with training material. State which split rule you need.

**Feedback loop.** Return a list of failed tasks and suspected ambiguities using the same `task_id` values, so changes can be targeted.

The content is generated. Include a review of free text for unintended resemblance to real entities in your acceptance steps.

A manifest, schemas and seed-based splits let your team validate, load and give targeted feedback without back-and-forth.

Audience:
Model labs
Content type:
Integration
Buyer question:
Scope and getting started

Content release: r-a220eb6e099b243b

More on Application intake