How should a pilot delivery be laid out, versioned and validated on arrival?

A pilot delivery works best when it is structured like a small release. The following layout is one that a buyer can ask for.

- **manifest.json**: release_id, version, generation date, file list, checksums, schema version. - **tasks.jsonl**: one task per line with task_id, instruction, snapshot_id, rule_tags and difficulty_band. - **snapshots/**: Parquet files per record type, partitioned by snapshot_id. - **references.jsonl**: expected final state and rationale per task_id. - **policies/**: policy text files with policy_id, matching the ids used in tasks. - **tools.json**: tool names, argument schemas and return shapes. - **CHANGELOG**: what changed between versions and which task ids were affected.

Checksums let you confirm that files arrived intact. Version fields let you tie each evaluation result to the exact data used, which matters if you later compare runs.

For environment use, ask for a reset command or script that loads a snapshot by id and clears all run state. For repeat testing, ask for seeds, so that the same task can be regenerated with variations such as different amounts or customers while keeping the same rule.

Validate on arrival: check that checksums match, that every foreign key resolves, and that every task has a reference. Report defects with task ids. Licence terms, delivery method and any limits on redistribution are agreed with MirrorFirm separately and are not implied by the layout.