Skip to content
MirrorFirm

What delivery contract makes a pilot easy to ingest and review?

A pilot delivery contract can define manifests, schema versions, dataset partitions and environment dependencies so labs can reproduce ingestion and isolate evaluation assets.

Ask for a manifest that lists each generated asset, its format, schema_version, content hash and intended use. Conversation records can arrive as JSONL, relational tables as Parquet and policies as text with stable document identifiers. The manifest should describe joins and the meaning of missing values.

Separate model-visible inputs from evaluator-only answers. A task_id can connect the two without placing expected actions inside the training prompt. The delivery specification can also identify related case families so the lab can control overlap between training and evaluation partitions.

For a simulated environment, request dependency information, a reset entry point, seed handling and a documented initial-state fixture. Clarify which components are part of the proposed delivery and which the lab must supply; do not assume a dataset includes a runnable environment.

The first ingestion check can validate hashes, schemas and foreign keys, then load a selected task into the lab’s harness. Record any conversion applied during import so later results can be traced to the delivered version and the actual inputs presented to the model.

Make the first delivery auditable and reproducible before it enters a training or evaluation pipeline.

Audience:
Model labs
Content type:
Integration
Buyer question:
Scope and getting started

Content release: r-a220eb6e099b243b

More on Hardship referral