Skip to content
MirrorFirm

Where does set-up material fit in a lab's training and evaluation pipeline?

Set-up material can feed supervised trajectories, preference data, held-out evaluation tasks and environment-based reinforcement loops, each using the same linked record model.

The same underlying records can serve different stages. A buyer would normally pick one first.

**Supervised trajectories.** Each task becomes a sequence of observation, action and result steps. Records are exported as JSONL, one trajectory per line, with the task prompt, tool calls and final state reference.

**Preference and critique data.** Pair a correct trajectory with a flawed one, such as an early activation, and label the failure mode. The pair shares the same starting state so the difference is attributable.

**Held-out evaluation.** Reserve tasks with distinct policies or exception types. A split field marks `train`, `dev` or `test`, so evaluation tasks do not leak into training.

**Environment loops.** A simulated environment exposes tools such as `search_party`, `create_account` and `set_status`. A reset call restores the seeded starting state, and a grader reads the end state.

Typical linking fields:

- `task_id` to `decision_id` - `decision_id` to `party_id` and `account_id` - `account_id` to `condition_id` and `case_id`

Because all material is generated, plan a disclosure and grounding review at the sample stage, and scope the stages you want in a pilot.

One linked record model can serve several pipeline stages, so a lab can pilot in one place and extend to others later.

Audience:
Model labs
Content type:
Integration
Buyer question:
What it is and why it matters

Content release: r-a220eb6e099b243b

More on Account setup