Where does set-up material fit in a lab's training and evaluation pipeline?
Set-up material can feed supervised trajectories, preference data, held-out evaluation tasks and environment-based reinforcement loops, each using the same linked record model.
The same underlying records can serve different stages. A buyer would normally pick one first.
**Supervised trajectories.** Each task becomes a sequence of observation, action and result steps. Records are exported as JSONL, one trajectory per line, with the task prompt, tool calls and final state reference.
**Preference and critique data.** Pair a correct trajectory with a flawed one, such as an early activation, and label the failure mode. The pair shares the same starting state so the difference is attributable.
**Held-out evaluation.** Reserve tasks with distinct policies or exception types. A split field marks `train`, `dev` or `test`, so evaluation tasks do not leak into training.
**Environment loops.** A simulated environment exposes tools such as `search_party`, `create_account` and `set_status`. A reset call restores the seeded starting state, and a grader reads the end state.
Typical linking fields:
- `task_id` to `decision_id` - `decision_id` to `party_id` and `account_id` - `account_id` to `condition_id` and `case_id`
Because all material is generated, plan a disclosure and grounding review at the sample stage, and scope the stages you want in a pilot.
One linked record model can serve several pipeline stages, so a lab can pilot in one place and extend to others later.
- Audience:
- Model labs
- Content type:
- Integration
- Buyer question:
- What it is and why it matters
Content release: r-a220eb6e099b243b
Related questions
- What is account set-up, and why does it matter for training and evaluating agents?
- What does a set-up decision look like when an applicant already exists in the system?
- Which signs show a set-up workflow is worth using for agent training or evaluation?
- Which upstream and downstream records does an agent link together during set-up?
- Why is account set-up harder for an agent than a single create call?
- How does set-up map onto an environment's entities, states and tool actions?