How do set-up stages map onto an agent evaluation harness?
The workflow decomposes into states, tool actions and handover events that a harness can expose, so each stage can be scored independently or as a whole.
A harness can model the workflow as a small state machine with tool actions attached to each stage.
**Stages**
- `decision_received` - `preconditions_checked` - `records_created_or_linked` - `handover_sent` - `activated` or `held`
**Typical tool actions**
- `get_decision(decision_id)` - `search_party(name, identifier)` - `create_party(fields)` and `link_party(decision_id, party_id)` - `create_account(party_id, product_code, terms)` - `set_status(account_id, status, reason)` - `create_task(role, subject, linked_ids)` - `send_notice(recipient_role, template_id, linked_ids)`
**Run record**
Each run can emit a JSONL line per tool call with `run_id`, `step`, `tool`, `arguments`, `result` and `state_hash`. A final line holds the end-state snapshot.
**Why it helps**
- Outcome scoring compares the end snapshot with the expected state. - Process scoring checks that `preconditions_checked` precedes any `set_status` to an active value. - Handover scoring checks that tasks and notices reference the right roles and records.
A task pack could ship the expected end state and ordering rules alongside each case. The schema is a starting point to agree in a pilot, and the data is generated, so check how it maps onto your own tooling.
A staged state machine lets you score outcomes, intermediate decisions and tool use from the same run.
- Audience:
- Evaluation teams
- Content type:
- Integration
- Buyer question:
- What it is and why it matters
Content release: r-a220eb6e099b243b
Related questions
- What is account set-up, and why does it matter for training and evaluating agents?
- What does a set-up decision look like when an applicant already exists in the system?
- Which signs show a set-up workflow is worth using for agent training or evaluation?
- Which upstream and downstream records does an agent link together during set-up?
- Why is account set-up harder for an agent than a single create call?
- How does set-up map onto an environment's entities, states and tool actions?