Skip to content
MirrorFirm

How do set-up stages map onto an agent evaluation harness?

The workflow decomposes into states, tool actions and handover events that a harness can expose, so each stage can be scored independently or as a whole.

A harness can model the workflow as a small state machine with tool actions attached to each stage.

**Stages**

- `decision_received` - `preconditions_checked` - `records_created_or_linked` - `handover_sent` - `activated` or `held`

**Typical tool actions**

- `get_decision(decision_id)` - `search_party(name, identifier)` - `create_party(fields)` and `link_party(decision_id, party_id)` - `create_account(party_id, product_code, terms)` - `set_status(account_id, status, reason)` - `create_task(role, subject, linked_ids)` - `send_notice(recipient_role, template_id, linked_ids)`

**Run record**

Each run can emit a JSONL line per tool call with `run_id`, `step`, `tool`, `arguments`, `result` and `state_hash`. A final line holds the end-state snapshot.

**Why it helps**

- Outcome scoring compares the end snapshot with the expected state. - Process scoring checks that `preconditions_checked` precedes any `set_status` to an active value. - Handover scoring checks that tasks and notices reference the right roles and records.

A task pack could ship the expected end state and ordering rules alongside each case. The schema is a starting point to agree in a pilot, and the data is generated, so check how it maps onto your own tooling.

A staged state machine lets you score outcomes, intermediate decisions and tool use from the same run.

Audience:
Evaluation teams
Content type:
Integration
Buyer question:
What it is and why it matters

Content release: r-a220eb6e099b243b

More on Account setup