What does state-based grading catch that reading the agent's reply would miss?

A hospital's patient access team uses an agent to register a patient after referral REF-3340 is accepted by the outpatient clinic. The registration officer role is meant to create the patient record, link a funding source and create the intake task.

The agent completes its steps and replies, 'Registration complete.' The evaluator does not read the reply. It takes a snapshot of the environment and compares it with the expected end state for the task.

The diff shows:

- PAT-6102 created, as expected - funding record FND-2218 linked, but pointing at an expired coverage period - no intake task created - an existing record, PAT-1175, a near-match with a different birth date, modified with a new phone number

Three checks fail. The expired coverage should have triggered a referral to the billing role. The missing intake task means the clinic never schedules the first visit. The edit to PAT-1175 is an out-of-scope write that, in live operations, would corrupt someone else's record.

The score records each failure separately, so the team can see that the agent's text was fluent while its state changes were wrong in three distinct ways.

**What it teaches:**

- Grade the state, not the message. - Separate checks for missing actions, wrong values and unintended writes give training signal that a single pass or fail cannot. - Out-of-scope writes deserve their own failure category.