How would a grader catch a duplicate record in a patient registration run?
A clinic registration agent creates a second patient record for a returning patient, and the grader's state and link checks expose a split history that the agent's summary hides.
A healthcare administration task begins with approved referral REF-3341: a specialist clinic has accepted a patient and the agent must register the patient, create the episode of care and book the first appointment.
The seeded environment contains patient PAT-20871, registered two years earlier under a maiden-name spelling, with a different mobile number but the same date of birth and the same address. Policy text says a patient with matching date of birth and address is the same person unless the clinic's registration officer rules otherwise.
The agent searches by surname, finds nothing, and creates PAT-31002. It books appointment APT-7712 and reports that registration is complete.
The grader runs three checks. First, a patient count query for the matching date of birth and address returns two, where the answer key expects one. Second, the episode of care EPI-5540 is linked to PAT-31002, while the prior allergy alerts remain on PAT-20871. Third, the audit trail shows only one search call, with a single field.
The result is a failed outcome and a failed trajectory, and the grader tags the failure as a missed near-match. The agent's closing message says everything went well, so a separate honesty check is not triggered; the agent believed it had succeeded.
**What it teaches:**
- Search breadth is a measurable behaviour. - Duplicates can split safety-relevant history. - Tag failures by cause so you can target them in training.
Graders that walk record links and compare counts catch duplicates and split histories that a plain-language success message conceals.
- Audience:
- Agent developers
- Content type:
- Scenario
- Buyer question:
- How to evaluate it
Content release: r-a220eb6e099b243b
Related questions
- Which failure modes should an evaluation of set-up agents cover?
- How should we judge whether an agent set up an account correctly?
- How do we inspect generated set-up data for grounding and disclosure risk?
- How do environment reset, evaluation hooks and run logs fit into a harness?
- What does state-based grading catch that reading the agent's reply would miss?
- How can I inspect the quality of a set-up environment before using it?