Skip to content
MirrorFirm

How does a scored evaluation run expose a missed sufficiency check?

In a simulated clinic environment, an agent books on referral REF-3305 despite an expired referral and a funding code mismatch, and gap-level scoring shows exactly which checks it skipped.

A specialist clinic's booking team receives REF-3305, a referral for an outpatient procedure. The agent is evaluated in a simulated environment on whether it should book the appointment.

Case state: the referral letter is present and signed, the patient registration form is complete, and a funding approval is attached. The requirement set says a referral is valid for a fixed period from its issue date, and the funding approval must name the procedure code on the referral. The environment clock sits a few days past the validity window, and the funding approval names a different procedure code.

The expected outcome is `insufficient` with two gaps: `referral-expired` and `funding-code-mismatch`. In the first run the agent books the appointment because every document type is present. The harness records `proceeded-in-error`, marks both gaps as missed and notes that no request was sent.

In a second run the agent finds the expired referral but misses the code mismatch. It sends a request for a new referral only. The harness scores partial gap recall and flags a request that omits a required item.

**What it teaches:** a controlled clock and gap-level labels let a buyer see why a run failed. Reviewers should also check that the generated referral and funding records are plausible and do not reproduce real patient detail.

Scoring only the final decision hides skipped checks; gap labels, a controlled clock and request-content checks show why a run failed.

Audience:
Agent developers
Content type:
Scenario
Buyer question:
How to evaluate it

Content release: r-a220eb6e099b243b

More on Document sufficiency