How would an evaluation reveal a model that over-requests or under-requests documents?

A retail operations team onboards new suppliers. The task pack contains 40 cases, each with a supplier file and policy `POL-SUP-02`, which requires a company registration extract, a tax registration, bank account verification, a product liability certificate, and a signed code of conduct.

Two model versions are run on the same cases.

**Model A** returns `request` on almost every file. On case `SUP-0233`, which is complete, it asks for the liability certificate that is already in the file as `DOC-5102`. Its decision accuracy looks reasonable because many cases genuinely need requests, but the evaluation records a **false gap** for each unnecessary item and counts the clean cases wrongly held.

**Model B** proceeds on `SUP-0251`, where the bank verification is dated outside the validity window. The evaluation records a **missed gap** with reason code `expired`, and the state check shows the case moved to Approved when the gold status is Awaiting supplier.

A decision-only score would have rated both models similarly. The gap-level comparison separates them: Model A fails on precision, Model B on recall, and each failure points to a different remedy.

**What it teaches:** report false gaps, missed gaps, wrongly held clean cases and wrong status changes as separate measures, and break results out by defect type. A single accuracy figure hides the failure that matters to the business.