Which failure modes should we score separately on sufficiency tasks?

**Which failure modes should we score separately?** Score them separately, because one overall accuracy number hides which behaviour is wrong.

- **Premature release:** the case advances while required items are still missing. - **Over-request:** the request asks for an item already satisfied, by a supplied document or a held record. - **Under-request:** the request leaves a real gap unnamed, so the customer must be contacted again. - **Wrong item:** the request names a document type that would not satisfy the rule, such as a payslip where tax returns are required. - **Stale acceptance:** an expired or out-of-window document is treated as valid. - **Party mismatch:** a document issued to someone else is accepted. - **Missed escalation:** a conflict between documents is resolved by guesswork rather than referred. - **State error:** the right decision is reached but status or clock is left wrong. - **Loop:** the agent re-requests items the customer has already supplied in a later reply.

**Which matter most?** That depends on your use. A lending buyer may weigh premature release most heavily, while a service-focused buyer may care more about over-requests. Make the weights explicit in the scorer instead of burying them.

**Is there a failure that only shows in multi-step runs?** Yes. Loops and ignoring a partial reply only appear once scripted replies are part of the episode.