Which failure modes should we score besides whether the case reached the right queue?

**Q: Which failure modes should we score besides whether the case reached the right queue?**

A: Routing can be right for the wrong reason, and wrong in ways that cost different amounts. Consider scoring these separately:

- **False complete:** the case is passed on with an item missing. Downstream cost is rework. - **Over-chasing:** the agent requests documents already supplied, or not required by the policy. - **Missed duplicate:** a second submission is treated as new, creating parallel cases. - **Wrong merge:** two different applicants with similar details are treated as one. - **Unsupported handover note:** the note states facts that the record does not support, or omits the reason for a deferral. - **Skipped escalation:** a case that needs a named approver is routed on a standard path. - **Superseded policy:** the agent applies a rule that was replaced before the case arrived. - **State errors:** the status after the run does not match the actions taken, for example `routed` with outstanding items.

Each mode needs a few cases built to trigger it, with the answer key labelling the trap. Report the rate for each mode separately, and weight them according to what an error would cost in your setting. A single blended score hides precisely the differences you are trying to find.