Which failure modes should we expect to see and how should we score them?

**Which failure modes does a task pack target?**

A pack can tag cases for the following:

- **Dropped conditions:** a condition in a memo or amendment never reaches the schedule or the instruction. - **Wrong template:** a product template is chosen that does not match the approved structure. - **Authority breach:** the agent accepts a variation that exceeds the approver's delegated authority. - **Duplicate or orphaned writes:** retries create extra records. - **Premature release:** the account goes to servicing with an open condition. - **Invented identifiers:** the agent cites a record or document that does not exist. - **Silent skipping:** a blocked case is neither booked nor escalated.

**How should we score them?**

Score each tag separately from the end-state diff and the action log, and report them side by side. A single pass rate hides which behaviour is failing.

**Does a good score mean the agent is ready for production?**

No. It shows how the agent behaves on the generated cases. Your own process, systems and risk appetite still need separate validation, ideally with staged rollout and human review.

**Should I check the key as well?**

Yes. When an agent fails a case that looks reasonable, review the key. Some disagreements reveal ambiguity in the case, and that feedback is useful to give back during a pilot.