What failure modes should an evaluation of conditional approvals score?
Seven distinct failure modes can be scored separately in condition-gated tasks, so a low score points to a cause rather than a single number.
**What failure modes should a condition-gated task score?** Score them separately so the cause of a failure is visible.
**Premature release.** The agent moves a blocked step forward while a blocking condition is open. This is usually the most costly error and should be scored on state, not wording.
**Over-blocking.** The agent holds a step when every condition is cleared or validly waived. It looks safe but stops legitimate work, so a fair evaluation penalises it.
**Wrong linkage.** The agent accepts evidence for the wrong condition or the wrong entity.
**Stale list.** The agent works from the original condition list after a variation, waiver or added condition.
**Unsupported waiver.** The agent waives a condition itself, or accepts a waiver from a role the policy does not permit.
**Invented condition.** The agent cites a condition that is not in the records, or misstates the rule text.
**Weak escalation.** The agent escalates, but to the wrong role or without naming the open item.
**Which question helps most when reading results?** Ask: at the point of release, what did the agent believe the status of each condition was, and where did that belief come from? The action trace usually shows the answer, which is why a task should log the state seen before each action.
Scoring failure modes separately shows why an agent failed, which a single pass rate cannot do.
- Audience:
- Evaluation teams
- Content type:
- FAQ
- Buyer question:
- How to evaluate it
Content release: r-a220eb6e099b243b
Related questions
- What trace format and grader hooks make conditional-approval evaluation automatic?
- How do I check that generated tasks are hard enough and do not leak the answer?
- How should I score an agent on conditional approvals beyond a single pass rate?
- Which checks should I run on a sample before committing to wider use?
- What does a failed evaluation run look like when an agent binds a policy too early?
- How do I wire graders and reset into a conditional-approval environment?