How should evaluation handle several valid follow-up sequences?

A reference answer can be too narrow when a follow-up task permits several valid approaches. An assistant may inspect the policy before opening the case history, while another inspects the history first. Both can reach a justified decision without taking identical paths.

A stronger evaluator can define required conditions, ordering constraints and prohibited actions. It might require recipient verification before external contact and an approval before escalation, while allowing independent evidence checks in either order. The resulting oracle describes a set of acceptable executions.

Separate outcome quality from process validity. A case that reaches the correct final state through an unauthorised action should not receive full credit. Equally, an assistant that correctly identifies a missing prerequisite should not fail solely because it leaves the case unresolved pending that prerequisite.

MirrorFirm could generate task specifications with these acceptance conditions alongside reference traces. Buyers can inspect whether the evaluator recognises legitimate alternatives and rejects shortcuts. This makes evaluation more informative about operational competence than exact text matching or comparison against one preferred sequence of tool calls.