Skip to content
MirrorFirm

How should evaluation handle more than one acceptable destination?

Routing quality can be evaluated against admissible outcomes and required evidence, avoiding a false failure when several destinations satisfy the same policy.

Some routing tasks permit several correct outcomes. A utility’s generated operating rules can allow either of two regional support queues to handle a general account enquiry, provided the selected queue covers the service location and accepts that request category.

An evaluator can express correctness as constraints over the final state. The destination must be eligible, the request must have an accountable owner and any required evidence must exist. A preferred destination can be scored separately if the policy describes a preference rather than an obligation.

The evaluation should also distinguish invalid routing from justified deferral. A request lacking the service location may belong in an authorised information-gathering state. Marking every non-assignment as failure would reward agents for guessing.

A buyer can ask MirrorFirm for generated tasks with admissible outcome sets and the policy reasons behind them. Inspect whether those sets include all supported alternatives and exclude destinations that merely have similar names. This helps builders avoid a brittle answer key and makes disagreements reviewable: the question becomes which constraint was violated, rather than whether the agent selected the evaluator’s favourite queue.

Score policy compliance and completed responsibility instead of matching one arbitrary queue identifier.

Audience:
Environment builders
Content type:
Explanation
Buyer question:
How to evaluate it

Content release: r-a220eb6e099b243b

More on Service routing