What should reviewers inspect before trusting routing benchmark results?

Review these properties before interpreting model scores, using a sample that covers ordinary cases and difficult boundaries so defects in the evaluation material are not mistaken for weaknesses in the agent.

- Have reviewers independently derive destinations from agent-visible evidence, then investigate disagreements instead of treating the generated answer as authoritative by default. - Check that each required decision has enough supporting evidence, and mark deliberate uncertainty explicitly so accidental omissions cannot silently become reasoning tests. - Inspect category coverage against the intended evaluation purpose, including whether rare but consequential routing boundaries receive enough attention to support separate analysis. - Search for answer clues in templates, ordering and metadata, especially repeated phrases that correlate with queues without expressing a business reason. - Review language variation for coherent requests, ensuring terse messages and imperfect grammar remain interpretable without introducing unintended changes to the requested outcome. - Maintain adjudication notes for contested cases, recording the decisive evidence and the rubric interpretation so later reviewers can reproduce the judgement. - Examine failures by case family before aggregation, checking whether one defective template or policy ambiguity is responsible for an apparent pattern of model weakness.