Which checks expose shortcuts that make a review benchmark misleading?

Apply these checks to generated review episodes before using results to compare agents, because accidental cues can make apparent success unrelated to the intended work.

- Shuffle record identifiers while preserving relationships, then check whether decisions change when account numbers no longer correlate with the expected review outcome. - Remove outcome hints from filenames and attachment titles, ensuring retrieval requires interpreting document content rather than recognising labels planted by the generator. - Inspect search ordering for answer leakage, particularly whether the decisive record always appears first regardless of the query or available competing evidence. - Check tool errors for hidden reasons and recommended outcomes that reveal more than the acting role could learn through an authorised business interface. - Vary irrelevant wording while preserving business facts, testing whether the agent relies on repeated phrases that happen to accompany successful closure cases. - Keep related generated case families together when splitting evaluation material, reducing the chance that memorised variants substitute for reasoning about unfamiliar situations. - Review counterpart helpfulness, ensuring a routine enquiry does not automatically return the complete decision rationale when a business role would provide narrower information.