How can customers check that an agent is reasoning rather than memorising dataset clues?

**How can customers check that an agent is reasoning rather than memorising dataset clues?**

Use paired cases that change a decisive fact while keeping irrelevant details stable. If the policy exempts work already dispatched, changing the dispatch state should change the expected intervention. Altering a decorative reference code should not.

**Which accidental clues should a reseller inspect?**

File names, record ordering and descriptive labels can reveal expected outcomes. A folder called `requires_reassessment` answers part of the task before the customer reads the policy. Identifier ranges can also become shortcuts if affected cases consistently occupy a recognisable range.

**How should the evaluation material be divided?**

Keep closely related variants together when separating training and evaluation material, so a near-copy does not appear on both sides. A customer can also request held-out combinations of policy conditions and work stages.

A generated pack could include controlled perturbations with declared expected invariances. The evaluator would check whether predictions change only when the governing facts change. For resellers, this provides a concrete way to assess the integrity of a repackaged benchmark after renaming files, merging collections or adding catalogue metadata.