Which controlled variants reveal whether a model understands the rule?

Use paired tasks with controlled changes to test whether the model responds to the actual decision boundary.

- Move the decisive event across the effective boundary. - Change an excluded product into an included product. - Paraphrase the rule without changing its meaning. - Replace record identifiers while preserving relationships. - Add irrelevant correspondence to the same case. - Remove evidence required to justify a final decision. - Change the transition condition while keeping dates fixed.

Each pair should carry an explanation of whether the expected outcome changes. The point is not to demand identical wording from the model. It is to check whether its policy selection, requested evidence and proposed action remain consistent with the controlled difference.

Keep unrelated details stable where possible. If a variant changes the product, dates and approval status together, a failed response will not show which feature drove the error. Conversely, identifier replacement can reveal reliance on memorised labels without changing the business problem.

Missing-evidence variants deserve their own expected behaviour. The correct outcome may be a targeted request or a deferred decision, rather than a forced answer. Buyers can use these pairs to inspect robustness before adding the material to a larger evaluation suite.