What should we inspect before trusting an environment's evaluation results?

Inspect these properties before using generated episodes for training or evaluation, because defects in the environment can make competent behaviour look incorrect or impossible.

- Check causal ordering in event histories, ensuring a reminder cannot cite a response or policy change that had not occurred when it was sent. - Verify every required decision has accessible supporting evidence, with tool permissions that allow the evaluated role to retrieve it during the assigned episode. - Confirm successful terminal states are reachable through permitted actions, rather than requiring hidden mutations or powers unavailable to the agent under evaluation today. - Probe invalid transitions directly, checking that the environment rejects impossible closures and unsupported escalations instead of relying on agents to avoid requesting them. - Inspect generated language for contradictions with structured state, distinguishing deliberately conflicting evidence from accidental disagreement that leaves the correct decision undefined for reviewers. - Review grounding and disclosure risk in generated content, recording questionable details for inspection without treating synthetic origin as an automatic assurance of suitability. - Ask an independent reviewer to trace disputed expected outcomes, documenting policy ambiguity before changing labels merely to match whichever agent behaviour appeared most often.