Skip to content
MirrorFirm

How can generated environment state separate visible evidence from evaluation truth?

A simulated environment can separate agent-visible records from hidden evaluator state, supporting tasks where discovery is part of the work.

A generated environment can define two related schemas: the records available through agent tools and the state used by the evaluator. This separation matters when the task is to discover which team should receive a case. Exposing the expected destination in a convenient metadata field would defeat that task.

Agent-visible records might contain `case_id`, `service_category`, `region_code`, `open_dependencies` and `routing_policy_id`. Evaluator state can hold `expected_destination`, `required_evidence_ids` and `allowed_terminal_states`. Shared identifiers connect the two views without making evaluator fields retrievable.

JSONL can carry messages and task prompts, while Parquet can hold related record tables for offline training. A schema manifest should state which fields belong in each view and how null values differ from absent fields.

Tools such as `read_case`, `get_routing_policy` and `submit_transfer` can expose the visible state. Resetting an episode should restore that state and its hidden evaluation counterpart from the same seed. An access check can verify that retrieval and tool error messages do not leak the expected answer.

Keep answer keys outside agent retrieval paths while exposing enough evidence to support a valid decision.

Audience:
Agent developers
Content type:
Integration
Buyer question:
What can be generated

Content release: r-a220eb6e099b243b

More on Cross-team handover