How can generated routing records and evaluator answers be packaged?
Linked tables and separate evaluator artefacts let teams load generated routing tasks while controlling which evidence and answers the agent can access.
A release can package requests as JSONL and larger relationship tables as Parquet. A `requests` record can contain `request_id`, `received_at`, `channel`, `body_ref` and `account_id`. Separate tables can describe `accounts`, `queues`, `documents` and `request_links`, with declared primary and foreign keys.
A task manifest can identify the starting request, permitted evidence records and schema version. Document references should resolve to supplied content, and timestamps should use a declared timezone convention. These details make loading failures distinguishable from agent failures.
Evaluator artefacts can live in a separate package containing `task_id`, `acceptable_queue_ids`, `required_evidence_refs` and `prohibited_actions`. The agent-facing export should not include those fields or filenames that reveal the answer. Separation must also hold in retrieval indexes and tool responses.
Validation can check referential integrity, unique identifiers and whether every visible reference resolves within the permitted evidence set. A schema can explicitly represent an intentionally unavailable document, allowing tests of incomplete evidence without making accidental broken links part of the benchmark. Package validation should run before any model results are interpreted.
Keep task evidence joinable and evaluator answers isolated through an explicit package schema.
- Audience:
- Evaluation teams
- Content type:
- Integration
- Buyer question:
- What can be generated
Content release: r-a220eb6e099b243b
Related questions
- Which linked records could a generated routing dataset contain?
- How could a generated environment model a request blocked by a prerequisite?
- Can a task pack include overlapping or changing routing policies?
- Which request variations should a generated routing pack cover?
- What tool interfaces could a simulated routing environment expose?
- Which generated components belong in a complete routing world?