What should an evaluation team request inside a generated task pack?
A task pack specification should cover agent inputs, permitted actions and evaluator-only material, including cases where no record change is justified.
Specify these pack components together so each generated case has an executable task, a defined information boundary and enough evaluator material to support a defensible result.
- An agent-facing request that states the business concern without revealing the expected correction, preserving the need to investigate evidence and choose an action. - An initial record snapshot containing the relevant account status, outstanding requests and available supporting material at the moment the agent receives the task. - A role profile describing what the acting agent can inspect, propose and execute, with permissions expressed separately from the requester's preferred outcome. - A task-specific action catalogue explaining required arguments and observable results, including how rejected actions appear to the agent during the evaluation run. - Evaluator-only outcome constraints describing acceptable final states and forbidden changes, allowing several valid action sequences where the supplied policy permits them. - A rationale annotation connecting each expected decision to the evidence that supports it, enabling reviewers to challenge an answer key rather than trust it automatically. - Negative cases in which the suspected discrepancy is resolved without an adjustment, preventing the pack from teaching agents that every correction request requires a posting.
Define the contents needed to run and score a task before assessing the writing quality of individual records.
- Audience:
- Evaluation teams
- Content type:
- Checklist
- Buyer question:
- What can be generated
Content release: r-a220eb6e099b243b
Related questions
- Which linked records could a generated correction dataset contain?
- Can generated tasks include policies that changed between the original entry and its correction?
- What files and environment state could a generated correction release include?
- What evidence conditions should a generated correction pack vary?
- How could a generated task represent a correction that must be split across destinations?
- Which schemas and tool contracts can represent a generated financial adjustment?