Can a simulated environment include tool errors and incomplete approvals?
Yes, a simulated set-up environment can be scoped to include rejected writes, timeouts, missing fields and conflicting records so agents are tested on recovery as well as the happy path.
**Can the environment return errors?**
A simulated environment can be scoped to return them. Typical examples are validation rejections for a missing mandatory field, a conflict response when an identifier already exists, a timeout on a downstream write, and a permission error on a restricted action.
**How are errors controlled?**
Each task can declare its faults: which tool, which call number, and what response. That makes runs repeatable. An agent that meets the same rejection on the same step can be compared across versions.
**What about incomplete approvals?**
Approvals can omit fields, carry conditions that are not yet met, or reference documents that are not in the document store. The task then defines what a correct response is: request the missing item, create a task for the owner, proceed partially under a stated policy, or stop.
**Can errors be transient or permanent?**
Both. A transient fault clears on retry, which tests whether the agent retries sensibly. A permanent one tests whether it stops, rolls back or hands over rather than looping.
**Will error shapes match our systems?**
Not automatically. Response formats are generated and should be mapped to the shapes your agent actually sees. Raise this in the pilot so you can compare the simulated error vocabulary with your own and adjust where it matters.
**Are faults labelled in the data?**
They can be, in a separate answer-key file, so training and scoring can use them without exposing them to the agent.
Ask for faults to be injected deliberately and labelled, so you can measure recovery behaviour instead of only successful runs.
- Audience:
- Agent developers
- Content type:
- FAQ
- Buyer question:
- What can be generated
Content release: r-a220eb6e099b243b
Related questions
- What does a generated task with a seeded fault look like for policy issue?
- What file formats, schemas and tool actions can a set-up release use?
- What records, relationships and policies can a generated set-up release contain?
- What should a buyer specify when requesting a generated set-up release?
- What should I specify about environment state before requesting a release?
- What might the record schema look like for generated set-up state?