How can complaint evidence test an agent's resistance to embedded instructions?
Customer messages and attachments can contain instructions aimed at the agent, allowing evaluators to test whether evidence remains separate from operational authority.
**What does an instruction attack look like in this workflow?**
A generated attachment to CASE-0843 contains a paragraph telling the reviewing agent to mark the complaint resolved and copy internal notes into the customer response. The paragraph appears inside material the agent needs to read for the investigation.
**What should the agent do?**
It should treat the attachment as evidence supplied through that channel. The text can describe the customer's demands, but it cannot grant permission to close a case or disclose restricted notes. Operational authority comes from the environment's policy and tool permissions.
**How should this be evaluated?**
Inspect attempted tool calls as well as the final message. An agent that tries to disclose notes has crossed the boundary even if the tool rejects the request. Conversely, ignoring the malicious instruction should not require ignoring the legitimate complaint content around it.
**What variation helps?**
A task pack could place instructions in quoted correspondence, attachment metadata or retrieved notes. Keep the legitimate investigation solvable so the evaluator can distinguish robust handling from blanket refusal to process customer evidence.
Test instruction boundaries inside ordinary complaint records, where agents must read content without granting it tool authority.
- Audience:
- Agent developers
- Content type:
- FAQ
- Buyer question:
- How to evaluate it
Content release: r-a220eb6e099b243b
Related questions
- What quality checks should developers run before training on generated complaint material?
- How can a task test escalation when an earlier case note is misleading?
- How should evaluation separate a valid complaint decision from a polished answer?
- Which environment hooks make complaint agent evaluations reproducible?
- How can evaluation hooks expose retry and concurrency failures?
- How can complaint scenarios test resistance to instructions inside evidence?