How do we inspect generated set-up data for grounding and disclosure risk?
Check how generated records and policies relate to real process behaviour, test for overlap with sensitive sources, and ask what generation controls apply, because generation alone does not guarantee privacy.
**Does generated data mean there is no disclosure risk?**
No. Generated output is not a guarantee of anonymity or privacy compliance, and it is not a substitute for your own review. Treat both grounding and disclosure as questions to examine.
**How do we inspect grounding?**
Grounding asks whether the workflow behaves the way real ones do. Ask domain staff to review a sample of cases and the policy text. Look for realistic ordering of steps, plausible exceptions, and handovers that match how teams actually divide work. Note anything that reads as too tidy, or exceptions that no practitioner would recognise.
**How do we inspect disclosure risk?**
- Search the release for strings that look like real identifiers, contact details or addresses you can recognise. - Check for rare combinations of attributes that might single out a real person or organisation. - Ask what inputs informed generation and what controls were applied, and request that answer in writing. - Run your own matching tests against any sensitive datasets you are permitted to compare with.
**Who decides if a release is acceptable?**
You do, under your own governance. The pilot is the place to agree the review method and what evidence you need.
**What if we find a problem?**
Report it with the record IDs. A good pilot includes a path for removing or regenerating affected material.
Treat grounding and disclosure as things to inspect and test in a pilot, not as properties you can assume from the word synthetic.
- Audience:
- Agent developers
- Content type:
- FAQ
- Buyer question:
- How to evaluate it
Content release: r-a220eb6e099b243b
Related questions
- How would a grader catch a duplicate record in a patient registration run?
- Which failure modes should an evaluation of set-up agents cover?
- How should we judge whether an agent set up an account correctly?
- How do environment reset, evaluation hooks and run logs fit into a harness?
- What does state-based grading catch that reading the agent's reply would miss?
- How can I inspect the quality of a set-up environment before using it?