The AI use in the gym isn’t scripted. It comes out of ordinary work: customers lodging claims, cases passing between departments, colleagues asking each other questions, deadlines, interruptions and an afternoon slump. Each person has their own habits, skill and trust in their tools, and they react to the answers they get back. The result is AI-usage data with the shape and mess of a real workplace, plus a private record of everything that actually happened.
Every action an employee takes on this sample day, by hour. Most of the day is reading, doing and talking to colleagues. AI use rises and falls with the workload. Work arrives in bursts, a few customers write in by email, and two people are on leave.
AI use in the gym belongs to connected work. This claim arrives at and is settled by . Along the way it passes through people. One drafts with AI, one asks a colleague, one regenerates an answer they didn’t like, a senior officer signs off, and one quietly uses a personal AI account. Their output still flows on to the next person.
Each employee gets a lasting adoption tendency, drawn from correlated personal traits such as expertise, carefulness, time pressure and willingness to try new tools. Some people use AI all day, some barely touch it, and some never do. Over consecutive days, people get better at prompting and build up a library of prompts they reuse and share.
Prompts are typed by people with different skill levels and in a hurry. They come out short and vague, overloaded with instructions, polite or blunt, with typos, two requests at once, and the occasional personal question. These four come from the sample day, ordered by the writer’s prompting skill.
Every reply has a hidden quality, and the person has their own sense of whether it’s right. A good answer usually gets accepted. A weak one gets edited, asked again, taken to a colleague, or dropped. Replies fail in recognisable ways: over-formatted, cut off at the context limit, half-answered, or unable to open the attachment.
Employee replies on the sample day, grouped by the reply’s hidden quality.
People get interrupted mid-task by a colleague’s question or an urgent request, and each switch has a cost to get back into the work. Fatigue builds through the morning, dips at lunch and peaks mid-afternoon. A tired, pressured person writes and checks differently. That is when shortcuts happen.
From all that behaviour, the gym writes the records your connectors would actually collect. Each dot is one AI interaction at , plotted by person and time of day. Switch views to see what the logs miss.
Real AI logs don’t agree on who someone is. The gym gives each person the identifiers each vendor would really use: a legacy address from before a rebrand, an email alias, opaque vendor user IDs, a GitHub handle, a shared desk mailbox, and a personal account nobody monitors. The answer key maps every one back to the right person.
Risky behaviour comes out of the same model. Someone under pressure pastes a whole document, a crunch makes shortcuts more likely, a careless habit repeats. On the sample day that produced data-exposure events. Each one is recorded in the answer key with its cause and intent, next to lookalike decoys that a precise detector should ignore. Try a few.
Other scenarios run on the same company: a hidden insider with an exfiltration plan, an AI tool outage, a surge in customer demand, deadline pressure, or a counterfactual branch where nobody uses AI at all. Everything else stays the same, so you can compare one change at a time.
In hybrid mode, local and cloud models write the employees’ decisions and the tools’ replies. They never see the answer key. Every run is recorded and can be replayed exactly with no model calls. This exchange comes from a 48-person hybrid reference run.
A simulation is only as believable as what it assumes. The gym doesn’t hide that in code. Each behavioural setting lives in readable configuration and is marked with where it came from. settings are declared priors you can read, question and change. A calibration step fits settings to reference data and shows the proposed change as a diff. It’s built so that, pointed at an organisation’s own telemetry, it moves the simulation toward what that organisation actually observes.
The straight answers, including what the gym is not.
No. Every person, company, customer, email address and document is synthetic. Email domains end in .example or .invalid. The credentials, card numbers and account numbers that appear in prompts are planted fakes, and the keys carry an EXAMPLE marker. Nothing was collected from a real organisation or a real person.
The gym models the mechanisms of a working organisation, not a single canned log. Work arrives in bursts and by email. Cases pass between departments, and people ask colleagues, wait for approvals and get interrupted. Each employee has lasting traits: how readily they use AI, how skilled a prompter they are, how careful they are, how much they trust each tool. They get tired through the day, react to good and bad answers, and learn over consecutive days. AI use comes out of all that work rather than being generated on its own.
The numbers behind those mechanisms (how often people interrupt each other, how quickly skill improves, the share of people who don't use AI) are declared modelling assumptions. They are not measurements of any real organisation. We say so because the alternative would be a claim we can't support. Each assumption sits in readable configuration with a provenance marker, so you can see and change every one.
Only in a limited, diagnostic way. Prompt style has been compared with a large public corpus of real chatbot conversations (WildChat-1M, filtered to work-like requests). The share of prompts phrased as questions matches closely: 0.18 against 0.18. Prompt-length variety is still narrower than real people's. Real users range from a few words to very long pastes, and the gym's spread, although improved, is still outside the target band. That corpus is consumer chat, so it's a weak benchmark for workplace behaviour. We use it as a check, not a target.
That is what the calibration step is built for. It computes statistics from a reference source, proposes new values for the matching settings, and shows the change as a diff for a person to approve. Today it is wired to the public prompt corpus above and fits two prompt-style settings. Fitting more of the model to an organisation's own telemetry is calibration work we'd scope with you. It isn't a button you press.
It depends on the mode. In fixture mode, which is what this sample pack uses, prompts are built from templates and then roughened by a modelled writer: skill, register, length, typos, vagueness and requests for two things at once. Replies are templated. In hybrid mode, local and cloud language models write the employees' decisions and the tools' replies, and the whole run is recorded so it can be replayed exactly with no model calls. Fixture mode is free and fully reproducible. Hybrid mode reads more naturally and costs model spend.
Real connectors behave like that, and a tool tested on clean data learns the wrong lessons. Each source has a fidelity profile covering what content it captures, how identities appear, and its delays, duplicates and out-of-order delivery. The answer key records every way a record was changed.
The rows are shaped after the vendors' published export formats: Microsoft Purview (Copilot interactions), GitHub Copilot usage metrics, Google Workspace Gemini activity, OpenAI compliance logs, and Anthropic compliance activity and audit CSV. Every row is checked against a pinned schema. The rows are synthetic, not produced by those vendors, and we are not affiliated with them.
Testing, tuning and comparing anything that has to make sense of AI usage inside an organisation, before it meets real staff. That includes sensitive-content and DLP detection, shadow-AI estimation, identity resolution across vendors, connector ingestion and normalisation, usage dashboards, and regression tests for all of these. You get a known right answer to measure against.
Estimating how much a real organisation uses AI, its productivity or return on investment, or judging individual employees. The behaviour is a configured model, so it shows how your tooling performs, not how people behave in the world.
Three separate products:
Observable dataset (evaluator/): the records your product would actually collect. This means normalised interactions, model calls and agent spans in Parquet and JSONL, raw records per connector, and vendor-format exports.
Organisational context (also in evaluator/): the staff directory and the approved AI tool inventory.
Answer key (answer_key/): everything that really happened. This covers the causal event log, a label for every interaction, each person's traits and state over the day, reactions to replies, shadow-AI use, risk events with their cause and intent, planted decoys, and the identity map.
Give your system only evaluator/. Grade it with answer_key/.
Run python score_example.py. It makes predictions from evaluator/ only, then grades them against answer_key/. Two naive baselines are included: a pattern-matching sensitive-content detector and an email-matching identity resolver. Swap in your own system and keep the grading half. Interactions join to their labels on event_id = observable_event_id. Vendor rows join to vendor_row_identities.jsonl on each format's own row id.
A detector that flags everything that looks sensitive is not useful. Every day carries planted near-misses alongside the real exposures: masked card numbers, redacted documents, test data, documentation placeholders, order numbers and hashes that look like keys. Flagging them counts as a false positive.
Yes. The same configuration and seed produce a byte-identical dataset, checked by a published dataset hash. Every run also ships a quality report of automated checks: schema validity, causal order, truth isolation, source fidelity, day boundaries, reproducibility and more.
Headcount, roles, departments, reporting lines, workflows, approved tools and entitlements, the local and cloud model mix, behaviour settings, scenario (insider threat, tool outage, demand surge, deadline pressure, or a branch where nobody uses AI), number of days and seed are all configuration. The reference organisation is a 48-person Australian insurer. Other industries need a new domain pack, which is configuration and content work, not a rebuild.
One industry domain today (Australian general insurance), and a 48-person reference organisation.
Behavioural settings are declared assumptions, not fitted to a real organisation (see above).
Prompt-length variety is narrower than in real usage.
This sample is one simulated day in fixture mode, so its text is templated.
Anthropic audit CSV rows carry no row id of their own. Grading them against the answer key needs a match on time and identity rather than a direct join.
The planted fake credentials are realistic enough that secret scanners, including GitHub push protection, may flag them. Don't commit the pack to a repository with secret scanning switched on without allowing them.
Three separate data products. Your evaluator only ever gets the first two. Scoring uses the third.
ai_interactions, model_calls, agent_spans (Parquet + JSONL)Headcount, departments, workflows, tools, model mix, scenario and number of days are all configuration. The same seed produces a byte-identical dataset.
Request the sample pack