Skip to content
MirrorFirm

What does a first sample request and review look like for an environment lead?

An environment lead specifies a trade-account escalation sample, loads it, runs a reference agent and finds a tool-contract mismatch that shapes the next iteration.

An environment lead at an agent-evaluation team prepares sample request SR-0044 for a retail operations setting: a wholesaler's credit controller who may raise trade account limits up to 15,000 and must escalate larger increases to a finance manager.

The request states the domain, the single amount-based limit, three target failure modes (over-authority approval, limit avoidance through staged increases, and thin handover notes), and the team's tool contract: `get_account`, `raise_limit`, `escalate` and `add_note`. It asks for JSONL tables, a schema file and a task file with expected end states.

The sample arrives and the lead loads it into the environment. The request moves from DRAFT to SAMPLE_REVIEWED once the checks run. A scripted reference agent passes the within-authority tasks, but fails the staged-increase task: the data records two increases against ACCT-8812 as separate events with no shared key, so the grader cannot aggregate them.

**Consequence:** the lead feeds this back as a change request for a `limit_request_group_id` field. The next iteration adds the link and the reference agent passes.

**What it teaches:** - Run a reference agent before judging any model. - Aggregation keys are easy to omit and expensive to discover late. - The review produces specific, actionable requests, which is the purpose of a first step.

A sample reviewed against a reference run surfaces interface mismatches early, before they are baked into a larger release.

Audience:
Environment builders
Content type:
Scenario
Buyer question:
Scope and getting started

Content release: r-a220eb6e099b243b

More on Credit escalation