Claude Managed Agents IX: Stop Conversing With Your Agent. Hand It a Spec and Walk Away.
Part 9: Why an AI agent stops needing you on every turn once you replace the conversation with a spec and let a separate grader score the work until it passes.
In this article, you will learn how to turn an AI agent session from an open-ended conversation into a job with a measurable definition of done. We cover why a separate grader is more trustworthy than an agent judging itself, how to write a rubric a grader can actually score, the five evaluation results and what each one means, and how to collect the deliverable when the work passes.
Part 9 of “Building with Claude Managed Agents”
Most AI agent work is a back-and-forth where you are the quality bar on every turn. Outcomes change that: you write an acceptance test once, a separate grader scores the work, and the agent revises until it passes.
Every time you use an AI agent, you are probably having a conversation. You send a message, the agent works, you watch the stream, you steer it if it drifts, it reports back, and you decide whether the result is good enough. If it is not, you send another message. That loop works fine for exploratory work. But look closely at where the judgment lives: in you, on every single turn.
You are the quality bar, applied by hand, one message at a time. For a task you run once, that is tolerable. For a task you run every month across hundreds of items, being the human in the loop on every iteration is exactly the toil you adopted an agent to escape. You did not automate the work. You automated the typing and kept the judging.
AI agent outcomes flip that relationship. Instead of conversing toward a result, you define the result up front: a description of what you want and a rubric that says how to judge it. The agent then works toward that target on its own. And here is the part that makes an outcome more than a fancy prompt: a separate grader evaluates the artifact against your rubric, hands the specific gaps back to the agent, and the agent revises. It repeats until the work passes or runs out of iterations. You wrote the acceptance test once. The agent runs against it until it passes.
That shift, from “chat until I am satisfied” to “here is the spec, tell me when it meets it,” is the thing Claude Managed Agents is genuinely built for. One housekeeping note before the mechanics: outcomes are part of the Managed Agents public beta, reached through the standard managed-agents-2026-04-01 beta header, with no separate access request. If you can already run Managed Agents, you can run everything in this part.
Code is shown in Python and TypeScript.
The grader is a second pair of eyes, on purpose
The mechanism worth understanding first is the grader, because it is what separates an outcome from the agent simply deciding it is done.
When you define an outcome, the harness automatically provisions a grader to evaluate the artifact against your rubric. The grader runs in a separate context window, so it is not influenced by the main agent’s implementation choices. That separation is the whole point. An agent grading its own work is a notoriously unreliable judge. It knows what it meant, so it tends to believe it succeeded. A grader that never saw the agent’s reasoning, only the rubric and the artifact, evaluates what is actually there.
The grader does not return a thumbs up or down. It returns a per-criterion breakdown. For each line in your rubric, it reports either that the artifact satisfies it, or the specific gap between the current work and the requirement. That structured feedback is what gets handed back to the agent for the next iteration, which is why a good rubric matters so much. The agent does not revise against a vague wish. It revises against a checklist of exactly what is still missing.
Writing a rubric the grader can actually score
The rubric is a Markdown document of per-criterion scoring, and it is required. The single most important property is that each criterion be explicitly gradeable.
“The data looks good” is unscoreable. The grader cannot tell whether that is met. “The report contains a Difference column with numeric values” is scoreable. It either does or it does not. The grader scores each criterion independently, so vague criteria produce noisy, unreliable evaluations, while concrete ones produce a clean pass-or-gap signal.
Here is a rubric for a reconciliation report that an invoice agent produces. Every line encodes a fact about what a correct report looks like:
# Reconciliation Report Rubric
## Coverage
- Every invoice in the input folder appears as exactly one row.
- Invoices with no matching ledger entry are present and marked "unmatched".
## Discrepancy logic
- Differences with absolute value under $1.00 are not flagged as discrepancies.
- Differences of $1.00 or more are flagged with status "discrepancy".
- Each flagged row shows both the invoice amount and the ledger amount.
## Vendor verification
- Vendor name and ID in each row match the billing system record, not the invoice text.
## Output quality
- Output is a single .xlsx file.
- Columns appear in order: Invoice #, Vendor ID, Vendor Name, Invoice Amount, Ledger Amount, Difference, Status.
- Discrepancy rows are sorted to the top.
Notice that every line is a fact the grader can check against the spreadsheet. If you do not have a rubric on hand and writing one from scratch feels hard, there is a reliable shortcut: give Claude an example of a known-good report and ask it to analyze what makes that artifact good, then turn that analysis into a rubric. Reverse-engineering criteria from a real example you trust usually beats inventing them in the abstract, because it surfaces the implicit standards you would otherwise forget to write down.
You can pass the rubric inline as text, or upload it once via the Files API and reference it by ID across many sessions. The second option is the better choice once a rubric stabilizes and you want one source of truth.
Defining the outcome
You create the session as usual, then send a user.define_outcome event. The notable thing is what you do not send: there is no user.message kicking off the work. The agent begins working the moment it receives the outcome, because the outcome is the instruction.
In Python:
session = client.beta.sessions.create(
agent=agent.id,
environment_id=environment.id,
title="Reconcile March invoices",
)
client.beta.sessions.events.send(
session_id=session.id,
events=[
{
"type": "user.define_outcome",
"description": "Reconcile the invoices in /mnt/invoices against the ledger and produce a report.",
"rubric": {"type": "text", "content": RECONCILIATION_RUBRIC},
# or: "rubric": {"type": "file", "file_id": rubric.id},
"max_iterations": 5, # optional; default 3, max 20
}
],
)And in TypeScript:
const session = await client.beta.sessions.create({
agent: agent.id,
environment_id: environment.id,
title: "Reconcile March invoices",
});
await client.beta.sessions.events.send(session.id, {
events: [
{
type: "user.define_outcome",
description: "Reconcile the invoices in /mnt/invoices against the ledger and produce a report.",
rubric: { type: "text", content: RECONCILIATION_RUBRIC },
// or: rubric: { type: "file", file_id: rubric.id },
max_iterations: 5, // optional; default 3, max 20
},
],
});The description is the goal in a sentence. The rubric is how it is judged. max_iterations caps how many revision cycles the agent gets, defaulting to 3 and maxing at 20. Set it with intent. Too low, and a nearly-good report gets cut off before it is fixed. Too high, and a fundamentally stuck agent burns budget chasing a target it cannot hit. Five is a reasonable middle for a task like this.
Watching it iterate
If you are a paid subscriber, thank you. Your support makes this work possible.
If you are a free subscriber and find these articles useful, please consider upgrading. A paid subscription is $80 per year or $8 per month.
