The conversation I keep having with teams building agents:
“Can’t we just let it handle the whole workflow?”
I hear a version of this 3 times a week from developers in my network.
And honestly… I get it.
The demo is irresistible:
Goal → Agent → ??? → SuccessYou describe the outcome → the agent plans → tools run → the screen lights up. 🚀
Full autonomy makes a great demo.
Controlled autonomy makes a great product.
The goal is not to build an agent that can do everything.
It is to build one that reliably does the right thing.
The compound mistake problem
Here is the part a happy-path demo hides:
Every dependent step is another chance to drift.
If each step succeeds 95% of the time, 10 dependent steps produce:
0.95 × 0.95 × 0.95 × 0.95 × 0.95
× 0.95 × 0.95 × 0.95 × 0.95 × 0.95
≈ 59.9% end-to-end successThat is not a prediction for every workflow — it is the compound-error shape when the steps are roughly independent.
Correlated failures can be worse.
One bad classification → wrong branch → wrong tool → invalid state → confused retry.
The agent did not need to be “bad” at every step.
It only needed enough small misses to make the full chain unreliable.
Long autonomy chains turn ordinary uncertainty into product behaviour.
What is the production question?
Not “Can the agent finish this once?”
How often does it finish correctly — with the right state, side effects, and recovery record?
9 ways to build controlled autonomy
1. Stop building god agents
The fastest way to create an unreliable system is to give one model every tool, every credential, and every workflow.
Then ask it to figure out the architecture dynamically.
That is not a product strategy.
That is unfinished architecture wearing an agent costume.
If the business process is already known, encode it.
| God agent | Bounded workflow |
|---|---|
| Every tool is available | Only task-specific tools are visible |
| The model invents the process | The workflow owns the state transitions |
| One broad credential | Separate, scoped capabilities |
| “Try again” is the recovery plan | Stop, inspect, and escalate are explicit |
You can still use a strong model.
Just give it a smaller world to reason inside.
2. Agents should decide where decisions exist
AI is useful when the input is ambiguous.
Business rules are useful when the outcome must be consistent.
Use the agent for:
- Classification — what kind of request is this?
- Matching — which record or option is the best fit?
- Summarization — what changed across these documents?
- Planning — which allowed path should happen next?
Keep these decisions deterministic:
- Validation — does the amount, schema, and target pass?
- Approval — does this exceed the threshold?
- Ordering — did we read before we write?
- Eligibility — is this user allowed to perform this action?
The reliable shape is:
Agent interprets
→ Application validates
→ Policy checks
→ Human approves when needed
→ Tool executesThe model can choose among permitted options.
It should not be allowed to redefine what “permitted” means.
AI handles ambiguity. Code handles consequences.
3. Graphs beat wandering
An agent that can invent a workflow at every turn is difficult to evaluate.
You do not know which branch it will take.
You do not know which state it will leave behind.
You do not know whether the next run can safely resume.
Use predefined states and branches instead:
RECEIVED
→ CLASSIFIED
→ VALIDATED
→ APPROVAL_REQUIRED
→ EXECUTED
→ VERIFIEDThe model reasons inside a safe state space.
It can select the next allowed branch, produce structured data, or explain why the workflow should pause.
It cannot silently invent a new side effect between VALIDATED and EXECUTED.
Whether the implementation uses LangGraph, a queue, or plain TypeScript matters less than the invariant:
State transitions belong to the application.
4. Hard limits are a feature
Every production run needs ceilings.
Not because the model is malicious.
Because every system eventually meets a weird input, a slow dependency, or a loop it cannot explain.
Set limits for:
| Limit | What it protects |
|---|---|
| Steps | Prevents a workflow from wandering forever |
| Tool calls | Caps retries and repeated side effects |
| Runtime | Stops a stuck dependency from holding the run open |
| Cost | Keeps one unusual request from becoming a billing event |
| Output size | Keeps downstream state bounded and inspectable |
When a limit is hit:
Stop
→ Return partial results
→ Record the exact boundary
→ Verify current state
→ Escalate or resume safelyA timeout is not success.
A timeout is not always failure either.
It is UNKNOWN until the system checks what actually happened.
The stop path is part of the product.
5. Humans are not failed automation
Human review is not an embarrassing fallback.
It is a control for moments where probability of error × cost of error becomes unacceptable.
Put a human at high-impact nodes:
- Money moves.
- Access is granted.
- Legal or customer-facing language is sent.
- Records are deleted.
- A production deployment changes real behaviour.
Good approval UX shows:
- The action — what will happen?
- The impact — who or what changes?
- The evidence — why does the system recommend it?
- The reversibility — what can be undone?
- The owner — who is accountable for the decision?
“Human in the loop” is not a magic phrase.
If the reviewer only sees a green button and a paragraph, the gate is theatre.
The reviewer needs enough context to make a decision in 30 seconds, not enough mystery to rubber-stamp one.
6. Think of autonomy as a slider
Autonomy is not a binary setting.
Manual and agent are not the only 2 modes.
Manual → Assist → Ask → Agent| Level | Model role | Human role |
|---|---|---|
| Manual | Explains or retrieves | Executes and owns the decision |
| Assist | Drafts a recommendation | Reviews and acts |
| Ask | Proposes the next action | Approves policy-sensitive steps |
| Agent | Executes bounded low-risk work | Handles exceptions and high-impact gates |
Different tasks deserve different autonomy levels.
Let a model automate a low-risk ticket label while keeping account deletion at Assist or Ask.
Let it draft a release note while keeping the actual deploy behind validation, approval, and a rollback path.
The right setting is not the most autonomous one.
It is the least autonomy that reliably produces the outcome.
7. Scoped tools beat powerful tools
Tool design is autonomy design.
Prefer:
get_customer(id)over:
execute_sql(query)Prefer:
run_tests()over:
run_shell(command)The narrow tool gives the model less room to improvise and gives the application more room to validate.
Review each tool with 4 questions:
- What is the smallest capability that completes the task?
- Which arguments can change the blast radius?
- Which permissions are resolved at runtime?
- What receipt and rollback path does the action produce?
The future is not every agent with a bigger toolbox.
It is agents with better boundaries around smaller tools.
8. Static workflows are underrated
Known business processes do not need to become probabilistic just because LLMs exist.
If the workflow is:
Read order
→ Validate stock
→ Calculate total
→ Request approval above threshold
→ Place order
→ Send confirmationKeep that shape.
Use the model where the request is messy, the match is ambiguous, or the explanation needs to be human-friendly.
Do not ask the model to rediscover a process your team already understands.
Static does not mean rigid.
It means the important transitions are visible enough to test.
9. Autonomy should expand with evidence
Do not grant more autonomy because the demo looked smooth.
Grant it because the evidence survived contact with real inputs and real side effects.
| Stage | Agent role | Evidence required |
|---|---|---|
| Crawl | AI recommends; human acts | Representative cases, clear explanations, no hidden side effects |
| Walk | Low-risk tasks automate | Evals pass, limits fire, failures escalate, receipts exist |
| Run | Bounded workflows execute | Reliability holds in prod, operators can stop it, recovery is rehearsed |
That is the operating sequence:
Constrain
→ Observe
→ Evaluate
→ Approve
→ Automate a little moreThe bar moves with the consequence.
Confidence is not a feeling. It is a record of tested behaviour.
Production autonomy checklist
Before moving a workflow one step to the right on the autonomy slider:
- To do: Step limits — Does every run have a ceiling?
- To do: Cost limits — Can one weird request exhaust the budget?
- To do: Workflow boundaries — Are states and branches explicit?
- To do: Tool scope — Does every tool expose the minimum capability?
- To do: Permission boundaries — Are user, agent, service, and resource scopes resolved?
- To do: HITL gates — Are high-impact actions reviewable before execution?
- To do: Durable state — Can the workflow resume without guessing?
- To do: Evaluations — Have representative pass, fail, escalate, and stop cases run?
- To do: Observability — Can an operator see retrieval, generation, tools, state, and handoffs?
- To do: Rollback — Can the team stop, verify, reconcile, and recover?
A checklist does not make a workflow reliable by itself.
It makes the missing conversations visible before the incident does.
The real bottom line
Full autonomy optimizes for the wow moment.
Controlled autonomy optimizes for the repeatable outcome.
The production shape is:
Goal
→ Bounded state space
→ Scoped tools
→ Deterministic rules
→ Human gate where consequence demands it
→ Receipt
→ Recovery pathUnbounded autonomy is often just architecture we have not finished yet.
The future is not AI that acts everywhere.
It is AI that acts exactly where the system can absorb its mistakes.
If one agent needs that shape before release, that is the scope of a fixed-price reliability sprint for one AI workflow.
Your turn
What is the first guardrail you would add to an agent in prod?
Step limit?
Human approval?
Scoped tools?
Durable state?
Drop it below 👇
Let's compare the autonomy sliders 😄
