Agentic Workflows Need Adult Supervision

by Josh Oransky

The agent demos are very compelling. A natural-language goal goes in, several minutes of autonomous work happen, and a thing that mostly works comes out. Then it ships to production and the questions start. Who reviewed it? What did it actually do? What if it does something else next time? This is the part the demo never shows. It is also the part that decides whether the workflow ships.

What "agentic" actually means in production

An agentic workflow is one where a model plans its own actions, executes them through tools, observes the result, and re-plans based on what it found. The autonomy is the point. It is also the problem. The same property that lets the agent solve novel problems lets it take actions you did not expect.

Three categories of action map to three different supervision regimes.

Reversible and low-impact. Searching a corpus, drafting a doc, proposing a tag set. The agent runs freely. The output gets reviewed by a human before it ships to a customer surface, but the agent doesn't need a gate before acting.

Reversible and high-impact. Editing content in the CMS, modifying a knowledge base, sending an internal notification. The agent proposes the action. A human approves it before execution. The approval step has a one-click audit trail.

Irreversible. Sending an external email, posting to social, publishing customer-visible content, calling a paid API at scale. The agent does not execute. It produces a structured request that a human runs from a separate interface.

Guardrails that actually catch things

"Add guardrails" usually means "write a system prompt that asks the model nicely not to do bad things." That is not a guardrail. A guardrail is something the model cannot route around.

• Tool allowlists. The agent has access to these specific tools and no others. Anything else is outside its capability surface, by design.
• Rate limits per action class. Reading 1000 documents is fine. Sending 1000 emails is not. The limit lives in the runtime, not the prompt.
• Spend caps. The agent's budget for the run is finite. When it hits the cap, it stops and returns to the supervisor.
• Audit logs that someone reads. Logging every tool call is necessary. Reviewing the logs weekly is what catches drift.

The point of supervision

Adult supervision doesn't mean "the human does the work." It means "the human owns the outcome." The agent is a force multiplier, and the human stays accountable. Skip that and you end up with workflows nobody can explain when they go sideways.

The teams shipping agentic work in production today are not the ones with the most autonomous agents. They are the ones who decided early which actions the agent can take alone, which it can propose, and which it can only request. That decision tree is the product. The model is just one component of it.