Your AI agent should not be allowed to email your customers

The industry's position, stated as fairly as I can: agents become useful when they can complete a task end to end. Detection without action is just a more expensive alert. If an agent can identify an at-risk customer and draft the right message, making a human click "approve" adds latency without adding judgment — and at scale, that human becomes a rubber stamp anyway.

I understand the argument. We built the opposite.

In Flolyt, no agent can send a message, apply a discount, change a retry window, or issue a refund. Not with a confidence threshold. Not with an override. The capability does not exist in the permission model, and it's enforced server-side rather than by instruction.

Here's the reasoning.

The asymmetry nobody prices correctly

An agent that reads your data and gets something wrong produces a bad recommendation. A human reads it, disagrees, and moves on. The cost is a few minutes.

An agent that acts and gets something wrong has already touched a customer. You cannot un-send an email. You cannot un-charge a card. You cannot retract a discount code that's now circulating in a deals community.

The error rates might be identical. The consequences are not remotely symmetrical, and permission models built around "how often is the agent right" are answering the wrong question. The question is what happens the one time it isn't.

Causal finding this is structural. Actions that leave your system boundary and reach a third party are irreversible by definition. No amount of model improvement changes that property.

The failure mode is correlated, not random

This is the part I think the industry underweights.

Human error is roughly uncorrelated. One person makes one mistake affecting one customer. Agent error is systematic: an agent that misreads a signal misreads it the same way for every customer matching that pattern, simultaneously, at machine speed.

Consider a concrete case. A support integration goes down. Ticket volume drops to zero. An agent monitoring support load reads low volume as high satisfaction, identifies a cohort as healthy, and suppresses a scheduled retention sequence for all of them.

One bad inference. Thousands of affected customers. No human in the loop, because the action was inside the agent's autonomy budget and nothing looked anomalous — the agent's confidence was high, because from where it sat the data was clean.

This is exactly why we made unavailability a first-class value in the data layer. But the data layer is a mitigation, not a guarantee, and a permission model shouldn't assume its own dependencies are perfect.

Where it breaks in practice

The most common real-world failure isn't a dramatic hallucination. It's an agent doing precisely what it was designed to do, against a situation nobody anticipated.

• The retention agent that messages a cohort during an active outage, congratulating them on their usage while the product is down.
• The discount agent that offers a win-back to a customer who churned over a billing dispute that's still open.
• The dunning agent that escalates on an account where support has already agreed to a payment plan.
• The upsell agent that pitches an account whose renewal is being renegotiated by legal.

Every one of these is a correct decision given the agent's inputs and a bad decision given the full context — and the full context lived in a support ticket, a Slack thread, or a legal review the agent had no access to.

Strong association the agent failures we see in the wild are overwhelmingly context failures rather than reasoning failures. The model reasoned fine. It just didn't know about the outage.

The model we enforce

Three levels, declared per capability, enforced at the API boundary:

Automatic. The agent acts alone. Reserved for reversible, internal-only operations: opening a Room, pulling evidence, computing a cohort, suppressing a metric it can no longer trust, flagging a data source. Nothing here leaves the system or moves money.

Proposed. The agent drafts and requests approval. This is where anything customer-facing lives — messages, discounts, retry-window changes, segment activations. The proposal lands in a Room's Plays panel or an Inbox, with the evidence attached and its weight labelled. A human approves, edits, or rejects.

Blocked. The agent cannot do it at all, and there is no approval that unlocks it. Refunds, permission changes, data deletion, anything touching billing configuration directly.

Two implementation details that matter more than the taxonomy:

Agents hold no credentials. They cannot authenticate to your Stripe account or your email provider. When a human approves a Play, the platform executes it using a service identity, and the audit log records which human approved what, when, on what evidence.

API keys inherit their creator's permissions and can never approve. This closes the obvious hole — using an integration to grant an agent the authority it was denied in the UI.

What it costs us

We lose deals to autonomy. Prospects comparing platforms sometimes prefer the one that promises hands-off operation. That's a real, countable cost and we've decided to keep paying it.

Approval fatigue is a genuine risk. If an agent generates forty proposals a day, humans will approve them in batches without reading, and we'll have built ceremony instead of governance. Our answer is to make agents propose less and better — an agent that surfaces three high-conviction proposals a week gets read. We're not certain we've calibrated this correctly, and it's the part of the design I watch most closely.

We're slower on genuinely time-sensitive interventions. A cart-abandonment message that needs to go out in fifteen minutes doesn't fit an approval loop. For that class of automation, a rules engine in your existing tool is honestly the better fit, and we say so.

When autonomy is fine

I don't think agents should never act. Three cases where I'd support it:

Reversible internal operations. Suppressing a broken metric, opening a Room, recomputing a segment. If undoing it costs nothing, gate it lightly.

Deterministic rules the human already wrote. "Retry a soft decline once after three days" is not agent judgment — it's a policy a human authored, and the agent is executing it. The approval happened when the rule was written.

Low-stakes, high-volume, well-bounded, reversible. Tagging tickets. Routing internally. Enriching records. Nobody needs a human in that loop.

The line I'd draw is not about confidence scores. It's about reversibility and reach. Can you undo it, and does it leave your boundary? Two noes means propose, not act.

What we can't tell you yet

I don't know where this settles in three years. If model reliability improves enough that context failures become rare, the calculus shifts and I'd want to revisit it honestly rather than defend a position because we shipped it.

What I don't think changes is the audit requirement. Even in a world of highly reliable agents, "which human authorized this customer-facing action" is a question regulators, auditors, and your own board will keep asking. A system that can't answer it has a problem that model quality doesn't solve.