Kill Switches for Autonomous Agents

In Part 2 I described the agent I run unattended: a daemon that wakes every 15 minutes, reads the state of my projects, does a bounded slice of work, and writes down what happened. That post spent one section on guardrails. This is that section, expanded into the thing I’d actually hand someone before they turned their own agent loose.

Here’s the reframe that should change how you build it. An autonomous agent is not a smarter chatbot. It’s a process that holds three dangerous capabilities at the same time: a paid model API key (it can spend), a shell (it can execute code), and outbound channels that speak as me (it can impersonate and make noise). Any one of those, unsupervised, is a liability. All three in one always-on process is a different category of risk.

So the security question is not “can it be jailbroken.” That’s the question you ask about a model you’re serving to strangers. The question for an agent you run yourself is blunter: what is the blast radius when it misbehaves, because eventually it will. Not because the model is adversarial, but because software running thousands of times a day on real-world inputs eventually hits the input you didn’t plan for. You design for the misbehaving cycle, not the well-behaved one.

This post is the controls that bound those three capabilities. None of them are exotic. All of them are load-bearing.

The Threat Model: Key, Shell, Send

Before the controls, get the threat model right, because the controls only make sense against it. Three capabilities, three distinct failure shapes.

The key — it can spend. The agent holds a credential for a paid model API. A retry loop, a runaway plan, or a quiet cache regression can turn “a few cents a cycle” into a bill that climbs while no one is watching. The damage is financial and it compounds silently. There’s no error, no crash, no alert by default, a meter running.

The shell — it can execute. The Act stage runs commands. A shell is the most general-purpose capability there is, which is exactly why it’s the most dangerous to hand to a process acting on its own judgment. The damage here is to your filesystem, your repos, your machine, and it can be irreversible in a way a bad message never is.

The channels — it can speak as me. The agent sends messages, opens PRs, posts updates under my identity. When it misbehaves here the damage is reputational and relational: noise to real people, an embarrassing or wrong message with my name on it, or (the failure I’ll get to below) a loop that spams.

There’s a unifying principle underneath all three, and it’s worth stating before the specifics: build the OFF switch before you build the capability. Not after the first incident. Before the first run. If you can’t articulate how you’d stop a capability instantly, you’re not ready to give it to an unattended process.

Kill Switches That Actually Kill

Start with the simplest control, because it’s the one people implement wrong most often: a kill switch is only a kill switch if flipping it actually stops the thing. The common failure is a switch that’s checked at startup, or behind a deploy, or in one process but not the others—a config flag with good intentions, not a kill switch. When you need to stop a misbehaving agent, you need it stopped now, not at the next restart and not after a five-minute pipeline.

So the contract I hold is specific:

• Every outbound path gates on the flag before it fires. Not most paths. Every send, every external action, checks the kill-switch state at the moment of sending. A path that doesn’t check is a path that can’t be stopped.
• The flag lives in shared state, read live. It’s a key every process reads on each send, not a value baked in at launch. That’s what makes one flip take effect across every process at once.
• It takes effect within about a second, no deploy. Flipping it is a hard “stop”: set the key, and within roughly a second every sender across every process honors it. No rebuild, no redeploy, no restart.
• One command to pause, one to resume. The whole point is that you can reach for it in the moment without thinking. A pause that requires a runbook is a pause you won’t use fast enough.

The build order is the part I want to nail down. For any new outbound capability, the off switch ships first. Before the agent can send on a new channel, the thing that stops it from sending on that channel exists and is tested. The capability comes second. This sounds pedantic until the first time you need it, and you will need it, usually at an inconvenient hour, and you will be very glad the stop button was already there and already worked.

Hard Spend Caps, Fail-Closed

Now the key. An unattended agent that holds a paid API credential needs a hard ceiling on spend, and the design detail that matters more than the ceiling itself is what happens when the system is uncertain.

The budget kill switch trips automatically when a ceiling is breached. That’s table stakes. The non-negotiable property is that it fails closed: if the system cannot confirm you’re under budget, it stops spending rather than assuming the best. Uncertainty resolves to “stop,” not “probably fine.”

This is the inversion most people get backwards. The intuitive default is fail-open: if the budget check errors out, keep working so the agent stays useful. That’s exactly wrong for an unattended spend path. The whole reason you built the cap is that no one is watching the meter. A fail-open cap protects you only when everything is healthy, which is precisely when you didn’t need protecting. The cap has to hold hardest when the system is confused, because a confused always-on agent with a live credential is the scenario that empties an account.

Freshness-gated calibration

Here’s the subtle failure I learned the hard way. A spend check often leans on some estimate or calibration: a ratio that maps raw usage to actual dollars, a recent measurement of what cycles cost. That input is itself a liability if you trust it blindly.

The rule: any estimate feeding the budget check must be freshness-gated. You only trust it if two things hold:

1. It’s recent enough. A calibration from last week may describe a regime that no longer exists. A pricing change, a model swap, or a caching change all invalidate it.
2. Its measurement window overlaps the current one. A measurement of a different period isn’t evidence about now. If the windows don’t overlap, the number is describing a different world.

When the input fails either gate, you don’t extrapolate from it. You fall back to raw numbers and let the kill switch act on those alone. Because here’s the trap: stale “we’re fine” data is worse than no data. No data makes you cautious. Stale reassurance makes you confident on bad information. It green-lights spend that the current reality wouldn’t justify. A budget control that trusts a comfortable old number is a budget control that fails exactly when the world has changed underneath it.

A watchdog on the switch itself

One more layer, because a kill switch is only as good as its integrity. What stops the budget switch from being silently cleared by a bug, a race, or a well-meaning process that “resets” state? A separate watchdog guards the kill switch itself. It watches for the pause flag being cleared without authorization, and if it sees a silent clear, it auto-reverts it. The control protecting your spend gets its own control protecting it. You don’t want the discovery that your budget cap was quietly switched off to come from the invoice.

A Real Sandbox Around the Act Stage

Now the shell, the broadest capability and the one that earns the most paranoia. The Act stage, where the agent actually executes its plan, runs under concrete limits. Not “the model knows not to do anything dangerous.” Limits enforced by the system, not by the model’s good judgment, because the model’s judgment is the thing under test.

Allowlist-only shell. The agent can run a fixed, known set of binaries, nothing else. Not “block the dangerous ones” (you’ll never enumerate them all), but “permit the safe ones.” On top of that: no shell metacharacters, so a command can’t smuggle a second command past the allowlist, and a hard rule against piping into rm, sudo, or eval. The whole class of “the model composed a clever destructive one-liner” is closed structurally, not hoped against.

Restricted write paths. File writes are confined to a couple of explicitly-blessed directories. The agent’s working scratch space and its own knowledge store, and that’s it. It cannot write to source it isn’t supposed to touch, cannot edit its own guardrails, cannot reach into the rest of the machine. The blast radius of a bad write is bounded to directories you’ve decided you can afford to lose.

SSRF protection. Any URL the agent fetches is checked first: requests to localhost and private IP ranges are blocked. This is the control that stops the agent from being steered, by a crafted input or a poisoned document or a manipulated plan, into hitting your internal services, your cloud metadata endpoint, your private network. An agent that can fetch arbitrary URLs is one prompt away from being a confused deputy inside your perimeter. The egress filter is what keeps “fetch this URL” from becoming “exfiltrate from the service next door.”

Per-process secret resolution. Secrets are resolved at the moment of execution, in the specific process that needs them, never injected session-wide into the environment. This one’s easy to get lazy about because session-wide injection is so convenient. But a credential sitting in the ambient environment is a credential that leaks into every log line, every crash dump, every child process, every error report. Resolving per-process at execution time means a leaked or logged environment is not a handed-over credential. The secret exists for the moment it’s used and isn’t lying around the rest of the time.

Why allowlist, not denylist, and why it isn't only for the shell

The deepest reason the sandbox is allowlist-shaped is that denylists lose. To block dangerous behavior by enumeration, you have to think of every dangerous thing in advance, and an agent generating commands from a language model will find the one you didn’t think of. Allowlists invert the burden: instead of imagining every attack, you enumerate the small set of things the agent legitimately needs, and everything else is denied by default. You’re far more likely to have a complete list of “what this agent does” than a complete list of “what could go wrong.”

The same shape shows up across every control in this post, which is not an accident:

• Spend — fail closed: deny the spend unless you can affirmatively confirm it’s under budget.
• Shell — allowlist the binaries: deny every command unless it’s on the known-good set.
• Writes — blessed directories only: deny every path unless it’s explicitly permitted.
• Egress — block private ranges: deny internal destinations by default.
• Outbound — kill switch gates every send: deny the send unless the switch says go.

The unifying instinct is default-deny, permit explicitly. For an unattended process acting on a model’s judgment, that’s the only posture that’s robust to the input you didn’t anticipate, which eventually is every input.

One Sender Per Channel: A Cautionary Tale

The outbound channel has a failure mode the other two don’t: it can feed back into itself. And the cleanest way to explain the rule is to tell you how I learned it.

The lesson generalized into a rule I now treat as non-negotiable: exactly one sender per channel, with cross-process deduplication in shared state. Two halves, both required.

One sender per channel. For each outbound channel, exactly one process is permitted to call the send API. Not “primary and backup,” not “whichever is up.” One owner, full stop. The moment two processes can both send on a channel, you’ve created the conditions for the echo loop, and “they probably won’t collide” is not a security property.

Cross-process deduplication in shared state. Before any send, the would-be sender checks shared state: has this message already gone out? The dedup record lives somewhere every process can see it (a shared store with a short time-to-live), not in a per-process ring buffer. This is the part people skip, and it’s the part that actually matters on a shared substrate.

Per-process safety is insufficient for one plain reason: a per-process dedup ring only knows what that process has sent. It is structurally blind to what a sibling process did. On a single machine running one process, that’s fine. The instant you have concurrent processes (multiple agents, multiple sessions, a failover that didn’t fully fail over), each one’s private memory of “what I’ve sent” is an island, and the collisions happen in the water between the islands. The coordination has to live in shared state because the problem is a coordination problem. You cannot solve a multi-process race with a single-process data structure.

This is the shared-substrate trap in miniature, and it’s worth internalizing because it generalizes far past messaging. Any contended resource (a channel, a queue, a credential, a lock) that multiple agents can touch needs exactly one owner or shared coordination, ideally both. Per-process correctness is necessary and nowhere near sufficient. Every “the agents went haywire together” story I’ve heard reduces to a missing boundary exactly like this one: two things that should have been one, or coordination that lived in the wrong place. Draw the boundary first.

The Trust Ladder: Shadow Before Real

The controls above bound what the agent can do. The trust ladder bounds what a new, unproven automation is allowed to do, and it’s the control that makes adding capabilities safe over time instead of a series of held breaths.

The principle: new automations don’t get to act for real on day one. They run in shadow first. In shadow mode, an automation does everything except the irreversible part. It observes, it decides, it proposes the exact action it would take, and it logs that proposal. But it doesn’t execute. You get a running record of “here’s what I would have done” with none of the consequences of having done it.

Then it graduates, but only on evidence. An automation moves from shadow to acting-for-real after it’s demonstrated a track record of correct proposals: a threshold of real successes, measured, not vibes. Promotion is earned, not granted because the code looks finished and the author is confident. “It seems to work” is the start of the evaluation, not the end of it.

This matters more than it sounds like it should, because the failure shape of an autonomous agent is different from a one-shot tool. An assistant’s mistake is one bad suggestion you reject and move on. An agent’s mistake repeats every cycle until someone notices. It’s confidently wrong, at scale, on a clock. Shadow mode is how you discover that an automation is confidently wrong before it’s wired to consequences. You watch its proposals fail harmlessly in the log instead of watching its actions fail expensively in production. You learn its failure modes on your terms.

The Watchdog: Bounded Restarts, Operator Intent

The last control is the one that catches the failure the other controls let through: a service that’s broken in a way that makes it keep dying and coming back. You want crashed services restarted; that’s basic resilience. What you don’t want is a genuinely broken service flapping forever, burning resources and noise, with an automatic restart loop papering over a real problem indefinitely. The watchdog that keeps background services alive does two things that make it safe rather than persistent.

It caps restarts per hour. A service is allowed a small number of restarts in a window, three per hour in my setup. Cross that threshold and the watchdog stops trying. The service quarantines, and the situation escalates to me. The reasoning: a service that’s died three times in an hour is not having bad luck, it’s broken, and the right response to “broken” is a human looking at it, not an infinite loop of hopeful restarts. Bounded restarts convert “flap forever, silently” into “try a few times, then get a person.” The cap is what turns resilience into a signal instead of a mask.

It respects operator intent. If I’ve deliberately disabled a service, taken it down on purpose for a reason, the watchdog must not “helpfully” bring it back. An automation that fights a human’s explicit decision is worse than no automation, because now you’re wrestling your own infrastructure. So before any restart, the watchdog checks whether the service was intentionally disabled, and if it was, it leaves it alone. The rule generalizes past this one watchdog: automation should never override a deliberate human action. Don’t fight the operator. The human’s explicit intent is the highest authority in the system, and every automated control should treat it that way.

Putting It Together

Step back and the shape is simple, even if the individual controls have teeth. Three dangerous capabilities, each bounded:

• The key (spend) — a budget kill switch that trips on breach and fails closed, fed only by freshness-gated calibration, guarded by a watchdog that won’t let it be silently cleared.
• The shell (execution) — an allowlist-only sandbox: known binaries, no metacharacters, no piping into destructive commands, writes confined to blessed directories, SSRF-guarded fetches, per-process secrets.
• The channels (impersonation) — kill switches that halt every send in about a second with no deploy, exactly one sender per channel, and cross-process dedup in shared state.

And two meta-controls that keep the boundaries honest over time:

• The trust ladder — new automations run in shadow and earn the right to act with evidence, not confidence.
• The watchdog — bounded restarts so broken services escalate instead of flap, and a hard rule never to override deliberate operator intent.

The thread tying all of it together is one instinct: default-deny, permit explicitly, and build the OFF switch before the capability. Fail closed on spend. Allowlist the shell. Block private ranges. Gate every send. Shadow before real. Each one is the same move: deny by default, permit the narrow known-good set, and make uncertainty resolve to “stop.” That posture is the only one that survives the input you didn’t anticipate, which over a long enough unattended run is every input.

One honest caveat to close on. None of this makes an agent safe in the absolute sense. It makes the blast radius bounded and known. That’s the actual goal, and it’s a more useful goal than “safe,” because “safe” is a claim you can’t verify and “bounded” is a property you can design and test. I can tell you exactly how bad my worst case is (which directories a bad write can reach, what my spend ceiling is, how fast I can stop every send), and I designed each of those bounds on purpose. That’s what lets me leave the agent running while I sleep. Not faith that it’ll behave. A known ceiling on what happens when it doesn’t.

The companion piece to this is cost: the spend cap here is the hard backstop, but the thing that keeps the agent cheap before the cap ever trips is prompt-cache discipline, and that fail-closed budget logic gets the full treatment in Prompt Cache Invariants. The architecture and the security model are two halves of the same constraint.

Running an agent that holds your keys, or about to? Tell me which capability scared you into building a control, and where the boundary turned out to be in the wrong place. The blast radius is where the real lessons are.