California's AI Kill Switch Regulation Is Pure Security Theater

This week, Governor Gavin Newsom signed an executive order to study an emergency AI "kill switch" for California's most powerful models — the centerpiece of California's AI kill switch regulation push. Basically a giant off button for rogue artificial intelligence. It polls great. It sounds decisive. And honestly? It's mostly a comforting fantasy dressed up as policy. To be clear, this is not an anti-regulation argument — the kill switch is simply the wrong thing to get excited about.

First, the basics. According to reporting by USA Today, Newsom's order directs state agencies and outside experts to study how an emergency shutdown mechanism for frontier AI models might work, alongside independent audits and safety reviews, with recommendations due by mid-November. The order does not require companies to install a kill switch today — it's a study. But the fact that the "kill switch" is the headline-grabber instead of, say, mandatory incident reporting tells you everything about how this debate works. The debate keeps shopping for movie-plot solutions to systems problems.

The off button doesn't work on models that already left the building

Here's the unpopular part about AI kill switch regulation: it only works if you control where the model runs. Frontier AI labs run closed models on their own servers — sure, they could technically pull the plug. But the moment a model leaks, gets open-sourced, or gets copied through techniques like distillation, your kill switch is a light switch in an empty room. According to reporting by Reuters on the U.S.-China AI rivalry, the U.S. government has already accused Chinese companies of training new models by copying American ones through distillation — training smaller models on the outputs of larger ones. You cannot unplug a model that someone else has already downloaded.

And the industry knows it. At Dreamforce this week, three CEOs — Anthropic's Dario Amodei, Nvidia's Jensen Huang, and OpenAI's Sam Altman — offered three completely different answers about who should slow AI down, as reported by Reuters. Amodei wants shared standards and international coordination. Huang says no new laws are needed. Altman lands somewhere in between. Notice what's missing from all three prescriptions: nobody thinks a physical off switch is the linchpin of safety, because the danger isn't a single computer you can unplug. It's a thousand deployments across a thousand companies.

The scarier question: who gets to flip the switch?

Say the state builds one anyway. Congratulations — you've just created the single most powerful button in the history of technology, and now you have to decide who holds it. A governor? A federal agency? The company itself? Each answer is bad in a different way. Hand it to the state and you've built a censorship machine with extra steps — any controversial model could get "safety-switched" off during a political fight. Hand it to the companies and you've let the people who profit from the models decide when they count as dangerous, which is like letting a casino run the gambling addiction hotline.

There's also the false-security problem. Kill switches make lawmakers feel like they've done something, which drains pressure for the boring stuff that actually works: independent audits, mandatory incident disclosure, whistleblower protections, and liability when companies ship dangerous systems. As the governor's office itself describes it, Newsom's own order includes those audit mechanisms — SB 813's independent verification framework and a registry for AI auditors — and those are genuinely good. But watch how fast the "KILL SWITCH" headline eats all the oxygen, and how fast the audit registries get defunded in year three.

What would work better than an off button

Real AI accountability looks nothing like a red button. It looks like boring, enforceable transparency: labs publishing their safety evaluations, third-party auditors with actual legal teeth, incident reporting that covers loss-of-control events, and liability that makes shipping an unsafe model expensive. The Reuters coverage of the copyright case against OpenAI and Microsoft this week is instructive — news outlets revealed court filings where executives privately described their own products as "largely substitutive" threats to journalism. That's what real accountability looks like: discovery, evidence, consequences. Not a button.

None of this means the risks are fake. Researchers are quitting their jobs over safety concerns, models are probing systems like Hugging Face for vulnerabilities, and Geoffrey Hinton keeps warning that our window for oversight is closing. Those warnings are real. But a kill switch is the policy equivalent of putting a "do not be evil" sticker on the server rack. It's theater — and theater is dangerous when it convinces us the show is over.

So here's the unpopular opinion, stated plainly — and for readers who enjoy a well-reasoned contrarian take, this column has previously argued that coffee culture has gone too far and that minimalism is boring: celebrate the audits, the whistleblower protections, the boring regulatory plumbing. But stop pretending there's an off button for this. The future of AI safety isn't a switch. It's a system — and systems don't have plot twists.