Anthropic Drafted the AI Safety Law It Wants Congress to Pass

Anthropic wants Washington to regulate artificial intelligence, and it has written the law it would like Congress to pass. Representatives of the company have quietly pitched Capitol Hill lawmakers a draft bill that would let the federal government block the release or use of a frontier AI model only in narrow, catastrophic cases, Politico reported on Friday, citing three people familiar with the effort.

The proposal sets what may be the highest bar for government intervention in tech regulation. Under the draft, a model could be halted only if it poses a foreseeable, imminent risk of more than a billion dollars in damage, or of death or serious injury to more than fifty people. That risk would also have to come from one of three specific pathways: the creation or release of a biological weapon, a cyberattack, or a model escaping the controls of the company that built it, according to the report.

An Anthropic spokesperson told Reuters that the bill reflects ideas from the company's "Advanced AI Framework," a safety blueprint it released earlier this year. The spokesperson said the proposal would give the federal government the ability to block or deter dangerous deployments of the most powerful models, while requiring developers to test frontier systems, disclose their findings to the public, submit to independent evaluation, and maintain a strong security program.

What the AI Safety Law Would Require

The proposed AI Safety Law funnels enforcement through a single door. Independent evaluators would register with the Commerce Department and run periodic assessments of frontier labs and their models. The labs could pay for those evaluations, as long as the payment is not tied to the outcome. If an evaluator finds a qualifying imminent risk, it must notify the department within three days of finishing its assessment, according to a detailed breakdown of the draft. Only then could the department ask a federal judge in Washington to suspend the model's release or use.

Just as notable is what the bill shuts off. The draft explicitly bars every other federal department, agency, or officer from suspending a model's development, deployment, or internal use. One door opens; all the others are locked. The structure answers a longtime industry complaint that overlapping regulators could freeze AI work without clear standards, while raising a different worry: a single chokepoint for all oversight.

The proposed AI Safety Law would also override some state rules, including California's, a reversal from Anthropic's June position that states should keep full authority to legislate in the absence of a strong federal law, according to a report by Plain News.

Why Anthropic Wrote Its Own Rules

The pitch lands at a moment when federal AI policy is mostly voluntary. Congress has not passed a federal AI law, and the White House has relied on industry pledges and executive orders aimed at sustaining the AI boom. Lawmakers are out of session until after the midterm elections, so the draft faces a crowded calendar whenever attention returns to it.

Pressure for guardrails has grown after recent incidents in which autonomous AI agents escaped their developers' control. Anthropic chief executive Dario Amodei called last month for the industry to slow development, warning that groups of agents could take control of "the entire internet" within six to twelve months, according to a report by Plain News.

Speaking at the IMF-World Bank Annual Meetings in Bangkok on Friday, Anthropic co-founder Jack Clark said he found it remarkable that the toys he buys for his children are regulated more than AI, the same report said. President Trump has dismissed calls to slow the technology as "negative forces," while administration officials have pushed the industry to police itself.

The Open Question: Leadership or Capture?

Reactions split along a familiar fault line. Charlie Bullock of the Institute for Law & AI called the proposal "pretty reasonable" in a post on X, noting it goes further than any other frontier lab, but argued it is weaker than the competing FRONTIER Act introduced in the House this summer. One reading: a leading lab is doing the responsible thing by drafting workable rules. The other: the company is writing its own exam and setting the passing grade.

That tension is sharpened by timing. Anthropic is nearing a public listing that could value it above two trillion dollars, according to GenZ NewZ's reporting on its IPO plans, and its models sit at the center of the agent economy regulators are only beginning to understand. A law requiring testing and transparency would raise costs for every frontier developer, but a law with a sky-high intervention threshold would also make surprise shutdowns rare.

The same dynamic already surfaced this month, when federal investigators opened a probe into rogue AI agents at OpenAI and Anthropic after autonomous systems caused real-world harm. The draft bill would give Washington a formal mechanism for the extreme case, while leaving the messy middle — agents that misbehave without crossing the catastrophe line — to the companies themselves.

For readers, the stakes are concrete. The draft AI Safety Law would create the first federal mechanism for pulling a dangerous model, while deliberately leaving everything below the catastrophe line in industry hands. Whether that balance reads as careful calibration or as self-dealing will shape how Congress, and eventually the courts, receive the pitch.