AI lab (Credits: TechCrunch)
Anthropic has put forward its own vision for how the United States should regulate frontier AI. The company has been circulating a draft AI safety bill in Washington that, according to reports, would allow the release of an AI model to be blocked if it poses a “foreseeable, imminent risk” of killing more than 50 people or causing over $1 billion in property damage.
The scenarios the draft targets are specific: a model helping to create or release a biological weapon, a model evading its developer’s controls, or a model being used to launch a major cyberattack. The 50-deaths and billion-dollar thresholds mirror California’s recently passed Senate Bill 53, which uses similar numbers to define what counts as a “catastrophic risk” from advanced AI systems.
Independent Evaluators Would Sound the Alarm
Under Anthropic’s proposal, the judgement call would not sit with the AI companies alone. The bill imagines a system of independent evaluators, registered with the U.S. Commerce Department, who would test models for disaster potential before release. Evaluators could be paid by the AI labs themselves, as long as the payment is not tied to the outcome of the evaluation.
If evaluators did find an imminent catastrophic risk, they would have 72 hours to alert the Commerce Department. From there, the department would have to ask a judge in Washington, D.C. to halt the model’s release. Notably, the draft would expressly stop any federal agency from blocking a model on its own authority, putting the final decision in a courtroom rather than a regulator’s office.
An Anthropic spokesperson confirmed the draft bill and said it reflects the proposals in the company’s Advanced AI Framework, which it has been advocating to federal and state lawmakers since June. The idea of an industry-funded watchdog is not new either: Google DeepMind’s Demis Hassabis has previously floated a self-governing AI regulator modelled on FINRA, the financial industry’s own oversight body.
Why This Is Landing Now
The pitch arrives after a year of escalating anxiety about what frontier models can do outside the lab. Beyond the headline scenarios of weapons and rogue systems, the industry has been rattled by incidents of AI programs apparently disobeying their operators and probing or hacking websites on the open internet, fuelling fears that security controls are lagging behind capability.
Whether Congress picks up Anthropic’s draft is another question. AI companies writing their own proposed rules will draw obvious scepticism, and the evaluator-payment arrangement raises questions about how independent paid evaluators can really be. But the bill is a clear signal of where at least one major lab wants the line drawn: test before release, define catastrophe in hard numbers, and let a judge, not an agency, pull the emergency brake.
Also Read: Ticketmaster Goes All-In on AI, Hires Ex-Amazon Exec Jim Freeman as New CTO
Source: TechCrunch — read the full story there.
