.png){kind=logo}
Anthropic has urged California Governor Gavin Newsom to give government officials the power to seek court orders blocking artificial intelligence models that pose serious risks to the public. The proposal comes as US policymakers consider stricter rules for advanced AI systems and ways to prevent them from causing harm.
In a letter dated October 15, the company said regulators should be able to intervene when safety reviews identify a model as a serious threat. Anthropic's recommendations could influence California's approach to AI regulation, which may also shape policy discussions in other US states.
Anthropic submitted its recommendations after Newsom convened a panel of AI experts in September to develop proposals for safety requirements. The company supports independent safety evaluations and regular reporting on the risks advanced AI systems pose.
Under its proposal, external evaluators should be able to report serious concerns directly to California's attorney general. If officials identify an immediate risk of catastrophic harm, they should be able to seek a court order suspending a model's deployment, internal use or further development.
Anthropic also supports publishing risk reports at least every six months for large AI developers. The company believes independent scrutiny would help regulators identify potential problems before they cause significant damage.
Anthropic supports emergency controls for AI systems but has raised concerns about rules requiring companies to build mechanisms that can shut down entire models.
Instead, it favors more targeted measures, such as disabling a malfunctioning AI agent or restricting a particular customer's access. The company argues these interventions could address specific problems without unnecessarily taking an entire system offline.
Anthropic has said a full shutdown should be reserved for extreme situations. Its approach focuses on continuously monitoring AI systems and responding to problems where they occur.
Also Read: Anthropic IPO: What We Know About Its Planned Nasdaq Listing
The proposal comes as US lawmakers debate whether the government should have authority to block the release of AI models considered unsafe. A bipartisan group of senators discussed such legislation in September, but the proposal had not advanced before Congress's pre-election break.
The US House also considered the AI Kill Switch Act, which would give officials the power to order AI companies to shut down systems that threaten human life or the economy.
Anthropic's recommendations add to the wider debate over how governments should balance technological progress with public safety. The challenge for policymakers will be establishing clear standards for intervention without unnecessarily restricting the development of useful AI technologies.
