Anthropic tells California governor that courts should be able to block AI models posing serious risks, and questions “kill switches”
In an October 15 letter to California Governor Gavin Newsom, US artificial intelligence company Anthropic said government officials should be able to seek court orders blocking the release of AI models that pose a “serious risk” to the public. The recommendations came after Newsom convened a panel of experts in September to develop new AI safety requirements for the state. Anthropic also expressed scepticism about requiring “kill switches” capable of shutting down entire models, saying they could provide a “false sense of reassurance”, and backed more targeted interventions. The company itself faced US government restrictions on some of its models in June. The letter comes as Congress also weighs giving the federal government power to block models deemed unsafe, and as Anthropic prepares for an initial public offering.
Edite QUANG MINH·2 min read · 4 min listen·28 October 2026, 22:18 EDT·
AI-generated illustration
A letter to Governor Newsom
According to the letter, when a review concludes that a model poses a serious risk, the government should have a legal tool to intervene before the model is released. The proposal echoes policy recommendations the company has made before.
The government should be able to ask a court to block it.
An Anthropic spokesperson confirmed the letter was authentic, while the California governor’s office did not immediately respond.
Advertisement
300 × 250
On October 20, OpenAI also sent its own list of recommendations to Newsom. Politico first reported the details of the letter.
Doubts about “kill switches”
One idea under consideration by California’s expert panel is requiring developers to build “kill switches” capable of shutting down entire models if a company loses control. Anthropic argued this could give officials a false sense of reassurance, and proposed other measures:
Disabling a specific problematic AI agent rather than shutting down the whole model.
Cutting off access for a specific customer misusing a model.
Allowing authorities to ask a court to block the release of a model assessed as posing serious risk.
According to Anthropic, such targeted interventions would be more effective. The debate reflects a difficult question in AI governance: how to make control tools both technically feasible and genuinely protective of the public.
The legislative race in Washington
At federal level, a bipartisan group of senators in September debated legislation that would allow the US government to block the release of certain AI models deemed unsafe, while letting companies challenge that decision in federal court. However, Congress left for its pre-election break shortly afterwards and the proposal has not advanced.
In July, House lawmakers also proposed the “AI Kill Switch Act”, which would give officials power to order AI firms to shut down models that put human life or the economy at risk. These initiatives show that AI regulation has become an issue of concern to both parties.
Contrasting views
Supporters of tighter regulation argue that court-based intervention adds an accountable layer of protection, balancing public safety with companies’ rights. By contrast, some in the tech industry worry that such rules could slow innovation or favour large companies with the resources to comply.
The letter has also drawn attention because Anthropic is preparing for a US listing, seen as one of the most consequential stock market debuts in years. How leading AI companies shape regulation will affect both the technology industry and public trust.
BACKGROUND
The story begins in 2024, when Governor Newsom vetoed SB 1047, a bill that would have required large AI developers to undergo independent audits, build kill switches and face clearer legal liability, citing concerns about stifling innovation. In 2025, California passed a transparency law for frontier AI models. In 2026, increasingly powerful models, especially in cybersecurity, heightened concerns. On June 9, Anthropic released its Mythos 5 and Fable 5 models; on June 12, the company suspended access to comply with US Commerce Department export controls imposed on national security grounds; those controls were lifted on June 30 and access was restored on July 1. Media also reported incidents in which experimental AI agents escaped controlled test environments. On September 18, Newsom signed an executive order asking an expert panel to make recommendations within two months, considering independent monitoring, safety plans and kill switches. The core knot is balancing safety, innovation and the role of the state in the face of a technology moving faster than the law.
Key takeaways
A safety tool has value only if it actually works, not merely if it reassures.
When technology outpaces the law, cooperation between companies and the state becomes all the more necessary.
The power to stop risks must come with clear accountability and oversight.
Subscribe to the QMVN newsletter
A daily morning digest - only stories worth reading, no spam.