California governor vetoes controversial AI safety bill

Author: Tom Mitchelhill, CoinTelegraph; Compiler: Tao Zhu, Golden Finance

California Governor Gavin Newsom vetoed a controversial artificial intelligence (AI) bill, arguing that the bill would hinder innovation and fail to protect the public from "real" threats posed by technology.

On September 30, Newsom vetoed SB 1047, the Frontier Artificial Intelligence Model Safety Innovation Act, which was strongly opposed by Silicon Valley.

The billproposed mandatory safety testing of AI models and other guardrails, which technology companies fear would stifle innovation.

Newsom said in a statement on Sept. 29 that the bill focuses too much on regulating existing top AI companies and doesn’t protect the public from “real” threats posed by new technologies.

“Instead, the bill applies strict standards to even the most basic functionality — as long as it’s deployed by large systems. I don’t believe this is the best way to protect the public from real threats posed by technology.”

SB 1047, authored by Sen. Scott Wiener, D-San Francisco, also requires California developers — including big names like ChatGPT maker OpenAI, Meta and Google — to implement “kill switches” for their AI models and publish plans to mitigate extreme risks.

If the bill goes through, AI developers could also be sued by the state attorney general in cases of ongoing threats from models like AI grid takeovers.

Newsom said he has asked the world’s leading AI safety experts to help California “develop actionable guardrails,” with a focus on creating “science-based trajectory analysis.” He added that he had ordered state agencies to expand their assessments of the risk of catastrophic events that could arise from AI development.

Although Newsom vetoed SB 1047, he said adequate safety protocols must be in place for AI, adding that regulators cannot "wait until a major disaster occurs before taking action to protect the public."

Newsom added that his administration has signed more than 18 bills on AI regulation in the past 30 days.

Politicians and Big Tech Oppose AI Safety Bill

Before Newsom made his decision, the bill was unpopular among lawmakers, consultants and big tech companies.

House Speaker Nancy Pelosi and companies like OpenAI said it would severely hamper the development of AI.

Neil Chilson, head of AI policy at the Abundance Institute, warned that while the bill focuses on models of a certain cost and scale (those costing more than $100 million), its scope could easily be expanded to hit smaller developers.

But some are open to the bill. Billionaire Elon Musk, who is developing his own AI model called “Grok,” is one of a handful of tech leaders who have supported the bill and AI regulation more broadly.

In an August 26 post to X, Musk said “California should pass SB 1047, the AI ​​safety bill,” but acknowledged that supporting the bill was a “tough decision.”