California Governor Newsom vetoes bill SB 1047 aimed at preventing AI disasters

California Governor Gavin Newsom vetoed bill SB 1047, which aimed to prevent bad actors from using artificial intelligence to inflict “critical harm” on people. The California state legislature passed the law on August 28 by a 41-9 margin, but many organizations, including the Chamber of Commerce, endorsed it. Urged Newsom to veto bill. in it veto message On September 29, Newsom said the bill was “well-intentioned” but “does not take into account whether an AI system is deployed in high-risk environments, involves critical decision-making, or uses of sensitive data.” “As long as a large system uses it, strict standards are applied to even the most basic functions.”

SB 1047 would mandate developers of AI models to adopt security protocols that would stop destructive use of their technology. This includes preventive measures such as testing and external risk assessment, as well as an “emergency shutdown” that will shut down the AI model completely. The first breach will cost at least $10 million, and subsequent breaches will cost $30 million. However, the bill has been revised to eliminate the state attorney general’s ability to sue AI companies for negligent practices unless a catastrophic event occurs. Companies will only be subject to injunctive relief and may be sued if their models cause critical harm.

This law will apply to AI models that cost at least $100 million to use and 10^26 FLOPS to train. It would also cover derivative projects where a third party invests $10 million or more to improve or modify the original model. Any company doing business in California will be subject to the rules if it meets other requirements. Referring to the bill’s focus on large-scale systems, Newsom said, “I do not believe this is the best approach to protecting the public from the real threats posed by technology.” The veto message adds:

By focusing only on the most expensive and large-scale models, SB 1047 creates a regulatory framework that could create a false public perception of security in controlling this rapidly advancing technology. Smaller, specialized models may emerge as equally or even more dangerous than the models targeted by SB 1047; This may come at the expense of potentially curtailing the innovation that fuels progress for the public good.

The previous version of SB 1047 would have created a new department called the Frontier Model Division to oversee and enforce the rules. Instead, the bill was amended ahead of a committee vote to place it in the hands of the administration’s Border Models Board within the Department of Government Operations. Nine members will be appointed by the state’s governor and legislature.

The bill faced a complicated path to a final vote. SB 1047 was authored by California State Senator Scott Wiener. TechCrunch: “We have a history of technology waiting for harm to happen and then wringing our hands. Let’s not wait for something bad to happen. Let’s get ahead of it.” Notable artificial intelligence researchers Geoffrey Hinton and Yoshua Bengio, as well as the Center for Artificial Intelligence Security, also supported the legislation. raise the alarm About the risks of artificial intelligence last year.

“Let me be clear – I agree with the author – we cannot afford to wait for a major disaster to occur before taking action to protect the public,” Newsom said in his veto message. he said. The statement continues as follows:

California will not abdicate its responsibility. Security protocols should be adopted. Proactive guardrails must be implemented and serious consequences for bad actors must be clear and enforceable. However, I disagree with the idea that to ensure the safety of the public, we should settle for a solution that is not determined by empirical trajectory analysis of AI systems and capabilities. Ultimately, any framework for effectively regulating AI needs to keep pace with the technology itself.

SB 1047 faced fierce opposition from across the tech world. Researcher Fei-Fei Li criticized invoice, as he did Meta Chief AI Scientist Yann LeCun for limiting the potential for discovering new uses of AI. Trade group representing tech giants like Amazon, Apple and Google in question SB 1047 would limit new developments in the state’s technology sector. Venture capital firm Andreeson Horowitz and several startups also questioned whether the bill would impose unnecessary financial burdens on AI innovators. Anthropic and other opponents of the original bill, adopted In the version of SB 1047 that passed the California Appropriations Committee on August 15.