California Governor Gavin Newsom has vetoed SB 1047, a bill aimed at preventing malicious actors from using AI to cause “serious harm” to humans. The California Legislature passed the bill by a 41-9 vote on August 28, but several organizations, including the Chamber of Commerce, Urged Newsom to veto the bill.. in him Veto message On September 29, Newsom said the bill was “well-intentioned,” but “doesn’t make sense whether AI systems are deployed in high-risk environments, involve critical decision-making, or involve the use of sensitive data. “Instead, this bill doesn’t take into account what’s going on.” We apply strict standards to even the most basic functionality as long as large systems deploy it. ”
SB 1047 would have made developers of AI models responsible for adopting safety protocols to prevent destructive uses of the technology. This includes precautions such as testing and external risk assessments, as well as “emergency shutdowns” that completely stop the AI model. The first violation will cost a minimum of $10 million, and subsequent violations will cost $30 million. However, the bill was amended to remove the power of state attorneys general to sue AI companies for negligent conduct, unless a catastrophic event occurs. Companies are only eligible for injunctive relief and can be sued if their models cause significant harm.
This law applies to AI models that cost at least $100 million to use and 10^26 FLOPS to train. It would also have covered derivative projects where a third party invested more than $10 million in developing or modifying the original model. All companies doing business in California are subject to this rule if they meet other requirements. “I don’t think this is the best approach to protecting the public from the real threats posed by technology,” Newsom said, noting the bill’s focus on large systems. The veto message adds:
By focusing only on the most expensive and large-scale models, SB 1047 establishes a regulatory framework that may give the public a false sense of security about the control of this rapidly changing technology. Smaller, more specialized models may emerge as equally or even more dangerous than those covered by SB 1047, at the cost of innovations that prioritize public interest and promote progress. itself may be suppressed.
An earlier version of SB 1047 would have created a new division called the Frontier Model Division to oversee and enforce the rules. Instead, the bill was changed prior to a committee vote to place governance in the hands of the Frontier Model Commission within the Office of Government Management. The nine commissioners are appointed by the governor and the state Legislature.
The bill had a complicated path to a final vote. SB 1047 was authored by California State Senator Scott Wiener, who said: tech crunch: “We have a history of wringing our hands and using technology waiting for harm to happen. Don’t wait for something bad to happen, get out before it happens.” Famous. AI researchers Jeffrey Hinton and Yoshua Bengio support the bill, and the Center for AI Safety has been sounding the alarm about the risks of AI over the past year.
“Let me be clear: I agree with the author. We cannot wait for a catastrophe to occur before taking action to protect our people,” Newsom said in his veto message. I mentioned it inside. The statement continues:
The State of California does not waive its liability. Safety protocols must be adopted. Aggressive guardrails must be implemented, and serious consequences for malicious actors must be clear and enforceable. However, I disagree that to protect public safety we must settle for solutions that are not based on empirical trajectory analysis of AI systems and capabilities. Ultimately, a framework for effectively regulating AI needs to keep pace with the technology itself.
SB 1047 attracted strong opposition from across the technology industry. Researcher Feifei Li criticized invoice, like I did Yann LeCun, Chief AI Scientist at Meta, for limiting the potential to explore new uses for AI. Industry associations representing technology giants like Amazon, Apple, and Google said SB 1047 would limit new development in the state’s technology sector. Venture capital firm Andreesson Horowitz and several startups also questioned whether the bill imposes an unnecessary financial burden on AI innovators. Antropic and other opponents of the original bill called for the following amendments: adopted In a version of SB 1047 that passed the California Appropriations Committee on August 15th.
