California legislature passes AI safety bill requiring safety testing of AI models that cost more than $100 million to develop

The California State Assembly passed the Safe and Secure Innovation for Cutting-Edge AI Models Act (SB1047), a state bill that requires safety testing of AI models. The bill will then be voted on again and signed by Governor Gavin Newsom to become law.

Contentious California AI bill passes legislature, awaits governor's signature | Reuters
https://www.reuters.com/technology/artificial-intelligence/contentious-california-ai-bill-passes-legislature-awaits-governors-signature-2024-08-28/

California State Assembly passes sweeping AI safety bill - The Verge
https://www.theverge.com/2024/8/28/24229068/california-sb-1047-ai-safety-bill-passed-state-assembly-governor-newsom-signature

SB1047 is a law that requires safety testing for cutting-edge AI models that cost more than $100 million to develop or require a certain amount of computing power. It also requires developers of AI models operating in California to incorporate a kill switch-like mechanism that stops the AI model when a problem occurs.

It would also give state attorneys general the power to sue developers if they don't comply when government systems, like the power infrastructure, are threatened by AI. SB 1047 would also require developers to hire third-party auditors to evaluate safety measures and provide protections for whistleblowers who report misuse of AI.

However, there are voices of opposition to SB1047, which has been passed in California, where many high-tech companies are based, as it would have a significant impact on the business operations of AI companies.

California's AI safety bill requiring 'kill switches' for AI models is criticized for forcing AI startups to withdraw and damaging open source models - GIGAZINE

'SB 1047 is poorly founded and could do more harm than good,' said Democratic Representative Nancy Pelosi, arguing that it could drive developers out of California and threaten the establishment of open source AI models.

In a letter to Senator Wiener, Google's parent company Alphabet, OpenAI, and Meta expressed concerns about SB 1047. Claude's developer Anthropic said it collaborated on drafting the bill, but that 'while the benefits of the bill are likely to outweigh the costs, there are still concerns and ambiguities we are concerned about.'

'This is the strongest bipartisan opposition I've ever seen,' said Martin Casado, general partner at venture capital firm Andreessen Horowitz, who said he hopes Newsom will veto the bill.

Meanwhile, Elon Musk, CEO of AI company xAI, supports the bill, saying, 'I believe California should pass SB 1047. For more than 20 years, I have advocated for regulating AI, just as we do any product or technology that poses a potential risk to the public.'

This is a tough call and will make some people upset, but, all things considered, I think California should probably pass the SB 1047 AI safety bill.

For over 20 years, I have been an advocate for AI regulation, just as we regulate any product/technology that is a potential risk…

— Elon Musk (@elonmusk) August 26, 2024