California State Assembly Passes Wide-Ranging AI Safety Act

The California State Assembly passed the Safe and sound and Confident Innovation for AI Models in Border States Act (SB 1047), Reuters Agency reportsThe bill is one of the first significant regulations on artificial intelligence in the United States.

The bill, which has been a flashpoint in Silicon Valley and beyond, would require AI companies operating in California to implement a series of precautions before training an advanced foundation model. They include allowing the model to be quickly and completely turned off, ensuring that the model is protected from “unsafe modifications after training” and maintaining a testing procedure to assess whether the model or its derivatives are particularly likely to “cause or enable critical harm.”

Senator Scott Wiener, the bill’s lead author, he said SB 1047 is a very sensible bill that asks enormous AI labs to do what they have already committed to: test their enormous models for catastrophic security risks. “We have been working hard all year with open source advocates, Anthropic, and others to refine and improve the bill. SB 1047 is well-calibrated to what we know about foreseeable AI risks and deserves to pass.”

Critics of SB 1047, including OpenAI and Anthropicpoliticians Zoe Lofgren AND Nancy PelosiAND California Chamber of Commerce — argue that it is too focused on catastrophic harm and may unfairly harm tiny, open AI developers. Bill In response, the bill was amended to replace potential criminal penalties with civil penalties, narrow the enforcement powers granted to the California Attorney General, and adjust the requirements for joining the Board of Frontier Models created by the act.

After the state Senate votes on the amended bill — a vote that is likely to pass — the AI ​​safety bill will go to Gov. Gavin Newsom, who will have until the end of September to decide its fate. According to New York Times.

Anthropic declined to comment beyond pointing to a letter sent by Anthropic CEO Dario Amodei to Gov. Newsome. last week. OpenAI did not immediately respond to a request for comment.