In a decisive vote of 41 to 16, the California State Assembly has passed SB 1047, a bill mandating safety testing for companies developing large generative AI models. Authored by Senator Scott Wiener, the legislation aims to safeguard against the misuse of AI technology in conducting cyberattacks on critical infrastructure or developing dangerous weapons. However, the bill continues to face criticism and opposition, notably from Speaker Emerita Nancy Pelosi, as it proceeds to Governor Gavin Newsom for final approval.

Senator Wiener expressed his pride in the collaborative efforts behind SB 1047, emphasizing that innovation and safety are not mutually exclusive in the technology sector. The bill's provisions seek to prevent AI from being harnessed for nefarious purposes, particularly in relation to cyber warfare and the development of chemical, nuclear, or biological weapons.

While the bill has gained significant support, including from high-profile figures like Elon Musk, it has also encountered resistance from influential opponents. Speaker Pelosi, along with three Silicon Valley Congress members, voiced concerns that the legislation could impede the growth of AI technology in California. Likewise, San Francisco Mayor London Breed called for further dialogue and collaboration between industry, government, and community stakeholders before proceeding with the bill.

Amid the ongoing debate, critics argue that the bill's potential legal repercussions from the state attorney general could discourage big tech companies from sharing open-source software with smaller entities, thus stifling innovation. Concerns have also been raised by Stanford professor and former Google executive, Andrew Ng, who contends that the legislation wrongly attempts to regulate AI technology rather than focusing on addressing harmful applications.

In response to the feedback and to increase the bill's chances of approval, Senator Wiener amended SB 1047 following consultation with leaders in the AI space, including safety groups, academics, and developers. These changes include removing the provision allowing the California Department of Technology to establish a division for continuous oversight and enforcement. The amendments have been made with the aim of garnering support from lawmakers and Governor Newsom, particularly considering the state's current budget deficit.

The bill specifically targets companies developing AI systems that cost over $100 million to train, leaving smaller AI projects unaffected. However, opponents argue that the mere threat of legal action could have a chilling effect on AI innovation and hinder the ability of researchers to study and analyze cutting-edge AI technologies.

While other jurisdictions, such as the European Union and the state of Colorado, have opted for comprehensive AI legislation, Senator Wiener explains that California's approach involves addressing specific issues through individual bills. He acknowledges the pros and cons of this methodical approach and highlights the need for harmonization among the various AI-related bills.

Now, with the bill's passage in the State Assembly, SB 1047 now heads to Governor Newsom's desk. He has until September 30 to either sign or veto the legislation. As the fate of the bill hangs in the balance, the debates surrounding AI safety, regulation, and innovation continue to shape the future of technology in California and beyond.