Key takeaways
- 01.Frontier-model coverageWould have applied to AI models meeting compute or cost thresholds (10^26 FLOPs or $100M+ training cost), targeting only the largest model developers.
- 02.Safety protocol mandateRequired developers to implement reasonable safety protocols, including a "kill switch" capability for models that pose catastrophic risk.
- 03.Vetoed September 2024Governor Newsom vetoed citing concerns about regulatory rigidity for emerging technology and the risk of stifling open-source development.
Background
I · ContextThe evolution of the AI Act reflects a shift from a product-safety approach to a fundamental rights-based framework. While initial drafts focused on high-risk industrial applications, the rapid commercial deployment of general-purpose AI necessitated a specific regulatory tier targeting generative outputs.
Article 50 is the Act's transparency backbone for content. It sits alongside the broader high-risk system regime but applies horizontally: any provider or deployer generating synthetic content in the EU market is potentially covered, regardless of whether the underlying system is classified as high-risk.
Detailed provisions
II · The obligationsSafety determination
Developers of covered models would have been required to make a "positive safety determination" before public deployment, demonstrating reasonable mitigation of catastrophic harms.
Shutdown capability
Required ability to promptly enact a "full shutdown" of all covered models in the developer's control, intended to address runaway-behavior scenarios.
Cybersecurity protections
Mandated reasonable protections against unauthorized access, model weight theft, and unauthorized fine-tuning that could circumvent safety measures.
Civil enforcement
California Attorney General granted authority to bring civil actions; statutory damages scaled with model compute and training cost.
Specific exemptions apply for artistic, creative, satirical, or fictional works, provided they do not infringe the rights of others. The Act delegates the exact technical specifications for watermarking and labeling to a forthcoming implementing act, expected in late 2026.