Anthropic IPO Filing Warns of AI Doomsday Risks Including Weapons, Rogue Systems

Anthropic disclosed in its Monday SEC filing that its AI models could threaten humanity itself, citing risks of bad actors building deadly weapons or the technology going rogue. The San Francisco company's initial public offering document devotes 93 of 335 pages to "risk factors," compared with 38 of more than 250 pages for rival SpaceX. The filing warns that rapid model development could outpace the company's safety measures and that systems "become more autonomous and capable of taking consequential actions with less human oversight" could "take harmful actions independently and at scale without human intervention." Anthropic also flagged risks from its models contributing to its own research, potentially progressing "beyond our ability to evaluate and safeguard new models." Other risks include misuse for cyberattacks, surveillance and biological weapons research; constraints on compute and data center opposition; and reliance on a small number of customers, including SpaceX and Microsoft, for much of its revenue. Anthropic positions its IPO as a step toward mitigating AI dangers, with founders writing that "anything so powerful can be used for ill as well as good."