While much has been made about the extent to which AI represents an existential threat to humankind, the specific mechanisms by which it could bring about that destruction are largely opaque. If nothing else, the 94 pages that Anthropic's prospectus devotes to risk factors make clear both where it sees the danger coming from, and the ways in which its researchers are running toward it in order to win the AI arms race
"Sufficiently advanced AI systems could, if not developed, deployed, and operated with appropriate safeguards, oversight and security measures, pose catastrophic or existential risks to humanity," the prospectus reads, citing concerns both from critics and regulators and from inside Anthropic itself. (It's the first of three times the phrase "existential risks" pops up.)
Anthropic is careful to tie that outcome, though, to a specific process: recursive self-improvement
