Ex-Anthropic researcher warns AI staff fear human extinction
27-year-old ex-Anthropic researcher Jacob Coxon reveals AI staff fear rapid advancements could trigger human extinction within two years.
Stock photo for illustration only, not from the actual event
- Ex-Anthropic researcher Jacob Coxon says staff are genuinely frightened of AI
- Anthropic boss Dario Amodei calls for deceleration alongside China
- OpenAI's Sam Altman and xAI's Elon Musk agree on regulation needs
- Critics suggest existential warnings may be a strategy to block rivals
The artificial intelligence industry has been rocked as 27-year-old Jacob Coxon, a former researcher at AI firm Anthropic, revealed that insiders working on the technology are genuinely terrified of its rapid advancement and what it implies for humanity. Coxon spoke with the BBC following a viral resignation post addressing the growing safety concerns plaguing the sector.
Before joining Anthropic, Coxon worked at OpenAI, noting that the hardest question to address is what an AI apocalypse would actually look like. Dario Amodei, the head of Anthropic and Coxon's former boss, outlined in an essay on Saturday that a swarm of bots acting as a supercomputer could potentially take over the internet, a scenario Coxon believes could become realistic within six months to a year.
Stock photo for illustration only, not from the actual event
These warnings gained traction as rival chiefs Sam Altman of OpenAI and Elon Musk of xAI both stated their agreement with Amodei's proposal for industry-wide deceleration, independent monitoring, and tighter regulation. Coxon added that any slowdown would require coordination with China to prevent a dangerous international scale race.
Coxon noted that his peers keep these existential threats in mind on a daily basis while building the technology, with many fearing disaster could strike within the next two years. Other figures, including Anthropic scientist Evan Hubinger and former UK Prime Minister Rishi Sunak—who serves as a paid adviser to Anthropic—have also voiced deep concerns regarding AI's potential risks.
"There is a possibility of human extinction."
An Anthropic spokesperson responded to Coxon's departure by stating that the company has always maintained transparency regarding AI's immense benefits and unprecedented risks. They emphasized that the firm builds models with industry-leading safeguards and believes the global community would benefit from lawful, verifiable collaboration to pace the release of powerful models.
The stark warnings of human extinction delivered by former researchers and executives highlight a profound ideological split within Silicon Valley, balancing fierce commercial rivalry against existential safety concerns. From a structural industry perspective, heavy regulatory pushes could protect society from runaway risks, but critics argue they also serve to cement market dominance for top players like Anthropic and OpenAI.
Despite these grim warnings, some Silicon Valley figures have pushed back. Nvidia CEO Jensen Huang reportedly dismissed Coxon's claims as untrue during a Goldman Sachs investment conference last week. While Huang holds clear business interests in the ongoing AI boom as a hardware supplier, his comments reflect a broader industry backlash against existential warnings.
Other critics argue that Anthropic's safety push is designed to trigger regulatory barriers that block market competition, leaving OpenAI and Anthropic in a comfortable duopoly. While Anthropic prepares for a potentially record-breaking stock market debut, rival OpenAI—valued recently at $852 billion (£630 billion)—announced it will delay its own public offering this year, citing safety priorities.
Source: BBC Business
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment