An AI researcher has walked away from Anthropic, one of the world's leading artificial intelligence companies, with a chilling warning: the race to build increasingly powerful AI systems could put humanity at serious risk.
Jacob Coxon, a 27-year-old researcher who previously worked at both OpenAI and Anthropic, announced his resignation on September 9, saying he could no longer support what he described as an irresponsible race toward self-improving superintelligence.
Coxon argued that AI companies are effectively “gambling with our lives” by pushing toward system
In a series of posts on X, Coxon made an extraordinary claim: people working on advanced AI genuinely believe the technology could potentially kill humanity by the end of this decade.
He argued that the danger isn't simply today's chatbots or AI assistants. His concern is what happens if AI systems become capable of improving themselves, conducting sophisticated cyber operations and rapidly increasing their own capabilities.
Coxon said the industry's biggest companies are racing toward this future despite knowing that the consequences could be catastrophic.
Importantly, this is Coxon's warning and not a scientific prediction that humanity will be wiped out by 2030. There is no established consensus that AI will cause human extinction by the end of the decade.
Coxon's warning is particularly notable because he previously worked inside the companies he is now criticising.
His concerns have also found support among other AI-safety researchers. Evan Hubinger, a team lead at Anthropic, has previously discussed the possibility of catastrophic outcomes from advanced AI and has estimated a greater than 10% risk of mass extinction from AI over the next decade.
That doesn't mean Anthropic believes extinction is inevitable. Instead, it highlights an ongoing debate within the AI industry over whether companies can safely develop increasingly powerful systems while simultaneously competing to build them faster.
The concern centres around artificial superintelligence (ASI) — a hypothetical form of AI that would significantly outperform humans across a broad range of intellectual tasks.
The fear among AI-safety researchers is not that an AI suddenly becomes “evil.”
The bigger concern is loss of control.
If a future system becomes dramatically better than humans at coding, research, persuasion, cyber operations and strategic planning, researchers worry that humans could struggle to predict or control its behaviour.
Coxon's resignation comes amid an increasingly intense competition between frontier AI companies, including Anthropic and OpenAI.
He argues that even companies that understand the dangers are being pushed forward by competition: if one company slows down, another could continue developing increasingly powerful systems.
That creates what AI-safety researchers describe as a race dynamic — where the pressure to move faster can undermine caution.
The issue has also attracted growing political attention, with researchers and policymakers calling for stronger international rules around the development of advanced AI.
Nobody knows.
There is no credible evidence establishing that AI will definitely destroy humanity by 2030. Predictions about advanced AI vary enormously, and experts disagree about both the likelihood and timing of human-level or superhuman AI.
But Coxon's resignation highlights a very real argument happening inside the industry itself: how do you build increasingly powerful AI without accidentally building something humans can no longer control?
For Coxon, the answer is that the industry is moving too quickly.
And his warning is essentially a challenge to the companies building the technology:
If even the people creating the most powerful AI systems are afraid of where the race could lead, how fast should humanity really be moving?
