AI can kill all humans within next decade: Anthropic researcher

Hyderabad: Evan Hubinger, Alignment Science lead at American artificial intelligence corporation Anthropic, headquartered in San Francisco, has warned that Artificial Intelligence (AI) can kill all humans within the next decade.

“I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to,” he added.

Another AI researcher at Anthropic, Jacob Coxon, who announced his resignation on Wednesday, September 9, alleged that leading technology firms, including OpenAI, are acting irresponsibly in an unchecked race toward developing superintelligence and “gambling with our lives”.

Coxon, who spent the past three years conducting pre-training research across both OpenAI and Anthropic, said that the competitive rush between labs is being pursued without adequate safeguards.

He warned that the technology is advancing rapidly toward superhuman systems that can hack anything, revolutionise any field overnight, and acquire real power and resources. He added that no other human activity poses a comparable level of danger.

In a post on social media platform X, he said, “I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives”.

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.

He further said, “At OpenAI, many have not deeply internalised the civilisational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first – they believe no one else will act responsibly, so they must do it themselves, despite the risk”.

A common response is “if they truly believe this, why are they still building it?” At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first – they believe no one else…

“Accepting this race and entering the ‘endgame’ is a hubristic gamble that should not be launched from a private company’s Slack,” he said, adding that many working in the sector earnestly believe AI could cause catastrophic harm by the end of the decade.

Accepting this race and entering the “endgame” is a hubristic gamble that should not be launched from a private company’s Slack. Attempting to speedrun alignment should require extraordinary confidence that there are no better trajectories available.

Coxon claimed that while executives and researchers often adopt measured language in public, many privately harbour deep concerns about the existential dangers posed by uncontrolled artificial intelligence. He also noted that security incidents, such as the recent Hugging Face breach, demonstrate the urgent need for pacing agreements between major US labs.

Urging his peers to reconsider their complicity, Coxon wrote: “If you are a lab researcher, I urge you to consider what the next few years will actually feel like. Do you want to kick off a superintelligent RL (reinforcement learning) run without a rigorous understanding of its mind? Should you put your head down because ‘it’s happening anyway’ – or take this moment to call for different conditions?”

If you are a lab researcher, I urge you to consider what the next few years will actually feel like. Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind? Should you put your head down because “it’s happening anyway” – or take this moment…

Responding directly to Coxon’s statement, Hubinger validated the existential concerns.

“Jacob is correct here– we really do earnestly believe AI could kill all humans! I personally think it is (more than) 10 per cent within the next decade,” Hubinger wrote on X.

Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to. https://t.co/QAIHiFP3QZ

Hubinger noted that while the risk posed by present models remains low, the primary threat stems from recursive self-improvement accelerating faster than previously anticipated.

Stay updated with our WhatsApp & Telegram by subscribing to our channels. For all the latest Technology updates, download our app Android and iOS.