Anthropic Researchers Warn of Existential AI Risk, Sparking Global Debate

Two leading researchers from AI safety firm Anthropic have publicly voiced grave concerns about the potential for artificial intelligence to pose an existential threat to humanity. Jacob Coxon resigned from the company to state that both OpenAI and Anthropic are "gambling with our lives" by pursuing self-improving superintelligence without adequate safety measures. He expressed that the race towards advanced AI is happening without responsible development practices.

Evan Hubinger, who heads Anthropic's alignment science, corroborated Coxon's warning, stating that there is a genuine belief within the company that AI could lead to human extinction. Hubinger estimated this risk to be over 10% within the next decade, adding that Anthropic currently lacks a clear plan for aligning superintelligence and is not demonstrably on track to develop one. Samuel Marks, who leads scalable oversight at Anthropic, echoed these sentiments, indicating a consensus among the safety-focused team regarding the severity of the risks.