The Anthropic Researcher Who Quit Over AI Fears Isn’t the Only One Sounding Alarm Bells: ‘Things Could Be Out of Control Next Year’
Jacob Coxon left OpenAI for Anthropic because of its reputation for AI safety. Now he’s walking away from the industry entirely.
When the people building AI start walking away in fear, it’s worth taking notice.
Jacob Coxon, a researcher at Anthropic who previously worked at OpenAI, is quitting the AI industry over concerns that companies are racing to build systems that could eventually become impossible to control. “We’re on track for a lot of the most aggressive of these scenarios,” Coxon told , warning that by the end of next year, “things could be out of control already.”
Coxon left OpenAI earlier this year for Anthropic because of the company’s reputation for AI safety. But he now believes even Anthropic can’t responsibly pursue AI that can outperform humans across a range of tasks without government intervention or an industrywide slowdown.
Evan Hubinger, Anthropic’s Alignment Science Lead, posted a tweet supporting Coxon that has grabbed national headlines. “We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade,” he wrote.
Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.
— Evan Hubinger (@EvanHub)
They are not the first AI insiders to walk away with serious concerns. Researchers and executives at OpenAI, Anthropic, and xAI have left over similar worries about safety, ethics, and the industry’s rapid pace. Anthropic scientist Evan Hubinger backed Coxon’s warning, writing that he believes there’s a greater than 10% chance AI could kill humanity within the next decade.
When the people building AI start walking away in fear, it’s worth taking notice.
Jacob Coxon, a researcher at Anthropic who previously worked at OpenAI, is quitting the AI industry over concerns that companies are racing to build systems that could eventually become impossible to control. “We’re on track for a lot of the most aggressive of these scenarios,” Coxon told , warning that by the end of next year, “things could be out of control already.”
Coxon left OpenAI earlier this year for Anthropic because of the company’s reputation for AI safety. But he now believes even Anthropic can’t responsibly pursue AI that can outperform humans across a range of tasks without government intervention or an industrywide slowdown.
