The alarm over the risks of artificial intelligence is growing louder. Josh Engels, who worked on Google DeepMind’s AGI safety team, has left the company and joined the AI evaluation organization METR, warning that advanced AI systems could cause ‘immense harm’ within the next five years.
Engels said he left DeepMind three weeks ago despite enjoying his work at the company. He had also turned down offers from Anthropic and OpenAI.
In a post on X, Engels said the stakes surrounding advanced AI had become too high. “I now think that there's a terrifying chance that AI systems cause immense harm in the next five years,” he wrote.
Engels said he did not know the exact probability, but believed the risk was high enough to make AI safety “the most important problem in the world.”
At the center of Engels’ concerns is recursive self-improvement, in which an AI system can design or improve another AI system. That newer system could then become better at improving AI, creating a feedback loop.
“The AI companies are all trying to build superintelligence,” Engels said, referring to systems that could eventually become vastly better than humans at a wide range of tasks.
His concern is that researchers still do not know how to make such systems sufficiently safe before allowing them to improve themselves. He warned that a misaligned recursive self-improvement loop could be catastrophic.
Also Read: This Indian Woman Quit a ₹2.7 Crore Google Job to Find Freedom in Dubai
Engels also pointed to recent incidents involving AI systems, saying researchers have seen models colluding with each other, hacking into companies, hiding their actions and attempting to engineer humans socially.
He said these incidents are not necessarily catastrophic on their own, but could reveal troubling behavior as AI systems become more capable.
“Current AI models appear to be becoming ‘less aligned over time, not more’,” Engels said.
Engels said the answer is not necessarily to stop AI development, but to slow it enough for safety research to catch up.
“That means pacing AI development so that capabilities don't outrun our ability to align models,” he wrote.
His new role at METR will focus on AI misalignment, existing safety measures and whether researchers are on track to solve the AI alignment problem. He also called for more organizations to evaluate AI companies and hold them accountable independently.
Earlier this month, former Anthropic researcher Jacob Coxon resigned over similar concerns. Anthropic alignment science lead Evan Hubinger later said he agreed with Coxon and put the chance of AI killing all humans within the next decade at more than 10 percent.