AI companies have been talking about how artificial intelligence could change jobs for years. Now, AI startup Anthropic is raising a different concern. The company said the bigger risk may not be job cuts alone. It could be a future where AI becomes so powerful that people struggle to keep it under control.
The warning comes from a new research paper released by Anthropic. The document is more than 10,000 words long and looks at how advanced AI could develop in the coming years.
Anthropic CEO Dario Amodei has often spoken about the effect AI may have on employment. He has warned that many office jobs could be affected as AI becomes better at handling tasks that once required human workers.
However, the company's latest paper shifts the focus. It said people should also think about what happens if AI systems become capable of improving future AI models on their own. According to Anthropic, AI is already helping with software development. The company says its Claude AI model now contributes a large share of code used internally. As AI tools become more capable, they may take on bigger responsibilities.
That possibility has raised concerns about how much human oversight will remain in the future.
AI tools today do far more than answer questions. They can write code, review information, summarize reports, and assist with research. Many businesses already use these tools to save time and improve productivity.
Anthropic believes this progress will continue. Future AI systems could handle longer and more difficult tasks with less human input.
While that may help businesses work faster, it also creates new questions. How do people monitor these systems? How do they make sure mistakes are caught quickly? And who is responsible if something goes wrong?
These are some of the issues the company wants governments, researchers, and technology firms to discuss now rather than later.
Also Read: Anthropic Scales Global Cybersecurity Push With Project Glasswing Expansion
The authors of the Anthropic post, company co-founder Jack Clark and Marina Favaro, head of its research institute, said the pause would be used to enable ‘societal structures and alignment research’ to keep up with AI advances. Alignment is industry shorthand for making sure the technology matches human values and intentions.
The proposed coordination would let advanced AI labs verify that global rivals have actually stopped or slowed their work, ‘and that a bad actor could not use the auspices of a coordinated slowdown to jump ahead in secret.’
One of the most notable points in the paper is the idea that AI development may need stronger safety checks as technology advances. Anthropic said companies should think seriously about ways to monitor powerful AI systems and support humans in remaining in control. The company also suggested that major AI developers may eventually need to work together on common safety standards.
As AI becomes more capable, the challenge will be making sure people remain in charge of the technology they create, according to Anthropic.