The people building AI models are now worried it’s moving to fast

Anthropic CEO Dario Amodei, whose company developed Claude, has called for the development of increasingly powerful AI systems to be deliberately paced, warning that the risks are becoming serious enough that governments, companies and independent experts need more time to understand and manage them.

He isn't calling for AI development to stop.

Instead, Amodei wants companies to avoid racing ahead so quickly that their ability to build powerful AI overtakes their ability to keep it safe.

His concerns include increasingly capable AI being used for cyberattacks and biological threats, as well as the longer-term possibility of humans losing control of systems that become significantly more capable than we are.

Perhaps most concerning is AI's growing ability to help develop the next generation of AI itself.

Amodei argues that as AI becomes better at assisting researchers and engineers who build AI, progress could accelerate even further — making it increasingly difficult for safety measures and regulation to keep up.

He wants independent experts to be allowed to evaluate powerful AI models as they are developed, common safety standards across the industry and eventually stronger international regulation.

And Amodei isn't alone.

OpenAI CEO Sam Altman has also expressed support for pacing development at the AI frontier and for independent safety evaluations, while Elon Musk has backed Amodei's warning.

It creates an extraordinary situation.

The companies are still competing to build the world's most capable AI.

But some of the people leading that competition are increasingly warning that getting there too quickly could be dangerous.

The question is no longer simply how powerful artificial intelligence can become.

It is whether humans can learn how to control it as quickly as we are learning how to build it.