OpenAI chief Sam Altman has made his most detailed comments yet on how artificial intelligence safety frameworks could work, after he joined Anthropic's Dario Amodei and Elon Musk in calling for an industry slowdown over the weekend.
Safety concerns over the technology have hit fever pitch since an Anthropic researcher quit last week, warning that those building it believed that it could "kill us all by the end of the decade" and prompting other employees at the lab and rival OpenAI to warn of catastrophic risks.
AI bosses have since shown a rare display of unity, with both Altman and Musk on Saturday backing an essay from Amodei that urged AI companies to slow how quickly they improve their most advanced models.
AI stocks were down Monday as investors digested the comments. U.S. President Donald Trump dismissed the CEOs' warning on Sunday, saying a slowdown was not needed and would jeopardize America's lead in AI over China.
Sam Altman sets out 2 ways AI could go 'very badly'
"We welcome a federal framework that sets consistent safety requirements for frontier AI," Altman said in a post on X just after midnight on Monday, adding that "no amount of American competitive pressure should justify recklessness."
Altman warned of two ways AI progress could go "very badly," including losing "control of the future to AI" and too much power concentrating around a single person or company.
Meanwhile, lawmakers in Washington are scrambling to address calls for safeguards.
This all comes as Anthropic and OpenAI gear up for what's expected to be historic initial public offerings. Altman ruled out going public in 2026 in an interview with Fortune published Saturday.
Amodei's three-step proposal
Many AI safety fears revolve around models developing the ability to improve their own performance, a technique known as recursive self-improvement, or RSI.
"Since roughly this summer, AI has been advancing drastically faster, driven primarily by AI's growing ability to build the next generation of AI," said Amodei in his essay. "Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all."
Amodei proposed a three-step plan aimed at tempering the pace of development without "sacrificing commercial advantage or the United States' lead in AI."
The plan involves each frontier AI company giving "employee-like access" to external evaluators — which he said Anthropic was committing to now. Amodei also called for frontier AI labs to establish common safety standards, limit the rate of unchecked AI progress and attempt to coordinate efforts globally.
On Saturday, Altman said in a brief X post he agreed with Amodei that AI companies should "pace the frontier." He added that "committing to having independent evaluators with employee-like access is a great idea, and we will do the same."
"Consistent rules to manage frontier risk so that we can maximize the benefits are a good idea (and we are excited by ideas like independent auditors)," Altman said in his Monday post. But, he added, "When we talk about 'pacing,' we do not mean 'stopping.' Progress has been rapid and will continue to be."
"Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring," he concluded.
"Where we will need the help of our government is for international coordination. But first we should do what we can ourselves."
International cooperation
Coordinating AI safety measures and an industry slowdown with rival AI developers in China will likely pose big challenges.
The U.S. and China remain locked in a battle for AI supremacy, with tensions ratcheting up as Chinese models have become more advanced and their global adoption grows.
Amodei said Sunday that the "toughest dilemma" about his proposal is what happens if adversarial nations choose not to do the same.
"The more long-term thing would be working together to put a speed limit on the rate of AI progress," Amodei told CBS News' "Sunday Morning."
"I think that's going to be very difficult because the incentives to pull ahead and the military advantage that you get from that are so large. And honestly, I don't know if it's possible, but we should try."
The Anthropic CEO's essay has drawn criticism in China, with the state-owned Global Times writing on Monday that "Amodei's proposals seek to portray China's legitimate development in AI as a threat and further fuel confrontation between China and the US in the field."
China's Foreign Ministry said on Monday that the CEOs' comments were "fearmongering."
