As competition in the artificial intelligence sector intensifies, OpenAI and Anthropic are increasingly warning about the potential dangers of advanced AI systems even as they push ahead with developing more powerful models in a race for technological supremacy.
Critics say it amounts to "fear-mongering," or a calculated move to win over investors.
Both San Francisco companies – makers of ChatGPT and Claude – are focused on the same scenario: so-called recursive self-improvement, a stage where AI systems could essentially design and train the next generation of AI with little human involvement.
Anthropic called that scenario plausible but uncertain last week. OpenAI went further Monday, predicting it would arrive by March 2028.
Is that a good thing or a bad thing? OpenAI treats it as a goal, seeing it as an automated researcher that could speed up scientific discovery.
The ChatGPT creators promise a personal AI for every person on Earth -- something they say could prove more transformative than electricity or the car.
Like those technologies, they argue, AI can be made safe once society builds the right guardrails, just as it eventually developed the seatbelt, speed limits and driver's licenses.
Anthropic also points to potential breakthroughs in medicine, technology and the economy, but sounds more cautious, openly warning of the risk of "loss of control."
"You want the option to be able to take your foot off the gas and put your foot on the brake," Anthropic co-founder Jack Clark said last week.
Both companies have floated the idea of international coordination to slow down the whole industry together – including Chinese rivals and Google.
That runs directly against the current dynamic, in which each lab is pushing hard to attract the massive investment needed to stay ahead.
Anthropic last week proposed a global coordination system to temporarily suspend the development of frontier AI – but only if no competitor takes advantage of the pause to get ahead.
That condition is widely seen as nearly impossible to meet anytime soon.
OpenAI on Monday also introduced the idea of a potential multilateral body that could "slow frontier development if necessary."
Neither idea is new. OpenAI has been calling since 2023 for an international AI authority similar to the IAEA, the United Nations nuclear watchdog, with the power to limit the "rate of growth in AI capability."
Today, both OpenAI and Anthropic are putting out new models roughly every four to six weeks.
In February, Anthropic updated its internal guidelines on what behaviors it will never allow.
The revised version now has two columns: what the company refuses to do on its own, and what it thinks the whole industry should avoid, but won't commit to unilaterally.
These pledges are resurfacing at a convenient moment: both companies are in the middle of a race to raise capital.
They each announced in early June that they had filed preliminary IPO applications – not yet public – seen as a way to gauge investor interest and prepare for a swift market listing.
Credibility is being questioned from multiple directions.
David Sacks, a tech entrepreneur who served as the Trump administration's top AI adviser, has accused Anthropic of "fear-mongering" – a charge that has spread widely across the industry and within the administration.
He argues the company is pushing for government regulation of AI with the hidden goal of sidelining competitors. French AI researcher Yann LeCun has made similar arguments.
From the other direction, the "Stop AI" movement says the commitments are empty words.
Both labs' talk of a possible slowdown "comes off to me as pre-IPO marketing, in an effort to address public backlash (against AI)," said Gary Marcus, an academic who has become one of the most prominent critics of AI hype.
Others, like Meredith Whittaker, president of the encrypted messaging app Signal, argue that "the real danger is not in the future" – the distant threats these companies keep raising – but already "in the present," playing out right now through job losses, expanded surveillance and growing concentrations of power.
