The U.N. chief urged on Monday for a global governance system to shape artificial intelligence for the good of humanity, while flagging its risks and warning against allowing the technology to "vibe-code" the future of humanity.
With AI advancing at "runaway speed," U.N. Secretary-General Antonio Guterres cautioned that "an experiment is being run on our own societies, without a plan and without consent."
"That is not sustainable," he said, speaking before a first Global Dialogue on AI Governance in Geneva, which brings together governments, tech companies, academia and civil society.
"AI is already transforming our world," he pointed out.
"The question is whether we will shape this transformation together, or let it shape us."
Guterres warned that AI systems were "no longer tools awaiting instruction."
"They are writing code, acting online and making choices with less and less human oversight," he pointed out.
"Our institutions were built to govern machines that follow commands. They are not ready for machines that decide."
Guterres voiced concern at how AI was further obscuring what is true and false, and highlighted a growing tendency to leave important tasks up to the technology and blindly trust the results.
So-called "vibe-coding," or using AI to tell a machine what you want instead of coding it yourself, "can do wonders," he acknowledged.
"But we cannot vibe-code the truth. We cannot vibe-code the future of humanity."
Another risk flagged by Guterres was the concentration of power in a handful of AI companies and in a handful of countries.
Most countries "have had no say in decisions that will shape their futures," he warned.
In the face of such challenges, he said countries faced a stark choice, "between governing by design and drifting by default."
The U.N. chief highlighted the potential of AI technologies for everything from accelerating development to improving health care and providing broader access to education.
But he insisted developments needed to be guided by several key priorities, including safety and respect for human rights, to ensure that people everywhere reap the benefits.
He called for "common methods to evaluate and verify risks" and jointly-agreed standards, particularly for ensuring the safety of children accessing AI systems.
"We do not let medicine reach a child until it is proven safe. We test every toy," Guterres pointed out.
"Yet AI has reached our children – their learning, their friendships, their most private questions – before anyone asked what it would do to them."
Guterres called for an AI Child Safety Pledge, requiring companies to prove that any system accessible to children is safe and has zero tolerance for sexual abuse.
The systems must also connect any child showing signs of distress to real human support, he said.
"No child should be a guinea pig for unregulated AI," he insisted.
Boosting AI capacity and access in developing countries was also key, he said, to ensure that the existing, deep digital divide does not "harden into an AI divide."
Guterres said he would urge the U.N. General Assembly to create a Global Fund for AI, "to build skills, data and affordable computing power everywhere."
Another priority was reducing AI's climate impact, he said, reiterating his call for companies to reveal their growing environmental footprint and to commit to powering every data center with renewable energy by 2030.
The U.N. chief, meanwhile, said that his biggest concern revolved around AI in military settings, and in particular so-called lethal autonomous weapon systems.
"Let us call them what they are: Killer robots," he said.
"Machines selecting and engaging their target and taking a life – without human control and judgement."
"That is morally repugnant... And it must be banned by international law."
Guterres stressed the urgency of creating sufficient guardrails to steer AI in a positive direction.
"We may be the last generation able to set the terms on which humanity and machines coexist," he said.
"The door is still open. It will not stay open long."
