The U.N. and Red Cross chiefs warned Tuesday that autonomous weapons, capable of killing without human control, could soon be used in war and urged urgent talks to limit them.
"We are now dangerously close to crossing a moral red line: the autonomous targeting of humans by machines," U.N. Secretary-General Antonio Guterres said in a joint statement with the President of the International Committee of the Red Cross Mirjana Spoljaric.
"Negotiations must begin now to urgently adopt a legally binding instrument to regulate autonomous weapon systems with clear prohibitions and restrictions," they said.
States meet in Geneva next week to discuss potential new rules amid mounting concerns over the role of AI-assisted semi-autonomous weapons that have been used in conflicts in Ukraine, Sudan, Gaza, Iran and the Gulf.
The discussions, which began in 2014, have made some progress but technology has also advanced. Fully autonomous weapons are already in use by some armies, such as anti-missile systems on warships.
The New York Times on Monday reported a Russian drone attack in Ukraine's Zaporizhzhia that was likely guided by "an experimental artificial intelligence system, not a human pilot."
The Times cited drone experts, military commanders and forensics to establish that the drone in question was human-programmed but chose its target on its own – a first such case.
"It is a development long watched for in the war in Ukraine, presaging a dystopian future in which killer robots roam the skies making life-or-death decisions that reduce targets to data points," the U.S. daily said.
Human rights groups worry that AI-driven systems cannot be adequately predicted or controlled, raising the risk of fatal errors such as a failure to recognize an injured or surrendering soldier. And once such errors occur, it is harder to hold perpetrators to account in any future war crimes cases, they say.
"States must show political courage and move beyond incremental discussions towards decisive action," the joint statement said, calling for those talks to begin in November when a major review conference that occurs only once every five years is planned in Geneva.
"Some decisions must forever remain with the human being, none more than the decision of taking or not a human life," Carolyne Melanie Regimbal, chief of service at the U.N. Office for Disarmament Affairs, told a Geneva press briefing after the joint statement's release.
While states agree international humanitarian law (IHL) applies to lethal autonomous weapons systems, specifically binding international standards for such systems remain virtually non-existent. Russia and the United States, among others, have opposed new legally binding instruments, arguing existing laws are sufficient.
The Geneva talks, which require consensus to lead to formal negotiations, are taking place within a challenging context of geopolitical tensions and recent European withdrawals from the landmine ban treaty, over Russian threats.
