Britain's King Charles III warned on Thursday of the "existential dangers" of artificial intelligence systems falling into the wrong hands, as he addressed the tech elite at a summit on the future of the technology.
Charles told an audience that included bosses from Nvidia, Anthropic, Google DeepMind and OpenAI that decisions taken today on the technology will "shape the world inherited by future generations," and he called on them to ensure that as they develop the technology, "our humanity remains sacred."
The event at Dumfries House in Cumnock, East Ayrshire, was also attended by guests including the chief adviser to the Vatican on the ethics of AI and technology, the U.K.'s AI minister, government advisers and others.
Giving his opening address, Charles told the audience the development of AI is "both intriguing and deeply concerning in equal measure."
"AI is already showing its immense capacity to improve and save life – for example, in the field of life sciences and medicine," he said.
"Yet, those who have created these technologies are now increasingly warning that AI risks developing darker capacities – perhaps even to take life.
"In this regard, if I may say so, there seems urgency in adequately considering the existential dangers of such technologies falling into the wrong hands, and being used in potentially catastrophic ways."
"Surely, we need sufficient means of control before it is all too late?"
"The decisions taken at this formative time will shape the world inherited by future generations," he added.
"And so, the task before you is not merely to advance technology, but to ensure that it remains firmly in the service of humanity, community and the natural world.
"As we embrace what is new, we must not lose what is timeless."
The event comes as an excerpt from a new book called "Swan Song," about the late Diana, Princess of Wales, written by her brother Earl Spencer, was published by the Daily Mail, claiming that after her death in 1997, the king allegedly said: "rest assured, we'll forget her soon enough."
Buckingham Palace pushed back against the claim, with a spokesperson saying Charles is "mindful that the pain of fraternal grief can cloud reason."
Doomsday warnings have been levelled at AI, seen by some as a threat to the existence of humanity, sparking a debate among industry leaders over how best to rein in the technology before it could escape human control.
On Thursday, the chief executive of Microsoft AI said: "We're essentially seeding a new silicon species" and suggested competitors Anthropic were paving the way.
Mustafa Suleyman told the BBC's Today program: "I think that if we all create AIs that are able to act autonomously, that can define their own objectives, that can earn money, that can own assets, that could run businesses, we're essentially seeding a new silicon species which will no doubt compete with us for resources – no matter how much it cares about humanity and loves us.
"Which is, I think, Anthropic's intention. I think that is mistaken and misguided but we have to empirically test it."
"And so if they believe that this is a safer route to developing AIs then they have to share the data to prove that, and I think instead we should be creating AIs that don't introspect."
Anthropic researcher Evan Hubinger sparked headlines around the world when he warned last week that he believed there was a greater than 10% chance AI could pose a species-ending risk within 10 years.