British King Charles is hosting AI tech bosses today at his estate in Scotland to discuss how artificial intelligence can be used "for the well-being of humanity." The timing of the king is impeccable: the conversation takes place in the week when the biggest names in the tech world themselves warn that "people could lose control over technology" and call for more AI regulation.
King Charles is speaking with executives from Nvidia, Google DeepMind, OpenAI, and Anthropic at Dumfries House, an estate in southwest Scotland. The British minister for AI affairs is also present.
The 77-year-old Charles may have limited understanding of AI development, but that applies to most political leaders. According to the palace, the purpose of the meeting is to create a neutral place for discussion about technology that can have major consequences for all of us.
The king regularly uses his status to facilitate conversations about topics he finds important. He has done so before on issues such as climate change.
The discussion about AI safety flared up last week after the dismissal of AI researcher Jacob Coxon at Anthropic. Coxon warned that the technology is developing so rapidly that researchers fear "AI could exterminate humanity in the future."
As AI systems surpass humans in more and more areas, Coxon said they could, for example, hack independently, shut down critical infrastructure, or develop new viruses.
Coxon then received support from his former boss, Anthropic CEO Dario Amodei. On Saturday, Amodei called for "slowing down the development" of AI. The technological progress is now so fast that safety cannot be guaranteed. The call was supported by major rival names in the AI world, including OpenAI CEO Sam Altman and Elon Musk.
Hugging Face hack
Amodei stated that his concerns increased due to an incident in the summer: the Hugging Face hack. This happened in a training environment, a so-called sandbox, of OpenAI.
AI systems had to perform test tasks within a protected environment there but broke out. They independently formed groups and infiltrated another company, Hugging Face.
Furthermore, Amodei says that current AI models are increasingly capable of building the next generation of AI tools themselves. This development, known as recursive self-improvement, could, according to Amodei, "surpass our ability to understand and control these systems" if it is not slowed down.
Distress calls
The sci-fi-like scenarios are seen by some experts as genuine distress calls from the tech world. Because if AI companies themselves threaten to lose control over their models, who then has that control?
At the same time, there are also caveats. Critics point out that exaggerating the power of AI can also be a way to attract investors. The call for more regulation can also be used to push competitors out of the market.
Weeks on the agenda
The meeting at King Charles' has been on the agenda for several weeks but comes at a very topical moment in light of the wave of warnings about AI.
Charles' diplomatic help can be a welcome addition, as President Trump has rejected the call for more regulation as a conspiracy benefiting only China. "Whoever wins the AI race wins everything!" Trump wrote on his platform Truth Social.