Seoul, South Korea―REAIM 2024―Internationally recognized experts in artificial intelligence have come together to raise concerns about autonomous AI systems. These leading voices warn that the development and spread of unregulated autonomous AI in the military sector poses serious risks to global peace and security. They argue that such advancements should be strictly prohibited, similar to global efforts to prevent the spread of nuclear weapons.
Saeed Al Dhaheri, Director of the Center for Future Studies at the University of Dubai, took the stage during the plenary session at the 2024 REAIM (Responsible AI in the Military Domain) Summit on Monday, calling for a robust international framework to regulate the development of autonomous weapons systems integrated with AI.
Referring to a statement published by the International Committee of the Red Cross (ICRC) in March of 2021, which warned that “weapon systems with autonomy in their critical functions of selecting and attacking targets are an immediate humanitarian concern,” Al Dhaheri explained that autonomous AI systems undermine global stability by challenging human accountability and increasing complexity-related risks, such as occur in system malfunctions.
“The proliferation of these systems could trigger an international arms race while increasing reliance on AI, potentially leading to decisions that erode global peace,” said Al Dhaheri. He further warned that in AI-driven cyber warfare, unreliable technology could be exploited by adversaries to disrupt citizens’ daily lives, target critical infrastructure, and threaten nuclear facilities, ultimately escalating cyber conflicts.
Mike Baylor, Chief Data and AI Officer at Lockheed Martin, compared the current situation with AI to a wildfire, particularly regarding command-and-control (C2) in firefighting. Just as firefighters rely on C2 systems—which feature a clear chain of command, robust communication, efficient resource management, and continuous situational awareness—to predict wildfire behavior and mitigate damage, Baylor argued that the international community needs to develop similar C2 systems to manage the rapid advancement of AI technology.
General Lee Young-su, Chief of Staff of the Air Force of the Republic of Korea, added his thoughts to Baylor’s assessment. “AI-powered systems are restructuring command-and-control in the military,” said the General. “The military cannot achieve its combat objectives without precise C2 systems in place. It is crucial to acquire systematic tools and build organized structures to mitigate the potential threat of unrestrained AI.”
Additionally, Frederick Choo, Deputy Secretary for Policy at the Ministry of Defense in Singapore, emphasized the importance of mitigation. “My job is to anticipate the worst-case scenarios and prepare as thoroughly as possible,” said Choo. “The train has pretty much left the station, with players pursuing AI regardless of warnings. There’s little effort to reverse this trend. What we need now is to come together and develop mitigation strategies focused on responsibility, reliability, and robustness.”
In a separate session following the plenary, Stuart Russell, renowned computer scientist for his contributions to AI and the author of “Artificial Intelligence: A Modern Approach” published in 1995, warned of the significant risks posed by autonomous weapons systems using AI. Russell defined “autonomous” as a state where human intervention is not required in managing the system, such as operating a weapon, monitoring through a camera, or deciding when to fire a missile.
“Since no human management or piloting is required, you could potentially launch as many weapons as you want. This means there is no limit to the number of weapons a single individual can deploy with the press of a button. This is what we refer to as scalable weapons of mass destruction,” the expert elaborated.
“Creating scalable weapons of mass destruction—cheap and easily proliferated—would undermine international security,” said Russell. “Verification and enforcement are essential for trust among parties. Consider the extensive time and effort invested in developing the science, engineering, and system planning for the verification regime on nuclear testing; without it, the treaty would never have been signed. We need to apply a similar approach to autonomous AI systems in the military realm.”
The REAIM Summit 2024 is the most comprehensive and inclusive platform for AI in the military domain. Held from September 9 to 10, the event is co-organized by the Ministry of Foreign Affairs (MOFA) and the Ministry of National Defense (MND) of the Republic of Korea.
Attendees included Minister of Foreign Affairs Cho Tae-yul, Minister of National Defense Kim Yong-hyun of the Republic of Korea, and heads of delegations from co-hosting countries: the Netherlands, Singapore, Kenya, and the United Kingdom. In total, around 2,000 people from more than 90 countries—representing government, industry, academia, and international organizations—traveled to Korea to attend the Summit.
In his opening remarks, Foreign Minister Cho emphasized the importance of establishing norms and governance for responsible AI to ensure international peace and security. He explained that the Korean government is hosting the REAIM Summit 2024 to foster a balanced understanding of AI’s impact on global stability, discuss the implementation of responsible AI applications in the military domain, and envision future governance with input from multiple stakeholders.
In his welcoming remarks, Defense Minister Kim described the application of AI in the military domain as a double-edged sword—capable of enhancing military capabilities but also potentially leading to unintended consequences if abused or misused. He emphasized the importance of promoting a balanced understanding of responsible AI use and fostering international cooperation.
Defense Minister of the Netherlands, Ruben Brekelmans, noted that the REAIM Summit has established itself as a key platform for multi-stakeholder dialogue on setting norms to ensure responsible applications of AI.
