A leading artificial intelligence expert has raised concerns that generative AI could pose threats to financial security. While the technology is increasingly being adopted for various business tasks in the financial sector, experts caution that the industry cannot rely solely on AI. This is particularly important given the sector’s responsibility for safeguarding personal information and assets.
On Wednesday, Lee Joo-hyun, Senior Member of the Artificial Intelligence Technology Team at the Financial Security Institute (FSI), delivered a presentation titled “Threats and Challenges of Adopting Generative AI in Finance” at Korea Fintech Week 2024 in Seoul, South Korea. The event centered on financial technology and services enhanced by AI.
Lee emphasized that generative AI is currently employed in hundreds of financial tasks, including customer counseling, investment analysis, and credit loan processing. However, he noted that AI still faces limitations and cannot address all issues. Additionally, the use of AI poses risks to customer information and assets.
The expert expressed concerns about the reliability of generative AI, noting that it has the potential to produce false information. This misinformation can lead to significant credibility issues when used unfiltered in business contexts, meaning without human review or moderation. He further explained that malfunctions in AI systems can undermine trust. Additionally, the expert pointed out that AI, on its own, cannot effectively avoid bias or ensure the accuracy of the information it provides.
Lee also warned that AI poses risks to customer privacy. He explained that through sophisticated attacks, AI systems could be manipulated into revealing sensitive customer information, such as names, social security numbers, and addresses. He added that copyright issues related to AI-generated content remain unresolved.
Governments are actively discussing solutions to address the challenges posed by generative AI. Countries like the United States, United Kingdom, and Singapore have established AI safety labs to research and verify the technology’s stability. Additionally, companies such as Google and Naver have introduced frameworks aimed at mitigating AI-related risks and ensuring its safe deployment, the expert noted.
“There are inherent risks with generative AI. Effectively preventing these risks is a core task and a crucial responsibility of the FSI,” stated Lee. “The FSI is currently analyzing global solutions to ensure the safety and reliability of AI systems.”
Additionally, the presentations highlighted the use of AI in finance and discussed regulatory measures implemented both in Korea and internationally.
Seo Ho-jin, Senior Team Manager of the Financial Innovation Support Team at the FSI, detailed how the financial sector is leveraging AI technology. He explained that AI is being used in various applications such as credit scoring, loan screening, investment recommendations, money laundering detection, and facial recognition. However, the use of generative AI remains limited in the domestic financial sector due to concerns about safety, reliability, and customer protection.
Seo noted that compared to other countries, South Korea has proposed and passed fewer regulations related to AI. Despite this, he emphasized that the government is gradually working on developing guidelines and regulations to address these emerging issues. The cautious approach reflects the sector’s need to balance innovation with robust safeguards.
The expert also stressed the importance of human oversight and judgment, even as AI technologies advance. Given that financial services manage people’s assets, human involvement is crucial. “AI should operate alongside existing services to address potential system outages, and there should be contingency plans to prevent service interruptions due to technical issues,” Seo advised.
Related article: AI security draws unprecedented attention, pulling top brains together in three-day workshop
Hongcheon, Gangwon―Released in 1966, the movie The Good, the Bad and the Ugly depicts an uneasy alliance and struggle among three men on a quest for hidden treasure. Echoing the film’s themes, a group of experts in artificial intelligence security recently held a workshop titled “The Good, The Bad, and The Ugly of AI Security.” This theme reflects the current uneasy relationship between humans and AI, particularly heightened by security concerns.
Kwon Taekyoung, a professor of information security and AI at Yonsei University, has been leading the AI Security Research Group since the beginning of this year. The group, established in 2019, operates under the Korea Institute of Information Security & Cryptology (KIISC), South Korea’s leading academic organization in cybersecurity. It organizes annual events, including the AI Security Workshop. READ MORE
