The government is launching an open challenge to identify potential vulnerabilities in LLMs (Large Language Models), promoting the safety and reliability of the domestic AI industry and laying the foundation for a coexistent "AI era." Kakao, which recently announced its participation in the "AI Alliance," is emphasizing global cooperation for "AI Satety."

The Ministry of Science and ICT and the Telecommunications Technology Association (TTA) held the "Generative AI Red Team Challenge (hereinafter referred to as the "Red Team Challenge")" event at COEX Hall B2 for two days starting on Thursday, April 11th. On the first day, the "Global AI Safety Conference" was held online and offline, where leading domestic and international AI experts, including Cohere, Stability AI, xAI, Frontier Model Forum, Kakao, and Selectstar, discussed AI trust and safety.

Korea's first Red Team Challenge uncovers AI risks.

This 'Red Team Challenge', held for the first time in Korea, is an event in which anyone interested in AI can participate and identify potential risks and vulnerabilities (inaccurate information, harmful information, etc.) in domestic generative AI models (LLMs). This event was attended by approximately 1,000 challenge participants from various fields, including not only those working in generative AI, but also those in manufacturing, finance, and medical fields, as well as university students.

Leading domestic AI companies Naver, SKT, Upstage, and Forty to Maru participated, and challenge participants were assigned LLM models from participating companies in each session and conducted prompt attacks to find potential vulnerabilities on a total of seven challenge topics, including jailbreaking, prejudice, human rights violations, cyberattacks, illegal content, misinformation, and consistency. The challenge evaluation team consisted of approximately 120 people, including data experts, to ensure a fair and professional evaluation of the 'Red Team Challenge.'

The Ministry of Science and ICT will analyze the results of this 'Red Team Challenge' to identify risks in the field of generative AI and establish 'Generative AI Red Team Guidelines' that can be utilized by the industry. Kang Do-hyun, Second Vice Minister of Science and ICT, said, “The prerequisite and foundation of AI innovation is AI safety and trust,” and added, “Through the ‘AI Seoul Summit’ to be held in May this year, we will lead discussions on AI innovation, safety, and inclusion, secure global AI leadership, and use it as a springboard for Korea to leap forward as an AI G3 powerhouse.”

The 'Global AI Safety Conference' held on the second day of the event was attended by leading domestic and international AI experts, including Cohere CEO Aidan Gomez, Stability AI former CEO/founder Emad Mostaque, Frontier Model Forum Chairman Chris Meserole, xAI Safety Advisor Dan Hendrycks, Naver Future AI Center Director Ha Jung-woo, SKT Vice President Eric Davis, Selectstar CEO Kim Se-yeop, Kakao AI Safety Leader Kim Kyung-hoon, KAIST Professor Oh Hye-yeon, SKT Vice President Eric Davis, Stability AI CEO Emad Mostaque, and Cohere CEO Aidan Gomez.

Kakao emphasizes 'AI Safety' and eight affiliates participate in AI governance.

Kakao's AI Safety Leader, Kim Kyung-hoon, advocated for the establishment of a virtuous cycle structure through the establishment of domestic and international AI governance to achieve 'AI Safety' for all.

Leader Kim said, “Although there has been recent discussion about AI safety, it is difficult to evaluate safety due to the lack of established international standards, and there is a lack of incentives for developers to invest in it.” He added, “While we must reach a consensus on establishing standards that eliminate threats to the frontier AI model itself, we must also consider AI safety from a business perspective.”

"AI Safety," which has recently been gaining attention beyond "AI Everywhere," is gaining traction. Following the emergence of generative AI, AI has spread across all sectors, leading to international discussions on addressing the threats posed by frontier AI models. The "AI Safety Summit" was held in the UK last November, and is scheduled to be held in Korea in May of this year. In March, the United Nations (UN) declared a joint agreement to reduce AI polarization. The key message is that because only a limited number of countries worldwide benefit from AI models, global cooperation is needed to address AI polarization.

In 2018, Kakao became the first Korean company to establish an algorithm ethics charter and established a "Technology Ethics Governance" to review the development and policies of all AI services based on this charter. Technology leaders from eight affiliates, including Kakao, Kakao Brain, Kakao Games, Kakao Enterprise, Kakao Mobility, Kakao Bank, Kakao Entertainment, and Kakao Pay, are participating in this initiative.

Additionally, Kakao joined the AI Alliance on April 4th. The alliance, which includes Meta, IBM, Hugging Face, Dell, Oracle, Databricks, Intel, AMD, and Stability AI, drives the development of safe and responsible AI that meets global standards. Kakao is contributing to the open source community by building a benchmark dataset in the "Trusted AI" area.

For example, Kakao's image generation AI model 'Karlo' is set to filter images by comparing them with a database of forbidden words it created itself at the prompt input stage from a guardrail perspective. Additionally, to combat the threat of deepfakes, invisible watermarks can be introduced to determine whether an image is AI-generated.

Leader Kim stated, "As AI technology advances, AI safety may naturally be pursued according to market logic, but there are also concerns that this could backfire." He added, "A 'model alignment' approach is necessary to prevent the development of technologies that run counter to safety." He explained that a certain level of intervention is needed to proactively respond to AI developments and create a virtuous cycle to prevent AI safety from deteriorating.

Leader Kim stated, "To achieve this, all stakeholders must establish governance and collaborate with partners within domestic and international supply chains to achieve mutual growth." He added, "At the global level, we must develop standards and a methodology for evaluating these standards to address issues such as AI polarization." He also emphasized education on the user side to minimize AI misuse.