The development of a technology capable of restoring high-quality 3D characters from a single high-resolution image is expected to contribute to the development of metaverse services that can be applied to real-life situations in the future.

The Korea Electronics Technology Institute (KETI, President Shin Hee-dong) announced on the 19th that it has developed an 'AI-based 3D object media restoration technology' through joint research with the Gwangju Institute of Science and Technology (GIST) AI Graduate School and will present it at CVPR (Computer Vision and Pattern Recognition), the world's most prestigious academic conference in the field of computer vision.

The biggest feature of the artificial intelligence (AI) algorithm developed by the KETI Intelligent Image Processing Research Center (Drs. Joo-Hong Yoon and Dr. Min-Kyu Park) and the GIST Professor Jeon Hae-Gon's team is that it can restore high-quality 3D people from a single high-resolution image.

In order to process high-resolution images with efficient memory, the KETI·GIST research team segmented each body part of the person in the video, including the face, torso, arms, and legs, and then estimated individual depth information through a multi-stage deep learning algorithm. />
The technology developed by the research team is expected to improve the difficulties (high cost, technical limitations, etc.) of the existing real-life-based method using studio multi-cameras, as it can significantly reduce the cost and time of work by restoring a 3D person using only a single video image.

According to Dr. Yoon Joo-hong of KETI, who participated in the technology development, the research team collaborated with 3D photo scan company EOIS Co., Ltd. to build a '2K2K dataset' containing 3D model information of approximately 2,000 virtual people, and plans to make it public through Github, the world's largest open source community.

In particular, the 2K2K dataset is expected to improve the 'data shortage' and 'male bias' phenomena that were pointed out as problems in 3D modeling in the past by satisfying demographic balance that takes into account various characteristics of characters such as race, gender, age, posture, and hairstyle.

This technology has already been recognized for its achievements, winning a silver prize at the 29th Samsung Human Tech Paper Awards hosted by Samsung Electronics in January, and is scheduled to be announced as one of the top 10% papers (Highlight) at 'CVPR 2023', which will be held in Vancouver, Canada for five days from the 18th.

Kim Je-woo, head of KETI's Intelligent Image Processing Research Center, said, "Realistic and efficient 3D character modeling based on AI algorithms is a key element in expressing virtual reality and digital twin space, and we expect it to ultimately contribute to the development of metaverse services that can be applied to real life."

In addition, GIST Professor Jeon Hae-gon said, “The research team was able to receive recognition for its high academic value by resolving the problems of data bias and low-quality human 3D datasets and proposing a baseline artificial intelligence algorithm that can easily restore them.”quo; he mentioned.

Meanwhile, this research was conducted through the 'Development of Object Media Processing Technology for Multi-Source Images' project hosted by KETI with the support of the Ministry of Science and ICT and the National IT Industry Promotion Agency, the 'Development of User-Participatory Metaverse Performance Solution Based on Neural Human Modeling' project hosted by the Ministry of Trade, Industry and Energy and the Korea Institute for Advancement of Technology, and the 'Convergence Cultural Virtual Studio for Implementation of AI-based Metaverse' project hosted by GIST with the support of Gwangju Metropolitan City.