A domestic research team has developed a technology that can create and edit three-dimensional images as if the eye could perceive them. This technology is expected to be of great help in creating next-generation content such as virtual reality (VR), augmented reality (AR), and holograms, and enjoying them in a more realistic way.

The Electronics and Telecommunications Research Institute (ETRI) announced that it has developed a technology called 'Unstructured Plenoptic Content Acquisition, Creation, Authoring and Visualization Platform SW' that allows even non-experts to create and edit 3D images without motion sickness.

Plenoptic technology is one of the technologies that can acquire high-dimensional light information and create various three-dimensional images that can be recognized by humans through computer calculations.

Although the market for immersive content is growing, the technologies currently in use have low resolutions or operate differently from the human eye, making them unable to provide full three-dimensionality.

This resulted in many limitations in users' ability to fully experience the experience, such as complaints of dizziness or motion sickness.

ETRI has developed a technology that reduces motion sickness while obtaining high-resolution stereoscopic images as naturally as the human eye.

The researchers used a dedicated plenoptic camera to capture the data and then processed it on a high-performance computer.

In particular, it has the great advantage of being able to freely change the focus and viewpoint of already recorded videos and photos.

For example, if you use this technology in a pansori performance, you can freely change the focus when editing the video to any desired area, such as the drummer or singer.

The viewpoint can also be changed to show characters that were not visible on the main screen due to filming equipment or microphones.

In particular, the research team's technology is plenoptic technology that implements binocular parallax, motion parallax, focus adjustment, and six degrees of freedom similar to the human eye, and can practically express a three-dimensional effect.

ETRI has been researching camera locations and filming targets for the past four years. He explained that this advanced result was made possible based on the know-how gained while conducting research on 'orthopedic plenoptic' technology, which must be performed in a state of near-immobility.

The technology developed this time is an 'atypical plenoptic' technology that allows free shooting without location or mobility restrictions, and its quality is also higher than that of typical technology for the same capacity. The video resolution was also increased from FHD to 4K.

ETRI's technology allows even non-experts to easily edit videos and create results in desired display formats, such as VR head-mounted displays (HMDs), AR glasses, and stereoscopic images without glasses.

Thanks to this, it is expected to be widely utilized to increase the production of realistic content that has been lacking by increasing accessibility for creators.

This technology has great potential for commercialization in various fields, including camera and display manufacturing companies, CG/VFX, content creation and editing companies, image/video tool SW and app developers, medical equipment such as endoscopes, process audit equipment such as semiconductors, security companies such as iris recognition and CCTV, astronomy, and the military.

In fact, ETRI used this technology to film intangible cultural assets such as Bongsan Mask Dance and Pansori at the Asia Culture Center in 2020, and also filmed a Taekkyeon martial arts demonstration at the UNESCO International Martial Arts Center in 2021.

Based on this, it was publicly disclosed at the 'Software Wave' exhibition in 2020 and the 'ICT Commercialization Festival' exhibition in 2021, and the technology was successfully transferred to related companies.

ETRI Holographic Research Lab Principal Researcher Kim Do-hyung, who led the technology development, said, “I hope that through this technology development, our country will be able to lead the industry and services related to next-generation realistic content technology and build its own digital image content ecosystem.”

The research team plans to conduct follow-up research to improve performance, such as increasing the resolution of stereoscopic images to 8K, expanding the user's virtual experience reproduction space, and reducing motion sickness. Additionally, we plan to further develop this technology to the point where it can be used for personal broadcasting using personal smart devices.

This study was conducted as part of the ‘Development of a platform technology for authoring/playback of ultra-high-resolution, non-standard plenoptic video for medium- to large-sized spaces’ project, which is part of the ‘Hologram Core Technology Development Project’ of the Ministry of Science and ICT.

In relation to this achievement, the research team announced three technology transfers, 32 patent applications, nine SCI papers, and seven standardization contributions.