6,600 daily data sets from elderly people living alone released
Accelerated Research on Home Service Robot AI


The Electronics and Telecommunications Research Institute (ETRI) announced on the 5th that it will release a 3D image dataset (DataSet) for research on recognizing the daily behavior of the elderly in a robotic environment through its website .

Research on senior care robots requires data capturing elderly individuals from the robot's perspective. However, there is a critical shortage of data suitable for robot research yet capable of recognizing human daily behavior.

The lack of data specifically targeting the elderly has posed challenges for research. Since 2017, the research team has been conducting behavioral recognition research for senior care robots, collecting data from a variety of environments.

Since 2018, ETRI has been collaborating with the Comprehensive Support Center for Senior Citizens Living Alone to capture data by using 3D cameras to film 30 homes where seniors actually live.

The total number of publicly available datasets is 6,589. It is significant as it is the world's first 3D behavioral recognition data captured from a robot's perspective in a senior citizen's actual living environment.

Last November, ETRI also collected data from 100 participants in an apartment testbed environment simulating a residential environment. At the time, ETRI secured 112,620 3D images of 55 daily activities, releasing the world's largest 3D image dataset (ETRI-Activity3D).

The high-quality data obtained through the research team's efforts is provided for research purposes to numerous domestic and international companies, schools, and research institutes through agreements.

In fact, the data released last year was utilized in various ways, including through agreements with about 30 domestic institutions, including large corporations and universities, and about 10 overseas research institutes and universities for research purposes.

Meanwhile, the Artificial Intelligence Research Group of the Korea Institute of Science and Technology (KIST), a joint research institute, also developed and released a platform that can create virtual behavioral data. By utilizing the platform, it is possible to obtain large amounts of diverse behavioral data by changing the robot's viewpoint, lighting, etc. in environments where direct filming is difficult.

In addition, ETRI is also releasing data and core technologies for learning nonverbal interaction behaviors of robots in open source form.

So far, the research team has revealed a total of eight technologies, including: △Action detection technology that detects the time when the elderly's daily actions occurred; △External feature recognition technology that recognizes external information such as facial features and clothing; △Technology that recognizes the color and style of clothing; and △Technology that automatically generates gestures appropriate for speech.

Related information can be found on the GitHub site .

Dr. Jae-yeon Lee of the ETRI Human-Robot Interaction Laboratory said, “I hope that by overcoming the difficulties in conducting related research due to lack of data, we can accelerate research on senior care robots that enable interaction between robots and seniors, and bring forward commercialization.”

The research team plans to add 20 households by the end of the project next year, further improving data quality and making all technologies developed through the project publicly available. Furthermore, based on the acquired data, the team aims to conduct various validation projects to obtain more practical data and software technologies.