KAIST-SM recognizes cough sounds in real time
Development of a camera that indicates the location of a coughing person
87.4% accuracy, expected to increase further upon commercialization.


As the COVID-19 pandemic continues to spread, demand for contactless infectious disease detection technologies is increasing.

The research team of Professor Yong-Hwa Park of the Department of Mechanical Engineering at the Korea Advanced Institute of Science and Technology (KAIST) announced on the 3rd that they have developed a cough recognition camera that recognizes coughing sounds in real time and displays the location of the coughing person as an image in collaboration with SM Instrument Co., Ltd.

The most common symptoms of COVID-19 are fever and cough. Currently, fever can be assessed without direct contact using a thermal imaging camera. However, it is difficult to easily identify the symptoms of a coughing person using a non-contact method.

To solve this problem, Professor Park's research team developed a deep learning-based cough recognition model that recognizes cough sounds in real time.

In addition, we developed a cough recognition camera that can track and record cough sounds, the location of coughing people, and even the number of coughs in real time by applying a cough recognition model to an acoustic camera to visualize cough sounds and coughing people using the same principle as a thermal imaging camera.

The research team expects that the cough detection camera will be used as medical equipment to detect the spread of infectious diseases in crowded public places or to constantly monitor the condition of patients in hospitals.

◇ Development of a cough recognition camera using CNN-based supervised learning.

The research team applied supervised learning based on a convolutional neural network (CNN) to develop a cough recognition model.

CNN is a type of artificial neural network used to analyze visual images, and supervised learning is a machine learning method for inferring a function from training data.

It receives the feature of a 1-second long sound signal as an input signal and outputs a binary signal of 1 (cough) or 0 (other).To optimize the learning rate, the learning rate value is set to be lowered when the learning rate stagnates for a certain period of time.

Next, for training and evaluation of the cough recognition model, we collected data sets from 'Audioset', a public voice data set widely used for research by Google and YouTube, as well as 'DEMAND', 'ETSI', and 'TIMIT'.

The audio set was used to construct the training and evaluation dataset, and for the other datasets, it was used as background noise for data augmentation to enable the cough recognition model to learn various background noises.

To augment the data, background noise was mixed into the audio set at a ratio of 15% to 75%, and the volume was adjusted by 0.25 to 1.0 times to adapt to different distances. The training and evaluation datasets were composed of the augmented dataset in a 9:1 ratio, and the test dataset was recorded separately in the office.

To optimize the model, we used five acoustic features, including the 'spectrogram', and seven optimizers to learn, and measured the accuracy of the test data set to confirm the performance. As a result, we were able to obtain a test accuracy of 87.4%.

The research team then applied the learned cough recognition model to an acoustic camera, which consists of a microphone array and a camera module that collects sound. The collected data then goes through a beamforming process to calculate the location of the sound source. If the cough recognition model identifies a cough, a contour line and a label indicating the cough origin are displayed at the location.

Professor Park's team conducted preliminary tests of the cough recognition camera in the final stage, and confirmed its applicability in the field by being able to distinguish between coughing sounds and other sounds even in various noisy environments and tracking the person coughing, their location, and the number of times they coughed in real time. They expect that if additional learning is conducted in real-world environments such as hospitals, the accuracy will increase beyond 87.4%.

Professor Park Yong-hwa stated, "With the continued spread of COVID-19, utilizing cough detection cameras in public spaces and crowded facilities will aid in the prevention and early detection of infectious diseases." He added, "In particular, applying them to hospital rooms will allow for 24-hour recording of patients' conditions, which can be used for treatment, reducing the burden on medical staff and enabling a more precise assessment of patient conditions."

Meanwhile, this study was conducted with the support of the Korea Institute of Energy Technology Evaluation and Planning (KITEP) under the Ministry of Trade, Industry and Energy.