Chewing Detection Using Brightness Changes in Video based on Deep Learning

Abstract

Chewing well is known to be beneficial for human health. However, a simple method to measure the number of chews for health guidance has not been established. In this paper, we propose a simple method to measure the number of chews using a photographic device such as a smart phone. When a video of chewing during eating is filmed, the brightness of the chewer's face changes as the jaw moves up and down due to chewing. When the values are graphed, the change in brightness results in a waveform shape that is easy to understand. Since the number of chews can be estimated from the number of waves in the waveform, the number of chews is measured using a neural network that counts the number of waves. To compensate for the small amount of data, we use a large amount of pseudowaveforms, such as sine waves. Then, a learning model that determines the number of repetitions is created, and a large amount of pseudo-waveform data is used for pre-training. The parameters of the trained model are determining by transfer learning so that the model can be applied to a small amount of data. As a result of learning the video waveform data, we were able to measure 96.6% of the data within ± 2 by moving all the parameters of the trained model.