Recognition of Air Traffic Controller’s Mouth State Based on Deep Convolutional Neural Network

Abstract

The fatigue of air traffic controllers (ATCs) is an critical factor that threatens the safety of aviation operation. The existing yawn detection methods are less adaptable to the ATCs. In order to distinguish the two states of ATC radiotelephony communications and yawning, a yawn detection model with higher accuracy is established. Firstly, the region of interest (ROI) is extracted from 68 feature points of the face to extract the image as input to the neural network; secondly, convolution neural network is used to construct the classification model, and the mouth changes are divided into three categories; finally, the CAUC-YH data set is used to simulate the video production training and verification of the controller’s work, and the times of three recognition results are recorded respectively to realize the yawn recognition in a section of controller’s work video. It has been verified that this method has an accuracy of 89.75% on the data set.