Automatic Estimation of Neonatal Sleep/Wake States in the NICU Using 3D CNN

In the neonatal intensive care unit (NICU), the preterm infant located in incubator takes various medical care day and night. Unusual environment in NICU may affect neurodevelopment of newborn subject, some researches evaluate the sleep/wake state of subject by visual or using the Actigraph attached on the leg. This paper proposes a sleep/wake status estimation method using video images and convolutional neural network (CNN) to reduce assessment time and improve the reliability. The Brazelton’s criteria evaluates the newborn’s sleep/wake states in six stages, the proposed method performs six-class classification using 3D CNN. In the experiment, we conducted 4 experiments by using original data, two different frame shifting, and using the frame differential. Experimental results using 16 video of 8 subjects showed that the training using original dataset achieved the highest macro-F1 value (0.766) which improves the macro-F1 value (0.765) of our previous result using support vector machine (SVM) and optical flow. Results also suggested that the 3D CNN improves the classification accuracy but the data augmentation using frame shift is not suitable to our dataset.