Evaluation of automated pediatric sleep stage classification using U-Sleep: a convolutional neural network.

Abstract

Study objectives: U-Sleep is a publicly available automated sleep stager, but has not been independently validated using pediatric data. We aimed to a) test the hypothesis that U-Sleep performance is equivalent to trained humans, using a concordance dataset of 50 pediatric polysomnogram excerpts scored by multiple trained scorers, and b) identify clinical and demographic characteristics that impact U-Sleep accuracy, using a clinical dataset of 3114 polysomnograms from a tertiary center.

Methods: Agreement between U-Sleep and 'gold' 30-second epoch sleep staging was determined across both datasets. Utilizing the concordance dataset, the hypothesis of equivalence between human scorers and U-Sleep was tested using a Wilcoxon two one-sided test (TOST). Multivariable regression and generalized additive modelling were used on the clinical dataset to estimate the effects of age, comorbidities and polysomnographic findings on U-Sleep performance.

Results: The median (interquartile range) Cohen's kappa agreement of U-Sleep and individual trained humans relative to "gold" scoring for 5-stage sleep staging in the concordance dataset were similar, kappa=0.79 (0.19) vs 0.78 (0.13) respectively, and satisfied statistical equivalence (TOST p < 0.01). Median (interquartile range) kappa agreement between U-Sleep 2.0 and clinical sleep-staging was kappa=0.69 (0.22). Modelling indicated lower performance for children < 2 years, those with medical comorbidities possibly altering sleep electroencephalography (kappa reduction=0.07-0.15) and those with decreased sleep efficiency or sleep-disordered breathing (kappa reduction=0.1).

Conclusions: While U-Sleep algorithms showed statistically equivalent performance to trained scorers, accuracy was lower in children < 2 years and those with sleep-disordered breathing or comorbidities affecting electroencephalography. U-Sleep is suitable for pediatric clinical utilization provided automated staging is followed by expert clinician review.