Analysis of ensemble-combination strategies for improving inter-database generalization of deep-learning-based automatic sleep staging

Abstract

Deep learning has demonstrated its usefulness in reaching top-level performance on a number of application domains. However, the achievement of robust prediction capabilities on multi-database scenarios referring to a common task is still a broad of concern. The problem arises associated with different sources of variability modulating the respective database generative processes. Hence, even though great performance can be obtained during validation on a local (source) dataset, maintenance of prediction capabilities on external databases, or target domains, is usually problematic. Such scenario has been studied in the past by the authors in the context of inter-database generalization in the domain of sleep medicine. In this work we build up over past work and explore the use of different local deep-learning model's combination strategies to analyze their effects on the resulting inter-database generalization performance. More specifically, we investigate the use of three different ensemble combination strategies, namely max-voting, output averaging, and weighted Nelder-Mead output combination, and compare them to the more classical database-aggregation approach. We compare the performance resulting from each of these strategies using six independent, heterogeneous and open sleep staging databases. Based on the results of our experimentation we analyze and discuss the advantages and disadvantages of each of the examined approaches.