Making data classification more effective: An automated deep forest model

Abstract

Despite a small overfitting risk, the deep forest model and its variants cannot automatically match data features; they rely on manual experience and comparative experiments for forest learner selection. This study proposes an automated deep forest model (ATDF) to enhance deep forest automation by automatically determining forest learners’ types and numbers based on training data. The model introduces a forest learner variability measure based on normalized mutual information, serving as a theoretical foundation for the automated process in deep forests. Then, a novel hierarchical clustering algorithm based on normalized mutual information is proposed to group forest learners at different granularities, determining the optimal forest learner type. This advanced technical method enables the determination of the model structure for stacking models, including deep forests. Finally, with the goal of maximizing cross-validation scores, the tree parson estimator-based Bayesian optimization algorithm determines the ideal number of forest learners for each type. Additionally, a standardized method for identifying forest learners is developed to guarantee the consistency of model outcomes. Most importantly, a series of comparative experiments on seven datasets from the UCI Machine Learning Repository confirmed the effectiveness and superiority of the proposed model. The results demonstrate that the proposed model has superior adaptability to new data and tasks, besides having a high level of automation, and performs excellently in the classification task.