A robust wrapper-based feature selection technique based on modified teaching learning based optimization with hierarchical learning scheme

Abstract

Feature selection is a pivotal preprocessing step in deploying machine learning solutions, aimed at removing redundant features from datasets while preserving predictive accuracy. Despite the popular use of wrapper-based feature selection techniques with metaheuristic search algorithms (MSAs) such as Teaching Learning Based Optimization (TLBO), balancing exploration and exploitation in identifying optimal feature subsets remains a fundamental challenge. This paper introduces an advanced wrapper-based feature selection method utilizing an enhanced TLBO variant, termed Modified TLBO with Hierarchical Learning Scheme (MTLBO-HLS). MTLBO-HLS incorporates significant enhancements in both teacher and learner phases to better balance exploration and exploitation, thus improving robustness in solving complex problems like feature selection. Specifically, the Multi-Hierarchical Teacher Phase (MH-TP) divides the MTLBO-HLS population into multiple group tiers based on fitness, where learners are guided by superior peers from better tiers. The Dynamic Peers Interaction Based Learner Phase (DPI-LP) enriches the learning process by enabling interactions with single and multiple elite peers and retaining valuable knowledge across dimensions. Extensive simulations demonstrate MTLBO-HLS’s superior performance in feature selection, consistently identifying the smallest feature subsets and achieving the highest classification accuracies across various datasets. These results highlight its potential to drive industrial innovation through enhanced machine learning model efficiency and accuracy.