Embedded feature selection using dual-network architecture

Feature selection is essential for eliminating noise, reducing redundancy, simplifying computational complexity, and lowering data collection and processing costs. However, existing methods often face challenges due to the complexity of feature interdependencies, uncertainty regarding the exact number of relevant features, and the need for hyperparameter optimization, which increases methodological complexity.

This research proposes a novel dual-network architecture for feature selection that addresses these issues. The architecture consists of a task model and a selection model. First, redundant features are fed into the selection model, which generates a binary mask aligned with the input feature dimensions. This mask is applied to a shifted version of the original features, serving as input to the task model. The task model then uses the selected features to perform the target supervised task. Simultaneously, the selection model aims to minimize the cumulative value of the mask, thus selecting the most relevant features with minimal impact on the task model’s performance.

The method is evaluated using benchmark and synthetic datasets across different supervised tasks. Comparative evaluation with state-of-the-art techniques demonstrates that the proposed approach exhibits superior or competitive feature selection capabilities, achieving a reduction of 90% or more in feature count. This is particularly notable in the presence of non-linear feature interdependencies. The key advantages of the proposed method are its ability to self-determine the number of relevant features needed for the supervised task and its simplicity, requiring the pre-definition of only a single hyperparameter, for which an estimation approach is suggested.