Spectral Band Attention Networks for Efficient Multi-Feature Fusion in Hyperspectral and RGB Data with Ensemble Deep Learning Networks

Abstract

Rapid and nondestructive seed variety identification is crucial for improving agricultural efficiency. Hyperspectral imaging is a powerful tool for this task; however, its high dimensionality and redundant bands can lead to overfitting, while its lower spatial resolution makes it challenging to distinguish individual seeds. A spectral band attention network is proposed to overcome the issue of high dimensionality and redundant bands. Further, an ensemble model is developed that integrates the spectral and spatial features extracted from hyperspectral and RGB images, respectively. A large dataset comprising 96 Indian wheat varieties was prepared using RGB and hyperspectral imaging (900-1700 nm). The ensemble model comprises four deep convolutional neural networks, Customized DenseNet, GoogLeNet, ResNet34, and DenseNet121, with a Support Vector Machine classifier for the final prediction of the seed class. The model’s performance was evaluated using spectral band subsets selected through the band selection techniques, which included Spectral Band Attention Network, Sparse Band Attention Network, Principal Component Analysis-loading, Successive Projection Algorithm, and Triplet-attention. The proposed spectral band attention network outperformed other methods, identifying 25 optimal spectral bands, enabling the ensemble model to achieve a test accuracy of 95.75%. These findings highlight the potential of the proposed spectral band attention network and ensemble model for accurately identifying the wheat varieties. The source code is available at GitHub Repository: SBAN for Multi-Feature Fusion