LightHybridNet-Transformer-FFIA: A hybrid Transformer based deep learning model for enhanced fish feeding intensity classification

Abstract

Accurate assessment of fish feeding intensity is important for efficient and sustainable aquaculture. This paper introduces LightHybridNet-Transformer-FFIA, a novel and parameter-efficient hybrid neural network for automated fish feeding intensity classification in aquaculture. Addressing the critical need for optimized feed management, our model utilizes the fusion of sonar imagery and Mel spectrograms to accurately assess feeding intensity levels. LightHybridNet-Transformer-FFIA integrates a Convolutional Neural Network branch for spatial feature extraction from sonar images with a Transformer branch for capturing temporal dynamics in Mel spectrograms, fused by a Feature Fusion and Interaction Aggregation (FFIA) module. Evaluated on the MRS-FFIA dataset, our model achieves a high validation accuracy of 95.42% and a macro-averaged F1-score of 95.40%, demonstrating competitive performance against state-of-the-art multi-modal models while utilizing a significantly smaller parameter footprint (0.102 million parameters). The architectural novelty and parameter efficiency of LightHybridNet-Transformer-FFIA present a promising solution for real-time aquaculture monitoring, enabling optimized feed delivery, reduced waste, and improved sustainability. This work highlights the effectiveness of hybrid CNN-Transformer architectures for multi-modal underwater sensing and contributes a practically deployable model for intelligent aquaculture management.