Fish feeding behavior recognition using time-domain and frequency-domain signals fusion from six-axis inertial sensors

In aquaculture, real-time recognition of fish feeding activities is important for enhancing feed conversion rate and reducing production costs. Therefore, this study uses a six-axis inertial sensor to collect water surface fluctuation caused by fish feeding, and proposes a time-domain and frequency-domain fusion model (TFFormer) for fish feeding behavior recognition, and identifies the feeding intensity of fish as four categories: Strong, Medium, Weak, and None. The implementation details are as follows: Firstly, the data collected by the six-axis inertial sensor is preprocessed using a sliding window to obtain time series data, and perform Fourier transform on it to obtain the frequency domain sequence. Then, the transformer is used to unify the time domain and frequency domain features respectively. A Mutual Promotion Unit (MPU) is established based on cross self-attention and a feedforward neural network (FFN). By integrating with a Global multimodal fusion (G) module, MPU establishes a global–local interactive learning framework to extract features from temporal and frequency domains, resulting in temporal-frequency interaction features. Finally, the introduction of supervised contrastive loss function supervises the training process, enhancing the accuracy of fish school feeding intensity classification. Experimental results demonstrate that the proposed TFFormer model effectively processes both temporal and frequency signals, achieving an accuracy of 91.52%, a 5.56% improvement over the baseline model and provides technical support for the development of intelligent feeding machines.