A Novel Continual Learning and Adaptive Sensing State Response-Based Target Recognition and Long-Term Tracking Framework for Smart Industrial Applications

Abstract

Purpose

With the rapid development of artificial intelligence technology, highly intelligent and unmanned factories have become an important trend. In the complex environments of smart factories, the long-term tracking and inspection of specified targets, such as operators and special products, as well as comprehensive visual recognition and decision-making capabilities throughout the whole production process, are critical components of automated unmanned factories. However, challenges such as target occlusion and disappearance frequently occur, complicating long-term tracking. Currently, there is limited research specifically focused on developing robust and comprehensive long-term visual tracking frameworks for unmanned factories, particularly those designed to integrate with embedded platforms and overcome various challenges.

Methods

We first construct three new benchmark datasets in the complex workshop environment of a smart factory (referred to as SF-Complex3 data), which include challenging conditions such as complete occlusion and partial occlusion of targets. A brain memory-inspired approach is used to determine uncertainty estimation parameters, including confidence, peak-to-sidelobe ratio and average peak-to-correlation energy, to develop a continual learning-based adaptive model update method. Additionally, we design a lightweight target detection model to automatically detect and locate targets in the initial frame and during re-detection. Finally, we integrate the algorithm with ground mobile robots and unmanned aerial vehicles-based imaging and processing equipment to build a new visual detection and tracking framework, smart factory complex recognition and tracking.

Results

We conducted extensive tests on the benchmark UAV20L and SF-Complex3 datasets. The proposed algorithm demonstrates an average performance improvement of 6% when addressing key challenging attributes, compared to state-of-the-art tracking methods. Additionally, the algorithm was capable of running efficiently on embedded platforms, including mobile robots and UAVs, at a real-time speed of 36.4 frames per second.

Conclusions

The proposed SFC-RT framework effectively addresses the challenges of target loss and occlusion in long-term tracking within complex smart factory environments. The framework meets the requirements for real-time performance, robustness and lightweight design, making it well suited for practical deployment.