AdaShift: Anti-Collapse and Real-Time Deep Model Evolution for Mobile Vision Applications

Abstract

As computational hardware advance, integrating deep learning (DL) models into mobile devices has become ubiquitous for visual tasks. However, “data distribution shift” in live sensory data can lead to a degradation in the accuracy of mobile DL models. Conventional domain adaptation methods, constrained by their dependence on pre-compiled static datasets for offline adaptation, exhibit fundamental limitations in real-time practicality. While modern online adaptation methodologies enable incremental model evolution, they remain plagued by two critical shortcomings: computational latency from excessive resource demands on mobile devices that compromise temporal responsiveness, and accuracy collapse stemming from error accumulation through unreliable pseudo-labeling processes. To address these challenges, we introduce AdaShift, an innovative cloud-assisted framework enabling real-time online model adaptation for vision-based mobile systems operating under non-stationary data distributions. Specifically, to ensure real-time performance, the adaptation trigger and plug-and-play adaptation mechanisms are proposed to minimize redundant adaptation requests and reduce per-request costs. To prevent accuracy collapse, AdaShift introduces a novel anti-collapse parameter restoration mechanism that explicitly recovers knowledge, ensuring stable accuracy improvements during model evolution. Through extensive experiments across various vision tasks and model architectures, AdaShift demonstrates superior accuracy and 100ms-level adaptation latency, achieving an optimal balance between accuracy and real-time performance compared to baselines.