CMF: Prediction refinement via complementary manifold-based multi-model fusion

Abstract

In current research on multi-model fusion, mainstream approaches predominantly focus on the design of fusion algorithms, while often overlooking the filtering or selection of outputs from individual base models prior to fusion. Moreover, most existing fusion methods exhibit a high degree of coupling, which limits their flexibility and adaptability in cross-scene applications. Consequently, once the fusion is completed, the model architecture tends to become fixed, making it difficult to integrate new models or replace outdated components. To address these limitations and achieve effective state-of-the-art (SOTA) breakthroughs in diverse single-label image classification tasks-such as fine-grained recognition or long-tailed distributions-without being constrained by model architecture, this paper proposes a highly generalizable multi-model complementary method. The proposed approach is applicable to single-label multi-class classification tasks in any deep learning domain and has achieved global SOTA performance on multiple image classification benchmarks. It imposes no restrictions on the architecture, parameter settings, or training strategies of the base models, enabling direct integration of existing SOTA models. Furthermore, the fusion process is fully decoupled, ensuring that the independent training of each base model remains unaffected and preserving the inherent advantages of their original training paradigms.