TransBranch: A transformer branch architecture for fine-grained recognition

In this paper, we present a novel architecture noted asTransBranch for the challenging fine-grained visual categorization tasks. Distinguished from traditional models based on cross-layer feature fusion, the proposed architecture enhances classification accuracy by strategically integrating image features in a delicate way: features with different levels are generated in parallel, then assembled via a designed content-aware cross-level fusion mechanism, by which the multi-level features compensate each other and highlight the discriminative cues for visually similar subcategories. To this end, we devise an adaptive weighting mechanism, which dynamically adjusts the weights of features at different levels based on the difficulty of distinguishing subcategories and the semantics of image contents. This mechanism identifies discriminative features from cluttered backgrounds and guides the model to focus on the rare categories, improving the recognition while alleviating the long-tail distribution issue. Furthermore, a multi-scale patch embedding strategy has been devised to ensure the completeness of semantic image contents during feature learning. Experimental results show the proposed model outperforms current transformer-based architectures across benchmarked datasets for fine-grained visual categorization, especially in distinguishing categories with extremely similar features.