Enhanced Feature Pyramid Networks by Feature Aggregation Module and Refinement Module

Feature pyramids executing refinements on the raw feature maps produced by the backbone (e.g., ResNet, VGG) are universally employed in object detection tasks (e.g., Faster R-CNN, Mask R-CNN, YOLO, SSD, RetinaNet) to mitigate scale variation problem. Although these object detections with feature pyramids accomplish a boost in accuracy without compromising speed, they have some limitations since that they only naturally design the feature pyramid with consecutive scales, the pyramidal architecture of the backbone, which are initially constructed for the classification task. This problem leads to the feature imbalance between high-level features and low-level features in object detection. In this work, the proposed method introduces Feature Aggregation Module (FAM) and Refinement Module (RM) to obtain more powerful feature pyramids for predicting objects of different scales. First, the multi-level feature maps (i.e., multiple layers) extracted by the backbone network are aggregated as the basic feature. Second, the basic feature is enhanced by a refinement module exploiting long-range dependency. Three, to construct a feature pyramid for object detection, the proposed FAM is used by converting the basic feature (after utilizing a refinement module) into multi-level features. Finally, refined multi-level features and raw features generated by the backbone could be enhanced through shortcut connections to capture more representative. To perform the efficiency, the proposed method integrates the FAM and the RAM into the architecture of Faster R-CNN called EFPN Faster R-CNN. Especially on the MS-COCO dataset, EFPN Faster R-CNN achieves 2.2 points higher Average Precision (AP) than FPN Faster R-CNN without bells and whistles.