Ranking-based adaptive query generation for DETRs in crowded pedestrian detection

Abstract

Variants of DEtection TRansformer (DETRs) have shown promising performance in crowded pedestrian detection. However, we observe that DETRs are sensitive to the hyper-parameter (the number of queries). Adjusting this hyper-parameter is crucial for achieving competitive performance across different crowded pedestrian datasets. Existing query generation methods are limited to generate a fixed number of queries based on this hyper-parameter, which often leads to missed detections and incorrect detections due to the varied number and density of pedestrians in crowded scenes. To address this challenge, we propose an adaptive query generation method called Ranking-based Adaptive Query Generation (RAQG). RAQG comprises three components: a ranking prediction head, a query supplementer, and Soft Gradient L1 Loss (SGL1). Specifically, we leverage the ranking of the lowest confidence score positive training sample to generate queries adaptively. The ranking prediction head predicts this ranking, which guides our query generation. Additionally, to refine the query generation process, we introduce a query supplementer that adjusts the number of queries based on the predicted ranking. Furthermore, we introduce SGL1, a novel loss function for training the ranking prediction head over a wide regression range. Our method is designed to be lightweight and universal, suitable for integration into any DETRs framework for crowded pedestrian detection. Experimental results on Crowdhuman and Citypersons datasets demonstrate that our RAQG method can generate queries adaptively and achieves competitive results. Notably, our approach achieves a state-of-the-art 39.4% MR on Crowdhuman.