Short-term memory and CenterTrack based vehicle-related multi-target tracking method

目的 车辆多目标跟踪是智能交通领域关键技术，其性能对车辆轨迹分析和异常行为鉴别有显著影响。然而，车辆多目标跟踪常受外部光照、道路环境因素影响，车辆远近尺度变化以及相互遮挡等干扰，导致远处车辆漏检或车辆身份切换（ID switch，IDs）问题。本文提出短时记忆与CenterTrack的车辆多目标跟踪，提升车辆多目标跟踪准确度（multiple object tracking accuracy，MOTA），改善算法的适应性。方法 利用小样本扩增增加远处小目标车辆训练样本数；通过增加的样本重新训练CenterTrack确定车辆位置及车辆在相邻帧之间的中心位移量；当待关联轨迹与检测目标匹配失败时通过轨迹运动信息预测将来的位置；利用短时记忆将待关联轨迹按丢失时间长短分级与待匹配检测关联以减少跟踪车辆IDs。结果 在交通监控车辆多目标跟踪数据集UA-DETRAC （University at Albany detection and tracking）构建的5个测试序列数据中，本文方法在维持CenterTrack优势的同时，对其表现不佳的场景获得近30%的提升，与YOLOv4-DeepSort（you only look once—simple online and realtime tracking with deep association metric）相比，4种场景均获得近10%的提升，效果显著。Sherbrooke数据集的测试结果，本文方法同样获得了性能提升。结论 本文扩增了远处小目标车辆训练样本，缓解了远处小目标与近处大目标存在的样本不均衡，提高了算法对远处小目标车辆的检测能力，同时短时记忆维持关联失败的轨迹运动信息并分级匹配检测目标，降低了算法对跟踪车辆的IDs，综合提高了MOTA。;Objective The task of multi-object tracking is often focused on estimating the number,location or other related properties of objects in the scene. Specifically,it is required to be estimated accurately and consistently over a period of time. Vehicle-related multi-target tracking can be as a key technique for such domain like intelligent transportation,and its performance has a significant impact on vehicle trajectory analysis and abnormal behavior identification to some extent. Vehicle-related multi-target tracking is also recognized as a key branch of multi-target tracking and a potential technique for autonomous driving and intelligent traffic surveillance systems. For vehicle-related multi-target tracking,temporal-based motion status of vehicles in traffic scenes can be automatically obtained,which is beneficial to analyze traffic conditions and implement decisions-making quickly for transportation administrations,as well as the automatic driving system. However,to resolve missed detection of distant vehicles or vehicle ID switch(IDs) problems,such factors are often to be dealt with in relevance to external illumination,road environment factors,changes in the scale of the vehicle near and far,and mutual occlusion. We develop an integrated short-term memory and CenterTrack ability to improve the vehicle multi-target tracking accuracy(multiple object tracking accuracy(MOTA)),and its adaptability of the algorithm can be optimized further. Method From the analysis of a large number of traffic monitoring video data,it can be seen the reasons for the unbalanced samples in the training samples. On the one hand,due to the fast speed of the captured vehicle target,the identified distant small target vehicle can be preserved temperorily,and it lacks of more consistent frames. On the other hand,the amount of apparent feature information is lower derived from small target vehicle itself,and the amount of neural networkextracted feature information is disappeared quickly many times. The relative number of distant small targets in the field of view is relatively small. After downsampling as a training sample,the feature quantity is disappeared very fast,resulting in an extensive reduction in the number of effective training samples,which is actually penetrated into the network. The small target vehicles cannot be detected after that. The small sample expansion method is proposed and adopted to increase the number of training samples,especially for small target vehicles in the distance. The CenterTrack is retrained with the increased samples,and the position of the vehicle can be determined where the vehicle is near or far in the image sequence and the center displacement between adjacent frames that is learnt from the CenterTrack. Due to it is assumed that a uniform linear motion is performed when the trajectory fails in matching the new detection in the short time,location of the trajectory in the current frame can be predicted through memorizing the short-term historical motion information of the trajectory when the new detection target-associated trajectory is failed. However,for the short-term memory method,it is challenged that there may be multiple trajectories competing for the same new detection target and it made a degradation of MOTA. To resolve trajectory competition and matching-derived performance degradation,we classify the trajectories further according to the length of the loss times with detection,and less loss,higher priority. The higher-level trajectories are preferred to match all new detection. This method can be used to preserve the integrity of the vehicle trajectory through reducing the missed far small vehicle and the false match between the trajectories and the detection,It can reduce the number of tracked vehicle IDs as well. Result To verify the effectiveness of the proposed algorithm in multiple scenarios,we extract data from two different datasets for testing. First,five sort of test sequences are extracted from such of multi-target tracking dataset like University at Albany detection and tracking(UA-DETRAC),the traffic surveillance scenery. The results demonstrate that our method proposed can maintain the advantages of CenterTrack and achieve nearly 30% improvement compared with CenterTrack in the scenes where CenterTrack performs not well. Compared to you only look once-simple online and realtime tracking with deep association metric(YOLOv4-DeepSort),it has achieved nearly 10% significant improvement in all four scenarios. The experimental results in Sherbrooke,as another traffic monitoring dataset,illustrate that short-term memory module and the remote small target vehicle expansion module can be used well compared to the original CenterTrack,and the proposed MOTA has a large performance improvement as well. Conclusion We analyze the challenges for detecting distant small target vehicles and vehicle tracking IDs in the vehicle multi-target tracking for the traffic monitoring scene. We resolve the imbalance in the number of samples between the distant small target vehicle and the nearby large target vehicle in the training sample in terms of expansion of the training samples of the small target vehicle in the distance as well,and the algorithm is improved for small target vehicle in the distance. At the same time,short-time trajectory memory module can be used to memorize the historical motion information of the failed trajectory to maintain the integrity of the trajectories when the losed detection appears again. Furthermore,the IDs can be reduced for tracking vehicles and the MOTA is improved in terms of the trajectories classification. Our CenterTrack-based algorithm proposed has been improving for such certain traffic video surveillance scene,and the experiments are carried out to validate the effectiveness of our algorithm proposed as well. Vehicle-related multi-target tracking technique has its potentials for developing the implementation for optimizing intelligent transportation and smart city strategies to a certain extent.