Heterogeneous Siamese Tracking System Based on PYNQ Framework

Deep neural network models have been gradually applied to the field of visual tracking due to their excellent feature expression capabilities. However, the model is large, which is expressed in the calculation of model parameters. As a result, the implementation platforms of visual tracking algorithm are usually limited by computing power, power consumption, portability, etc. In this paper, we propose a Siamese network tracking scheme based on PYNQ framework, which is deployed on ZYNQ platform. By optimizing the calculation process, Siamese Network accelerated IP core and Region Proposal Network accelerated IP core are designed. The double buffer structure is adopted to effectively call different feature map calculations and reduce off-chip memory access. Python is used to call the accelerated IP core at the top level as the hardware coprocessor to realize the data interaction from the bottom level to the top level and update the system running results asynchronously in Jupyter notebook. We achieve an average 36.7FPS on Xilinx ZCU104 platform, which illustrates that our method has the important practical benefit of allowing lightweight architectures to achieve good performance at high framerates.