Cross-task and time-aware adversarial attack framework for perception of autonomous driving

Despite the rapid advances in adversarial machine learning, state-of-the-art attack methods encounter practical limitations in the field of onboard perception that require real-time and multi-task processing. Conventional attacks typically target a specific perception task, such as object detection or segmentation, making it difficult to penetrate an entire multi-task perception module simultaneously. Although several cross-task transferable attacks have been proposed, these studies predominantly rely on model ensembling or iterative searching, both of which are often time-intensive and fail to meet the real-time processing requirements of autonomous driving platforms. To address these limitations, we propose Perception Streaming Attack (PSA), which is a non-iterative cross-task adversarial attack framework. We firstly propose Priori Perturbation Generator (PPG) to calculate a priori perturbation by leveraging the perturbation of previous frame as well as the motion information between the previous and current frames. Then, we propose Posterior Perturbation Updater (PPU) to refine the priori perturbation and obtain the final adversarial example for current frame. Comprehensive experimental evaluations on BDD100k and NuImages datasets demonstrate that the proposed PSA, compared with the state-of-the-art attacks, can effectively and efficiently attack across different tasks used in onboard perception. We also deploy our Perception Streaming Attack framework on a single-board computer (NVIDIA Jetson AGX Xavier) to validate the on-board performance. The experimental results show that the proposed PSA can successfully run at 12 Hz and effectively erase at least 76% objects that should be sensed.