Realtime Video Latency Reduction for Autonomous Vehicle Teleoperation Using RTMP Over UDP Protocols

Real-time video streaming with low latency is essential for autonomous vehicles’ teleoperation. Studies show that the latency for self-driving teleoperation should not exceed 50 milliseconds. Latency could be caused by hardware, software, or network factors. Here, we focus on latency reduction due to network factors. We applied two data protocols: user datagram protocols (UDP) and real-time messaging protocol (RTMP), and afterward tuned the encoder along with data compression. The glass-to-glass method was applied to measure the network performance and video latency. The measurement of video latency achieves 300 milliseconds, which implements a direct connection (without a broadcaster) with the UDP data protocol. While this may still exceed the requirements, the study could be seen as a preliminary effort to deal with data latency for teleoperation driving.