GenRTC: Generative Real-Time Video Conferencing via Joint Adaptive Coding and Bandwidth Estimation

Real-time video broadcasting has become increasingly prevalent, enabling wide-scale distribution of video content to diverse audiences. However, network conditions often remain volatile, particularly in bandwidth-constrained environments such as subways or congested public areas, where both broadcasting and real-time communication (RTC) systems struggle to maintain optimal performance. In this paper, we propose GenRTC, a generative video conferencing system designed to mitigate bandwidth constraints by integrating joint scalable generative face coding and network-aware adaptation. This approach significantly improves the system’s robustness against network volatility, guaranteeing low-latency and high-fidelity video conferencing across a broad spectrum of bandwidth scenarios. GenRTC integrates conventional video codecs and generative face video encoding within a latency–quality joint adaptive codec controller, which dynamically selects the most appropriate encoding scheme to maximize perceptual quality under the latency constraints of real-time applications across diverse network conditions. In addition, GenRTC incorporates a bandwidth prediction module that delivers accurate and temporally stable low-bandwidth estimation. Experimental results demonstrate that GenRTC achieves a good latency-quality trade-off. GenRTC reduces frame timeouts from 60.4% to 5.4% on average under 200 Kbps. At a similar missrate, it improves PSNR by 7.2 dB. Specifically, at 20 Kbps, GenRTC achieves an 8% missrate and 32dB PSNR, while traditional RTC systems cannot operate at such ultra-low bandwidth. To facilitate a more intuitive comparison of perceptual quality, demo videos are provided at https://github.com/IreneLu12/GenRTC