Switched error concealment and robust coding decisions in scalable video coding

Abstract

This work introduces two complementary techniques to improve the packet loss resilience of scalable video coding systems. First, a "switch per-pixel" error concealment (SPEC) scheme is proposed, which allows the decoder to exploit information from both the current base layer and previous enhancement-layer frame for the reconstruction of missing enhancement-layer blocks. Based on the packet loss history and the quantized base-layer data, the algorithm switches per pixel between the two information sources. SPEC is shown to consistently outperform standard concealment methods. The second main contribution is concerned with encoder decision optimization. Enhancement layer prediction modes are selected so as to minimize the overall decoder reconstruction distortion, which is due to quantization, packet loss and error propagation. The distortion computation uses a recursive optimal per-pixel estimate (ROPE) to accurately account for the effects of error concealment as well as spatial and temporal error propagation. Simulation results show that ROPE-based mode selection substantially outperforms conventional prediction mode selection schemes. Finally, the combination of SPEC at the decoder and ROPE-based mode selection at the encoder is shown to achieve significant additional performance gains.