A recursive optimal spectral estimate of end-to-end distortion in video communications

End-to-end distortion estimation is critical to effective error-resilient video coding. The recursive optimal per-pixel estimate (ROPE) is a known approach to compute up to second moments of decoder-reconstructed pixels, and thereby optimally estimate the distortion. ROPE accurately accounts for encoding/decoding operations that are recursive in the pixel domain, and their interaction with packet loss and decoder concealment. The premise of this work is that considerable gains could be recouped by a dual estimation technique that would perform its recursion in the transform domain. This opens the door to accurate distortion estimation in conjunction with estimation-theoretic source coding approaches that involve transform domain operations, including improved prediction in both single-layer and scalable video coding. We present a novel recursive optimal estimate that operates entirely in the transform domain, namely, the spectral coefficient-wise optimal recursive estimate (SCORE). The method overcomes intricacies due to motion compensation from “off-grid” blocks. We first demonstrate that its accuracy matches ROPE in the usual setting where ROPE is known to be optimal. Then we consider an enhanced encoding scenario involving spectral operations that cannot be accurately tracked by ROPE, but for which SCORE still maintains optimality and hence enables substantial end-to-end performance gains over a large range of packet loss rates.