Learning based compression of surface light fields for real-time rendering of global illumination scenes

We present an algorithm for compression and real-time rendering of surface light fields (SLF) encoding the visual appearance of objects in static scenes with high frequency variations. We apply a non-local clustering in order to exploit spatial coherence in the SLF data. To efficiently encode the data in each cluster, we introduce a learning based approach, Clustered Exemplar Orthogonal Bases (CEOB), which trains a compact dictionary of orthogonal basis pairs, enabling efficient sparse projection of the SLF data. In addition, we discuss the application of the traditional Clustered Principal Component Analysis (CPCA) on SLF data, and show that in most cases, CEOB outperforms CPCA, K-SVD and spherical harmonics in terms of memory footprint, rendering performance and reconstruction quality. Our method enables efficient reconstruction and real-time rendering of scenes with complex materials and light sources, not possible to render in real-time using previous methods.