Quality assessment of video with film grain

Abstract

Film grain noise originally arises from small metallic silver particles on processed photographic celluloid. Although modern digital video acquisition systems are capable of largely reducing noise, sometimes to nearly invisible levels, the look of cinematic film grain has not gone away. Instead, content creators often purposely introduce simulated film grain in post-production to emulate dust in the environment, enrich texture details, and develop a certain visual tone. Despite the artistic benefits, film grain has posed significant challenges to video delivery systems. Compressing and transmitting videos containing film grain noise is extremely costly due to the large number of bits required to encode the noisy pixels of much higher entropy than the typical visual content of the scene. Heavy compression may remove film grain, but meanwhile, remove meaningful texture content in the visual scene or deteriorate the artistic effect of the creator's intent. It also casts major challenges to quality control of video delivery systems, for which film grain-susceptible fidelity measures are highly desirable for measurement and optimization purposes. Here after describing the characteristics of film grain and its impact to video quality, we present a novel framework that unifies natural video quality assessment and creative intent friendly video quality assessment. We also demonstrate an instantiation of the framework in the context of film-grained content in terms of predicting the perception of different groups of subjects.