Enabling universal media experiences through semantic adaptation in the creative drama productionworkflow

Abstract

Viewers today have access to broadcasted audiovisual media via a multitude of platforms: from high definition movie theaters to smart cellular phones on the road, and online using the web. In order to provide these audiences with a proper universal media experience, each class of media consumption devices should receive a product not only adapted to its capabilities and usage environment, but also in a manner as to aid a better conveyance of the cinematic narrative, i.e., with proper subject framing, shots and sensible editing decisions. This paper presents a semantic adaptation system that incorporates the media adaptation process from the start of the production process. This allows producers, directors and other creative staff to operate in a multi-platform-aware workflow that compiles products with optimal adaptations for all intended distribution platforms. Production metadata obtained from various steps in the production process is used to guide the scaling, cropping and reframing operations applied to audiovisual material. In particular, this paper demonstrates how our system can be employed to adapt a single drama product into different versions, from high definition wide-shot cinematography to tightly shot cellular phone video.