Orchestration: tv-like mixing grammars applied to video-communication for social groups

Abstract

This paper reports research into video-mediated synchronous communication within social groups. The ultimate aim of the research is to create a more natural medium for interaction, aware of the context in which it operates, able to continuously adapt itself to the communication needs and optimise the way in which it captures and transmits aspects of the communication. This, is hypothesised, can be achieved by equipping each of the various locations involved in the communication with multiple controllable video cameras and microphones, and mixing the resulting content through techniques similar to those used in television?a process referred to as "orchestration". Through orchestration, each location should be able to receive the appropriate perspectives and levels of detail, thus generating experiences in which the spatial separation between participants is minimised. The paper defines the concept of orchestration and presents two major evaluation experiments that provide supporting evidence for the main assumption and motivate further research, in richer interaction contexts, into this concept.