Next generation of system architectures for tele-immersive environments

Abstract

Summary form only given. The tele-immersive 3D multi-camera room environments are starting to emerge in order to assist in distributed physical activities such as physical therapy, sport activities, and entertainment, and with them new challenging research questions. These environments need 3D multi-camera setups at the sending side and multi-display setups at the receiving side connected via appropriate network infrastructure. One important question is what is the next generation of system architectures that allow to build these environments in a flexible manner, with COTS components and for broader audience. In this paper we discuss challenges of system architectures as well as present possible solutions. We investigate the design space between the 3D multi-camera/multi-display tele-immersive edges and the general purpose computing and communication infrastructure available today. Especially, we analyze on our cross-layer control and streaming framework over general purpose delivery infrastructure, called TEEVE (tele-immersive environment for everybody), the difficulties and effectiveness of tele-immersive architectural constructs such as capturing, reconstructing and displaying 4D content, and coordination, synchronization and QoS-enabled delivery of tele-immersive 3D visual streams to remote room(s) over Internet 2. The first experiments of TEEVE and few other next generation architectures are encouraging, as we start to sustain communication of up to 12 3D streams with several frames per second (e.g., TEEVE can reach around 5 frames per second) over Internet 2, but a lot of work and challenges remain to be solved.