Immersive Gastronomic Experience with Distributed Reality

Abstract

We have developed an immersive gastronomic experience as a proof of concept of Distributed Reality, a type of Augmented Virtuality which combines a reality transmitted from a remote place, using 360◦ video, with a local reality, using video see-through. In order to reach fully immersive experience, local objects of interest such as hands and local food are segmented using red chrominance keying. Only those segmented objects are merged with the remote reality, enabling this way to increase self-presence and to allow user interaction. More concretely, the gastronomic experience consists of tasting small pieces of food, while being immersed in a remote place designed to pair with the food, thus creating an innovative concept with potential impact in hospitality and tourism industries. An evaluation performed with 66 users shows that it provides good levels of immersion, local interactivity, and general user satisfaction. The application achieves real time performance and has been developed for a smartphone mounted on a consumer headset, thus being easy to deploy and to reuse in other use cases.