Real-time image-based 3D avatar for immersive game

Abstract

We developed an action game which is based on the interaction between real-time 3D avatar of the player and the virtual game characters. The 3D avatar is reconstructed from multi-view images of the player with image-based modeling and rendering techniques. The 3D avatar can be dynamically reconstructed and rendered in real time by using the Hardware-accelerated Visual Hull(HAVH) method. The visual appearances and physical activities of the player are projected onto the avatar. The player can see themselves in the virtual 3D space and interact with the virtual objects through the bodily movements of the avatar in the gaming world. The combination of movement-based interaction and realistic visual appearance makes games more realistic and immersive.