Garage: GPU particle-based AR content for futuristic experiences

Abstract

In futuristic AR, we believe that both real and virtual objects surrounding the player should be interactive and controllable. With this in mind, we introduce ”Garage”, which is an AR project based on our particle-based system capable of outputting a variety of interactive content. In our system, everything in the player’s surrounding environment is presented as particles and adequately simulated in real-time on a desktop PC equipped with a discrete GPU and rendered to in-house HMD, while their color and depth information is captured and sent by RGB camera and LiDAR sensor. LiDAR data is converted to particles using an algorithm that calculates world coordinated positions from depth values. Our system content and provided experiences have time flexibility and spatial control features that cannot be achieved in conventional polygon mesh-based systems. Additionally, the AR information output is presented in an appealing visual style that conserves GPU resources. Because of these features, Garage has a profound impact on AR immersion, and can augments our perception of the world in a limitless and creative manner. To demonstrate the abilities of our system and build new interactive AR experiences, in this project, we designed a game and created several demos that can give players interesting glimpses into the future world of augmented reality.