An Interactive and Immersive Remote Education Platform based on Commodity Devices

Abstract

Virtual reality (VR) holds a great potential to provide interactive and immersive learning experiences for students in remote education by using existing mobile devices, which is extremely meaningful during the current pandemic. In such a VR application, satisfactory user experience requires: 1) high-resolution panoramic image rendering; 2) high frame rate; 3) synchronization among users. This requires that either mobile devices perform fast image rendering or today’s wireless network can support multi-Gbps traffic with extremely low delay, neither of which is the case in current practice. In this demo, we develop a platform for interactive and immersive remote education based on commodity devices, where a server performs rendering to ensure that the rendered images have high-resolution (2560×1440 pixels) and are displayed at a high frame rate (60 frames per second) on the client-side. We further leverage motion prediction to overcome the diverse round-trip time (RTT) between a server and users and ensure synchronization among users (average 9.2 ms frame latency difference among users), which improves at least 60% and 20% compared to the existing local-rendering and server-rendering methods, respectively.