A Two-Stage Deep Reinforcement Learning Framework for MEC-Enabled Adaptive 360-Degree Video Streaming

Abstract

The emerging multi-access edge computing (MEC) technology effectively enhances the wireless streaming performance of 360-degree videos. By connecting a user's head-mounted device (HMD) to a smart MEC platform, the edge server (ES) can efficiently perform adaptive tile-based video streaming to improve the user's viewing experience. Under constrained wireless channel capacity, the ES can predict the user's field of view (FoV) and transmit to the HMD high-resolution video tiles only within the predicted FoV. In practice, the video streaming performance is challenged by the random FoV prediction error and wireless channel fading effects. For this, we propose in this paper a novel two-stage adaptive 360-degree video streaming scheme that maximizes the user's quality of experience (QoE) to attain stable and high-resolution video playback. Specifically, we divide the video file into groups of pictures (GOPs) of fixed playback interval, where each GOP consists of a number of video frames. At the beginning of each GOP (i.e., the inter-GOP stage), the ES predicts the FoV of the next GOP and allocates an encoding bitrate for transmitting (precaching) the video tiles within the predicted FoV. Then, during the real-time video playback of the current GOP (i.e., the intra-GOP stage), the ES observes the user's true FoV of each frame and transmits the missing tiles to compensate for the FoV prediction errors. To maximize the user's QoE under random variations of FoV and wireless channel, we propose a double-agent deep reinforcement learning framework, where the two agents operate in different time scales to decide the bitrates of inter- and intra-GOP stages, respectively. Experiments based on real-world measurements show that the proposed scheme can effectively mitigate FoV prediction errors and maintain stable QoE performance under different scenarios, achieving over 22.1% higher QoE than some representative benchmark methods.