Using Deep Reinforcement Learning (DRL) to Optimize Quality in 360-Degree Video Tile Management

360-degree videos inherently require significant storage space because each segment consists of many tiles, each of which is further transcoded and stored in multiple versions. It is thus impractical to store all transcoded versions, which makes it essential to make effective use of limited storage space. However, the inefficiency of existing heuristic-based management schemes arises from the challenge of incorporating various factors, such as variable bandwidth requirements influenced by network conditions, tile access distribution, and video quality dependent on content. To address this, we propose a new storage space management scheme, which combines the dueling deep Q-network (DQN) algorithm based on the field-of-view (FoV) distribution and the greedy algorithm that considers the overall video popularity. We first model an environment in which the agent can determine the versions for each tile to achieve the best video quality under various storage limit conditions. The dueling DQN environment comprises 1) an action space determining version combinations for each tile within specified storage limits, 2) an observation space enabling the agent to learn variable bandwidths and tile access distributions, and 3) a reward model deriving the expected video quality for different actions. Building upon the dueling DQN model correlating storage limits with expected video quality, we present a greedy algorithm that selects versions among multiple videos within storage limits for the purpose of maximizing popularity-weighted video quality. Extensive simulations evaluated the proposed scheme under various storage limits, bandwidth changes, and FoV distributions, demonstrating an improvement in overall popularity-weighted video quality ranging from 0.49% to 37.77% (with an average improvement of 13.96%) compared to existing benchmark schemes.