Haotian Guo;Feng Wang;Wei Zhang;Yifei Zhu;Laizhong Cui;Jiangchuan Liu;Fei Richard Yu;Lei Zhang
{"title":"移动360度视频流的联合适配与增强","authors":"Haotian Guo;Feng Wang;Wei Zhang;Yifei Zhu;Laizhong Cui;Jiangchuan Liu;Fei Richard Yu;Lei Zhang","doi":"10.1109/TMC.2025.3555322","DOIUrl":null,"url":null,"abstract":"Tile-based streaming and super resolution (SR) are two representative technologies adopted to improve bandwidth efficiency of 360° video streaming. The former allows selective downloading of contents in the user viewport by splitting the video into multiple independently decodable tiles. The latter leverages client-side computation to enhance the received video to higher quality using advanced neural network models. In this work, we propose a Collaborated Streaming and Enhancement (CSE) adaptation framework for mobile 360° videos, which integrates super resolution with tile-based streaming to optimize the user experience with dynamic bandwidth and limited computing capability. To effectively enhance the tile-based video streaming through SR, we propose to adaptively group the tiles for quality enhancement adapting to the content similarity. We also identify and address several key design issues to integrate SR into tile-based video streaming including unified video quality assessment, computational complexity model for super resolution, and buffer analysis considering the interplay between transmission and enhancement. We further formulate the quality-of-experience (QoE) maximization problem for mobile 360° video streaming and propose a rate adaptation algorithm to make the best decisions for download and for enhancement based on the Lyapunov optimization theory. Extensive evaluation results validate the superiority of our proposed approach, which demonstrates stable performance with considerable QoE improvement, while enabling a trade-off between playback smoothness and video quality.","PeriodicalId":50389,"journal":{"name":"IEEE Transactions on Mobile Computing","volume":"24 8","pages":"7726-7741"},"PeriodicalIF":9.2000,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Joint Adaptation for Mobile 360-Degree Video Streaming and Enhancement\",\"authors\":\"Haotian Guo;Feng Wang;Wei Zhang;Yifei Zhu;Laizhong Cui;Jiangchuan Liu;Fei Richard Yu;Lei Zhang\",\"doi\":\"10.1109/TMC.2025.3555322\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Tile-based streaming and super resolution (SR) are two representative technologies adopted to improve bandwidth efficiency of 360° video streaming. The former allows selective downloading of contents in the user viewport by splitting the video into multiple independently decodable tiles. The latter leverages client-side computation to enhance the received video to higher quality using advanced neural network models. In this work, we propose a Collaborated Streaming and Enhancement (CSE) adaptation framework for mobile 360° videos, which integrates super resolution with tile-based streaming to optimize the user experience with dynamic bandwidth and limited computing capability. To effectively enhance the tile-based video streaming through SR, we propose to adaptively group the tiles for quality enhancement adapting to the content similarity. We also identify and address several key design issues to integrate SR into tile-based video streaming including unified video quality assessment, computational complexity model for super resolution, and buffer analysis considering the interplay between transmission and enhancement. We further formulate the quality-of-experience (QoE) maximization problem for mobile 360° video streaming and propose a rate adaptation algorithm to make the best decisions for download and for enhancement based on the Lyapunov optimization theory. Extensive evaluation results validate the superiority of our proposed approach, which demonstrates stable performance with considerable QoE improvement, while enabling a trade-off between playback smoothness and video quality.\",\"PeriodicalId\":50389,\"journal\":{\"name\":\"IEEE Transactions on Mobile Computing\",\"volume\":\"24 8\",\"pages\":\"7726-7741\"},\"PeriodicalIF\":9.2000,\"publicationDate\":\"2025-04-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Transactions on Mobile Computing\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10947060/\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Mobile Computing","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10947060/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
Joint Adaptation for Mobile 360-Degree Video Streaming and Enhancement
Tile-based streaming and super resolution (SR) are two representative technologies adopted to improve bandwidth efficiency of 360° video streaming. The former allows selective downloading of contents in the user viewport by splitting the video into multiple independently decodable tiles. The latter leverages client-side computation to enhance the received video to higher quality using advanced neural network models. In this work, we propose a Collaborated Streaming and Enhancement (CSE) adaptation framework for mobile 360° videos, which integrates super resolution with tile-based streaming to optimize the user experience with dynamic bandwidth and limited computing capability. To effectively enhance the tile-based video streaming through SR, we propose to adaptively group the tiles for quality enhancement adapting to the content similarity. We also identify and address several key design issues to integrate SR into tile-based video streaming including unified video quality assessment, computational complexity model for super resolution, and buffer analysis considering the interplay between transmission and enhancement. We further formulate the quality-of-experience (QoE) maximization problem for mobile 360° video streaming and propose a rate adaptation algorithm to make the best decisions for download and for enhancement based on the Lyapunov optimization theory. Extensive evaluation results validate the superiority of our proposed approach, which demonstrates stable performance with considerable QoE improvement, while enabling a trade-off between playback smoothness and video quality.
期刊介绍:
IEEE Transactions on Mobile Computing addresses key technical issues related to various aspects of mobile computing. This includes (a) architectures, (b) support services, (c) algorithm/protocol design and analysis, (d) mobile environments, (e) mobile communication systems, (f) applications, and (g) emerging technologies. Topics of interest span a wide range, covering aspects like mobile networks and hosts, mobility management, multimedia, operating system support, power management, online and mobile environments, security, scalability, reliability, and emerging technologies such as wearable computers, body area networks, and wireless sensor networks. The journal serves as a comprehensive platform for advancements in mobile computing research.