移动360度视频流的联合适配与增强

IF 9.2 2区 计算机科学 Q1 COMPUTER SCIENCE, INFORMATION SYSTEMS
Haotian Guo;Feng Wang;Wei Zhang;Yifei Zhu;Laizhong Cui;Jiangchuan Liu;Fei Richard Yu;Lei Zhang
{"title":"移动360度视频流的联合适配与增强","authors":"Haotian Guo;Feng Wang;Wei Zhang;Yifei Zhu;Laizhong Cui;Jiangchuan Liu;Fei Richard Yu;Lei Zhang","doi":"10.1109/TMC.2025.3555322","DOIUrl":null,"url":null,"abstract":"Tile-based streaming and super resolution (SR) are two representative technologies adopted to improve bandwidth efficiency of 360° video streaming. The former allows selective downloading of contents in the user viewport by splitting the video into multiple independently decodable tiles. The latter leverages client-side computation to enhance the received video to higher quality using advanced neural network models. In this work, we propose a Collaborated Streaming and Enhancement (CSE) adaptation framework for mobile 360° videos, which integrates super resolution with tile-based streaming to optimize the user experience with dynamic bandwidth and limited computing capability. To effectively enhance the tile-based video streaming through SR, we propose to adaptively group the tiles for quality enhancement adapting to the content similarity. We also identify and address several key design issues to integrate SR into tile-based video streaming including unified video quality assessment, computational complexity model for super resolution, and buffer analysis considering the interplay between transmission and enhancement. We further formulate the quality-of-experience (QoE) maximization problem for mobile 360° video streaming and propose a rate adaptation algorithm to make the best decisions for download and for enhancement based on the Lyapunov optimization theory. Extensive evaluation results validate the superiority of our proposed approach, which demonstrates stable performance with considerable QoE improvement, while enabling a trade-off between playback smoothness and video quality.","PeriodicalId":50389,"journal":{"name":"IEEE Transactions on Mobile Computing","volume":"24 8","pages":"7726-7741"},"PeriodicalIF":9.2000,"publicationDate":"2025-04-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Joint Adaptation for Mobile 360-Degree Video Streaming and Enhancement\",\"authors\":\"Haotian Guo;Feng Wang;Wei Zhang;Yifei Zhu;Laizhong Cui;Jiangchuan Liu;Fei Richard Yu;Lei Zhang\",\"doi\":\"10.1109/TMC.2025.3555322\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Tile-based streaming and super resolution (SR) are two representative technologies adopted to improve bandwidth efficiency of 360° video streaming. The former allows selective downloading of contents in the user viewport by splitting the video into multiple independently decodable tiles. The latter leverages client-side computation to enhance the received video to higher quality using advanced neural network models. In this work, we propose a Collaborated Streaming and Enhancement (CSE) adaptation framework for mobile 360° videos, which integrates super resolution with tile-based streaming to optimize the user experience with dynamic bandwidth and limited computing capability. To effectively enhance the tile-based video streaming through SR, we propose to adaptively group the tiles for quality enhancement adapting to the content similarity. We also identify and address several key design issues to integrate SR into tile-based video streaming including unified video quality assessment, computational complexity model for super resolution, and buffer analysis considering the interplay between transmission and enhancement. We further formulate the quality-of-experience (QoE) maximization problem for mobile 360° video streaming and propose a rate adaptation algorithm to make the best decisions for download and for enhancement based on the Lyapunov optimization theory. Extensive evaluation results validate the superiority of our proposed approach, which demonstrates stable performance with considerable QoE improvement, while enabling a trade-off between playback smoothness and video quality.\",\"PeriodicalId\":50389,\"journal\":{\"name\":\"IEEE Transactions on Mobile Computing\",\"volume\":\"24 8\",\"pages\":\"7726-7741\"},\"PeriodicalIF\":9.2000,\"publicationDate\":\"2025-04-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Transactions on Mobile Computing\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10947060/\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"COMPUTER SCIENCE, INFORMATION SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Mobile Computing","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10947060/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0

摘要

基于平铺流和超分辨率(SR)是提高360°视频流带宽效率的两种代表性技术。前者通过将视频分割成多个独立可解码的块,允许在用户视口中选择性地下载内容。后者利用客户端计算,利用先进的神经网络模型将接收到的视频提高到更高的质量。在这项工作中,我们提出了一种针对移动360°视频的协作流媒体和增强(CSE)适应框架,该框架将超分辨率与基于tile的流媒体相结合,以优化动态带宽和有限计算能力的用户体验。为了有效地通过SR增强基于tile的视频流,我们提出根据内容相似度对tile进行自适应分组以提高质量。我们还确定并解决了将SR集成到基于贴片的视频流中的几个关键设计问题,包括统一的视频质量评估、超分辨率的计算复杂性模型以及考虑传输和增强之间相互作用的缓冲区分析。我们进一步制定了移动360°视频流的体验质量(QoE)最大化问题,并提出了一种基于Lyapunov优化理论的速率自适应算法,以做出下载和增强的最佳决策。广泛的评估结果验证了我们提出的方法的优越性,它展示了稳定的性能和相当大的QoE改进,同时实现了播放平滑性和视频质量之间的权衡。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Joint Adaptation for Mobile 360-Degree Video Streaming and Enhancement
Tile-based streaming and super resolution (SR) are two representative technologies adopted to improve bandwidth efficiency of 360° video streaming. The former allows selective downloading of contents in the user viewport by splitting the video into multiple independently decodable tiles. The latter leverages client-side computation to enhance the received video to higher quality using advanced neural network models. In this work, we propose a Collaborated Streaming and Enhancement (CSE) adaptation framework for mobile 360° videos, which integrates super resolution with tile-based streaming to optimize the user experience with dynamic bandwidth and limited computing capability. To effectively enhance the tile-based video streaming through SR, we propose to adaptively group the tiles for quality enhancement adapting to the content similarity. We also identify and address several key design issues to integrate SR into tile-based video streaming including unified video quality assessment, computational complexity model for super resolution, and buffer analysis considering the interplay between transmission and enhancement. We further formulate the quality-of-experience (QoE) maximization problem for mobile 360° video streaming and propose a rate adaptation algorithm to make the best decisions for download and for enhancement based on the Lyapunov optimization theory. Extensive evaluation results validate the superiority of our proposed approach, which demonstrates stable performance with considerable QoE improvement, while enabling a trade-off between playback smoothness and video quality.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
IEEE Transactions on Mobile Computing
IEEE Transactions on Mobile Computing 工程技术-电信学
CiteScore
12.90
自引率
2.50%
发文量
403
审稿时长
6.6 months
期刊介绍: IEEE Transactions on Mobile Computing addresses key technical issues related to various aspects of mobile computing. This includes (a) architectures, (b) support services, (c) algorithm/protocol design and analysis, (d) mobile environments, (e) mobile communication systems, (f) applications, and (g) emerging technologies. Topics of interest span a wide range, covering aspects like mobile networks and hosts, mobility management, multimedia, operating system support, power management, online and mobile environments, security, scalability, reliability, and emerging technologies such as wearable computers, body area networks, and wireless sensor networks. The journal serves as a comprehensive platform for advancements in mobile computing research.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信