DAVE: Dynamic Adaptive Video Encoding for Real-time Video Streaming Applications

Siqi Huang, Jiang Xie
{"title":"DAVE: Dynamic Adaptive Video Encoding for Real-time Video Streaming Applications","authors":"Siqi Huang, Jiang Xie","doi":"10.1109/SECON52354.2021.9491588","DOIUrl":null,"url":null,"abstract":"Real-time video streaming applications have become tremendously popular in recent years, such as remote control and video conferencing applications. A key characteristic that differentiates these applications from traditional live streaming applications is that these applications have a very low-latency requirement for interactivity. The stricter low-latency requirement brings many challenges: the video has to be encoded in a real-time manner; the substantial resources on the server or cloud cannot be utilized for encoding; and the adaptation strategies in live streaming applications are not adequate for real-time video streaming, such as adaptive bitrate selection (ABR). In addition, the video perceptual quality of current real-time video streaming systems is usually sacrificed to meet the very low-latency requirement.To address these challenges, in this paper, a new real-time video streaming protocol, DAVE (Dynamic Adaptive Video Encoding for real-time video streaming applications), is proposed. In the proposed real-time video streaming system, captured video frames are encoded with different configurations. Since the video encoding configuration determines the video data size, quality, and encoding time, we first conduct an experimental study on the impact of each configuration parameter. Based on our experimental findings, we then propose a super resolution based video encoding configuration selection algorithm which does not use a fixed strategy to determine the encoding configurations as in existing real-time video streaming systems but uses a reinforcement learning based model to learn the optimal video encoding configuration that includes the configuration of both regular video encoding parameters and the up-scale of super resolution models. As a result, DAVE can optimize the performance of real-time video streaming systems based on user Quality of Experience (QoE) metrics. To the best of our knowledge, this is the first work that incorporates super resolution and reinforcement learning in the protocol design for real-time video streaming systems. Extensive evaluations show that DAVE can substantially improve the video perceptual quality by 15% and can also reduce the end-to-end latency by 20%, as compared with existing systems1.","PeriodicalId":120945,"journal":{"name":"2021 18th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON)","volume":"3 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2021-07-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"4","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"2021 18th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/SECON52354.2021.9491588","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 4

Abstract

Real-time video streaming applications have become tremendously popular in recent years, such as remote control and video conferencing applications. A key characteristic that differentiates these applications from traditional live streaming applications is that these applications have a very low-latency requirement for interactivity. The stricter low-latency requirement brings many challenges: the video has to be encoded in a real-time manner; the substantial resources on the server or cloud cannot be utilized for encoding; and the adaptation strategies in live streaming applications are not adequate for real-time video streaming, such as adaptive bitrate selection (ABR). In addition, the video perceptual quality of current real-time video streaming systems is usually sacrificed to meet the very low-latency requirement.To address these challenges, in this paper, a new real-time video streaming protocol, DAVE (Dynamic Adaptive Video Encoding for real-time video streaming applications), is proposed. In the proposed real-time video streaming system, captured video frames are encoded with different configurations. Since the video encoding configuration determines the video data size, quality, and encoding time, we first conduct an experimental study on the impact of each configuration parameter. Based on our experimental findings, we then propose a super resolution based video encoding configuration selection algorithm which does not use a fixed strategy to determine the encoding configurations as in existing real-time video streaming systems but uses a reinforcement learning based model to learn the optimal video encoding configuration that includes the configuration of both regular video encoding parameters and the up-scale of super resolution models. As a result, DAVE can optimize the performance of real-time video streaming systems based on user Quality of Experience (QoE) metrics. To the best of our knowledge, this is the first work that incorporates super resolution and reinforcement learning in the protocol design for real-time video streaming systems. Extensive evaluations show that DAVE can substantially improve the video perceptual quality by 15% and can also reduce the end-to-end latency by 20%, as compared with existing systems1.
戴夫:实时视频流应用的动态自适应视频编码
近年来,实时视频流应用变得非常流行,例如远程控制和视频会议应用。将这些应用程序与传统的直播流应用程序区分开来的一个关键特征是,这些应用程序对交互性的延迟要求非常低。更严格的低延迟要求带来了许多挑战:视频必须实时编码;服务器或云上的大量资源无法用于编码;直播应用中的自适应策略如自适应比特率选择(ABR)不足以适应实时视频流。此外,当前实时视频流系统通常会牺牲视频感知质量来满足非常低延迟的要求。为了解决这些问题,本文提出了一种新的实时视频流协议DAVE(动态自适应视频编码,用于实时视频流应用)。在所提出的实时视频流系统中,捕获的视频帧以不同的配置进行编码。由于视频编码配置决定了视频数据的大小、质量和编码时间,我们首先对每个配置参数的影响进行了实验研究。基于我们的实验结果,我们提出了一种基于超分辨率的视频编码配置选择算法,该算法不像现有的实时视频流系统那样使用固定的策略来确定编码配置,而是使用基于强化学习的模型来学习最优视频编码配置,包括常规视频编码参数的配置和超分辨率模型的升级。因此,DAVE可以根据用户体验质量(QoE)指标优化实时视频流系统的性能。据我们所知,这是第一个在实时视频流系统的协议设计中结合超分辨率和强化学习的工作。广泛的评估表明,与现有系统相比,DAVE可以将视频感知质量大幅提高15%,还可以将端到端延迟降低20% 1。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信