基于深度学习的视频超分辨率及其在视频会议中的应用

Yinyan Lin, Chaoyang Zou, Yingjie Feng, Mingwei Liang
{"title":"基于深度学习的视频超分辨率及其在视频会议中的应用","authors":"Yinyan Lin, Chaoyang Zou, Yingjie Feng, Mingwei Liang","doi":"10.1109/ICARM52023.2021.9536106","DOIUrl":null,"url":null,"abstract":"As a remote communication method, remote video scene is widely used in some occasions such as telecommuting, telelearning, and teleconsultation. However, the remote video scene requires a large network bandwidth for image information transmission, resulting in the insufficient real-time performance. In this paper, applying the super-resolution image restoration method to a remote video scene only requires 1/16 of the original network bandwidth, but a good visual effect of image reconstruction can be obtained. The Enhanced Information Multi-Distillation Block (EIMDB) and the Pixel-Level Information Distillation Block (PIDB) are proposed, which can improve the super-resolution effect of the image with a small amount of calculation. Finally, a novel real-time remote video communication super-resolution network (RVCSRN) is proposed which achieves a good balance between speed and restoration effect, and can effectively improve the visual effect of the remote video scene. In addition, since the pictures processed in the remote video scene are different from those processed by the general super-resolution method, these pictures have a compression loss, so a remote video scene dataset (RVSet) is created to obtain a better super-resolution effect.","PeriodicalId":367307,"journal":{"name":"2021 6th IEEE International Conference on Advanced Robotics and Mechatronics (ICARM)","volume":"5 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2021-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":"{\"title\":\"Deep Learning Based Video Super-Resolution and its Application in Video Conferences\",\"authors\":\"Yinyan Lin, Chaoyang Zou, Yingjie Feng, Mingwei Liang\",\"doi\":\"10.1109/ICARM52023.2021.9536106\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"As a remote communication method, remote video scene is widely used in some occasions such as telecommuting, telelearning, and teleconsultation. However, the remote video scene requires a large network bandwidth for image information transmission, resulting in the insufficient real-time performance. In this paper, applying the super-resolution image restoration method to a remote video scene only requires 1/16 of the original network bandwidth, but a good visual effect of image reconstruction can be obtained. The Enhanced Information Multi-Distillation Block (EIMDB) and the Pixel-Level Information Distillation Block (PIDB) are proposed, which can improve the super-resolution effect of the image with a small amount of calculation. Finally, a novel real-time remote video communication super-resolution network (RVCSRN) is proposed which achieves a good balance between speed and restoration effect, and can effectively improve the visual effect of the remote video scene. In addition, since the pictures processed in the remote video scene are different from those processed by the general super-resolution method, these pictures have a compression loss, so a remote video scene dataset (RVSet) is created to obtain a better super-resolution effect.\",\"PeriodicalId\":367307,\"journal\":{\"name\":\"2021 6th IEEE International Conference on Advanced Robotics and Mechatronics (ICARM)\",\"volume\":\"5 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2021-07-03\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"1\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2021 6th IEEE International Conference on Advanced Robotics and Mechatronics (ICARM)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/ICARM52023.2021.9536106\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2021 6th IEEE International Conference on Advanced Robotics and Mechatronics (ICARM)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ICARM52023.2021.9536106","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 1

摘要

远程视频场景作为一种远程通信方式,广泛应用于远程办公、远程学习、远程会诊等场合。然而,远程视频场景需要很大的网络带宽来传输图像信息,导致实时性不足。本文将超分辨率图像恢复方法应用于远程视频场景,所需网络带宽仅为原网络带宽的1/16,却能获得良好的图像重建视觉效果。提出了增强信息多蒸馏块(Enhanced Information Multi-Distillation Block, EIMDB)和像素级信息蒸馏块(Pixel-Level Information Distillation Block, PIDB),以较少的计算量提高图像的超分辨率效果。最后,提出了一种新型的实时远程视频通信超分辨率网络(RVCSRN),该网络在速度和恢复效果之间取得了很好的平衡,可以有效地改善远程视频场景的视觉效果。此外,由于远程视频场景处理的图像与一般超分辨率方法处理的图像不同,这些图像存在压缩损失,因此创建远程视频场景数据集(RVSet)以获得更好的超分辨率效果。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Deep Learning Based Video Super-Resolution and its Application in Video Conferences
As a remote communication method, remote video scene is widely used in some occasions such as telecommuting, telelearning, and teleconsultation. However, the remote video scene requires a large network bandwidth for image information transmission, resulting in the insufficient real-time performance. In this paper, applying the super-resolution image restoration method to a remote video scene only requires 1/16 of the original network bandwidth, but a good visual effect of image reconstruction can be obtained. The Enhanced Information Multi-Distillation Block (EIMDB) and the Pixel-Level Information Distillation Block (PIDB) are proposed, which can improve the super-resolution effect of the image with a small amount of calculation. Finally, a novel real-time remote video communication super-resolution network (RVCSRN) is proposed which achieves a good balance between speed and restoration effect, and can effectively improve the visual effect of the remote video scene. In addition, since the pictures processed in the remote video scene are different from those processed by the general super-resolution method, these pictures have a compression loss, so a remote video scene dataset (RVSet) is created to obtain a better super-resolution effect.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信