{"title":"基于深度学习的视频超分辨率及其在视频会议中的应用","authors":"Yinyan Lin, Chaoyang Zou, Yingjie Feng, Mingwei Liang","doi":"10.1109/ICARM52023.2021.9536106","DOIUrl":null,"url":null,"abstract":"As a remote communication method, remote video scene is widely used in some occasions such as telecommuting, telelearning, and teleconsultation. However, the remote video scene requires a large network bandwidth for image information transmission, resulting in the insufficient real-time performance. In this paper, applying the super-resolution image restoration method to a remote video scene only requires 1/16 of the original network bandwidth, but a good visual effect of image reconstruction can be obtained. The Enhanced Information Multi-Distillation Block (EIMDB) and the Pixel-Level Information Distillation Block (PIDB) are proposed, which can improve the super-resolution effect of the image with a small amount of calculation. Finally, a novel real-time remote video communication super-resolution network (RVCSRN) is proposed which achieves a good balance between speed and restoration effect, and can effectively improve the visual effect of the remote video scene. In addition, since the pictures processed in the remote video scene are different from those processed by the general super-resolution method, these pictures have a compression loss, so a remote video scene dataset (RVSet) is created to obtain a better super-resolution effect.","PeriodicalId":367307,"journal":{"name":"2021 6th IEEE International Conference on Advanced Robotics and Mechatronics (ICARM)","volume":"5 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2021-07-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"1","resultStr":"{\"title\":\"Deep Learning Based Video Super-Resolution and its Application in Video Conferences\",\"authors\":\"Yinyan Lin, Chaoyang Zou, Yingjie Feng, Mingwei Liang\",\"doi\":\"10.1109/ICARM52023.2021.9536106\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"As a remote communication method, remote video scene is widely used in some occasions such as telecommuting, telelearning, and teleconsultation. However, the remote video scene requires a large network bandwidth for image information transmission, resulting in the insufficient real-time performance. In this paper, applying the super-resolution image restoration method to a remote video scene only requires 1/16 of the original network bandwidth, but a good visual effect of image reconstruction can be obtained. The Enhanced Information Multi-Distillation Block (EIMDB) and the Pixel-Level Information Distillation Block (PIDB) are proposed, which can improve the super-resolution effect of the image with a small amount of calculation. Finally, a novel real-time remote video communication super-resolution network (RVCSRN) is proposed which achieves a good balance between speed and restoration effect, and can effectively improve the visual effect of the remote video scene. In addition, since the pictures processed in the remote video scene are different from those processed by the general super-resolution method, these pictures have a compression loss, so a remote video scene dataset (RVSet) is created to obtain a better super-resolution effect.\",\"PeriodicalId\":367307,\"journal\":{\"name\":\"2021 6th IEEE International Conference on Advanced Robotics and Mechatronics (ICARM)\",\"volume\":\"5 1\",\"pages\":\"0\"},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2021-07-03\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"1\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"2021 6th IEEE International Conference on Advanced Robotics and Mechatronics (ICARM)\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1109/ICARM52023.2021.9536106\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"2021 6th IEEE International Conference on Advanced Robotics and Mechatronics (ICARM)","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1109/ICARM52023.2021.9536106","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 1
摘要
远程视频场景作为一种远程通信方式,广泛应用于远程办公、远程学习、远程会诊等场合。然而,远程视频场景需要很大的网络带宽来传输图像信息,导致实时性不足。本文将超分辨率图像恢复方法应用于远程视频场景,所需网络带宽仅为原网络带宽的1/16,却能获得良好的图像重建视觉效果。提出了增强信息多蒸馏块(Enhanced Information Multi-Distillation Block, EIMDB)和像素级信息蒸馏块(Pixel-Level Information Distillation Block, PIDB),以较少的计算量提高图像的超分辨率效果。最后,提出了一种新型的实时远程视频通信超分辨率网络(RVCSRN),该网络在速度和恢复效果之间取得了很好的平衡,可以有效地改善远程视频场景的视觉效果。此外,由于远程视频场景处理的图像与一般超分辨率方法处理的图像不同,这些图像存在压缩损失,因此创建远程视频场景数据集(RVSet)以获得更好的超分辨率效果。
Deep Learning Based Video Super-Resolution and its Application in Video Conferences
As a remote communication method, remote video scene is widely used in some occasions such as telecommuting, telelearning, and teleconsultation. However, the remote video scene requires a large network bandwidth for image information transmission, resulting in the insufficient real-time performance. In this paper, applying the super-resolution image restoration method to a remote video scene only requires 1/16 of the original network bandwidth, but a good visual effect of image reconstruction can be obtained. The Enhanced Information Multi-Distillation Block (EIMDB) and the Pixel-Level Information Distillation Block (PIDB) are proposed, which can improve the super-resolution effect of the image with a small amount of calculation. Finally, a novel real-time remote video communication super-resolution network (RVCSRN) is proposed which achieves a good balance between speed and restoration effect, and can effectively improve the visual effect of the remote video scene. In addition, since the pictures processed in the remote video scene are different from those processed by the general super-resolution method, these pictures have a compression loss, so a remote video scene dataset (RVSet) is created to obtain a better super-resolution effect.