基于差分信息的双目融合立体图像质量评价跨域特征交互网络

IF 4.8 1区 计算机科学 Q2 ENGINEERING, ELECTRICAL & ELECTRONIC
Yongli Chang;Guanghui Yue;Bo Zhao
{"title":"基于差分信息的双目融合立体图像质量评价跨域特征交互网络","authors":"Yongli Chang;Guanghui Yue;Bo Zhao","doi":"10.1109/TBC.2025.3525976","DOIUrl":null,"url":null,"abstract":"Recently, convolutional neural network (CNN) based stereo image quality assessment (SIQA) has been extensively researched, achieving impressive performance. However, most SIQA methods tend to only mine features from distorted stereo image, neglecting the exploitation of valuable features present in other image domains. Moreover, some simple fusion strategies like addition and concatenation for binocular fusion further limit the network’s prediction performance. Therefore, we design a cross-domain feature interaction network (CDFINet) for SIQA in this paper, which considers the complementarity between different domain features and realizes binocular fusion between the left and right monocular features based on difference information. Specifically, to boost the prediction ability, we design a dual-branch network with image and gradient feature extraction branches, extracting hierarchical features from both domains. Moreover, to explore more proper binocular information, we propose a difference information guidance based binocular fusion (DIGBF) module to achieve the binocular fusion. Furthermore, to better achieve information compensation between image and gradient domain, binocular features obtained from image domain and gradient domain are fused in the proposed cross-domain feature fusion (CDFF) module. In addition, considering the feedback mechanism of the visual cortex, higher-level features are backpropagated to lower-level regions, and the proposed cross-layer feature interaction (CLFI) module realizes the guidance of higher-level features to lower-level features. Finally, to encourage a more effective way to get the perceptual quality, a hierarchical multi-score quality aggregation method is proposed. The experimental results on four SIQA databases show that our CDFINet outperforms the compared mainstream metrics.","PeriodicalId":13159,"journal":{"name":"IEEE Transactions on Broadcasting","volume":"71 2","pages":"593-605"},"PeriodicalIF":4.8000,"publicationDate":"2025-01-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Cross-Domain Feature Interaction Network for Stereo Image Quality Assessment Considering Difference Information Guiding Binocular Fusion\",\"authors\":\"Yongli Chang;Guanghui Yue;Bo Zhao\",\"doi\":\"10.1109/TBC.2025.3525976\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Recently, convolutional neural network (CNN) based stereo image quality assessment (SIQA) has been extensively researched, achieving impressive performance. However, most SIQA methods tend to only mine features from distorted stereo image, neglecting the exploitation of valuable features present in other image domains. Moreover, some simple fusion strategies like addition and concatenation for binocular fusion further limit the network’s prediction performance. Therefore, we design a cross-domain feature interaction network (CDFINet) for SIQA in this paper, which considers the complementarity between different domain features and realizes binocular fusion between the left and right monocular features based on difference information. Specifically, to boost the prediction ability, we design a dual-branch network with image and gradient feature extraction branches, extracting hierarchical features from both domains. Moreover, to explore more proper binocular information, we propose a difference information guidance based binocular fusion (DIGBF) module to achieve the binocular fusion. Furthermore, to better achieve information compensation between image and gradient domain, binocular features obtained from image domain and gradient domain are fused in the proposed cross-domain feature fusion (CDFF) module. In addition, considering the feedback mechanism of the visual cortex, higher-level features are backpropagated to lower-level regions, and the proposed cross-layer feature interaction (CLFI) module realizes the guidance of higher-level features to lower-level features. Finally, to encourage a more effective way to get the perceptual quality, a hierarchical multi-score quality aggregation method is proposed. The experimental results on four SIQA databases show that our CDFINet outperforms the compared mainstream metrics.\",\"PeriodicalId\":13159,\"journal\":{\"name\":\"IEEE Transactions on Broadcasting\",\"volume\":\"71 2\",\"pages\":\"593-605\"},\"PeriodicalIF\":4.8000,\"publicationDate\":\"2025-01-22\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Transactions on Broadcasting\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10849593/\",\"RegionNum\":1,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"ENGINEERING, ELECTRICAL & ELECTRONIC\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Broadcasting","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10849593/","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
引用次数: 0

摘要

近年来,基于卷积神经网络(CNN)的立体图像质量评估(SIQA)得到了广泛的研究,并取得了令人瞩目的成绩。然而,大多数SIQA方法倾向于只从扭曲的立体图像中挖掘特征,而忽略了对其他图像域中存在的有价值特征的开发。此外,双目融合的一些简单的融合策略,如加法和拼接,进一步限制了网络的预测性能。为此,本文设计了一种面向SIQA的跨域特征交互网络(CDFINet),该网络考虑了不同域特征之间的互补性,并基于差异信息实现了左右单目特征的双目融合。具体来说,为了提高预测能力,我们设计了一个带有图像和梯度特征提取分支的双分支网络,从两个领域提取层次特征。此外,为了探索更合适的双目信息,我们提出了基于差分信息制导的双目融合(DIGBF)模块来实现双目融合。此外,为了更好地实现图像和梯度域之间的信息补偿,本文提出的跨域特征融合(cross-domain feature fusion, CDFF)模块将图像域和梯度域获得的双目特征进行融合。此外,考虑到视觉皮层的反馈机制,将高阶特征反向传播到低阶区域,提出的跨层特征交互(cross-layer feature interaction, CLFI)模块实现了高阶特征对低阶特征的引导。最后,为了更有效地获取感知质量,提出了一种分层多分值质量聚合方法。在四个SIQA数据库上的实验结果表明,我们的CDFINet优于主流指标。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
Cross-Domain Feature Interaction Network for Stereo Image Quality Assessment Considering Difference Information Guiding Binocular Fusion
Recently, convolutional neural network (CNN) based stereo image quality assessment (SIQA) has been extensively researched, achieving impressive performance. However, most SIQA methods tend to only mine features from distorted stereo image, neglecting the exploitation of valuable features present in other image domains. Moreover, some simple fusion strategies like addition and concatenation for binocular fusion further limit the network’s prediction performance. Therefore, we design a cross-domain feature interaction network (CDFINet) for SIQA in this paper, which considers the complementarity between different domain features and realizes binocular fusion between the left and right monocular features based on difference information. Specifically, to boost the prediction ability, we design a dual-branch network with image and gradient feature extraction branches, extracting hierarchical features from both domains. Moreover, to explore more proper binocular information, we propose a difference information guidance based binocular fusion (DIGBF) module to achieve the binocular fusion. Furthermore, to better achieve information compensation between image and gradient domain, binocular features obtained from image domain and gradient domain are fused in the proposed cross-domain feature fusion (CDFF) module. In addition, considering the feedback mechanism of the visual cortex, higher-level features are backpropagated to lower-level regions, and the proposed cross-layer feature interaction (CLFI) module realizes the guidance of higher-level features to lower-level features. Finally, to encourage a more effective way to get the perceptual quality, a hierarchical multi-score quality aggregation method is proposed. The experimental results on four SIQA databases show that our CDFINet outperforms the compared mainstream metrics.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
IEEE Transactions on Broadcasting
IEEE Transactions on Broadcasting 工程技术-电信学
CiteScore
9.40
自引率
31.10%
发文量
79
审稿时长
6-12 weeks
期刊介绍: The Society’s Field of Interest is “Devices, equipment, techniques and systems related to broadcast technology, including the production, distribution, transmission, and propagation aspects.” In addition to this formal FOI statement, which is used to provide guidance to the Publications Committee in the selection of content, the AdCom has further resolved that “broadcast systems includes all aspects of transmission, propagation, and reception.”
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信