Energy-Efficient Saliency-Guided Video Coding Framework for Real-Time Applications

IF 3.7 2区 工程技术 Q2 ENGINEERING, ELECTRICAL & ELECTRONIC
Tero Partanen;Minh Hoang;Alexandre Mercat;Joose Sainio;Jarno Vanne
{"title":"Energy-Efficient Saliency-Guided Video Coding Framework for Real-Time Applications","authors":"Tero Partanen;Minh Hoang;Alexandre Mercat;Joose Sainio;Jarno Vanne","doi":"10.1109/JETCAS.2024.3525339","DOIUrl":null,"url":null,"abstract":"The significant growth in global video data traffic can be mitigated by saliency-based video coding schemes that seek to increase coding efficiency without any loss of objective visual quality by compressing salient video regions less heavily than non-salient regions. However, conducting salient object detection (SOD) on every video frame before encoding tends to lead to substantial complexity and energy consumption overhead, especially if state-of-the-art deep learning techniques are used in saliency detection. This work introduces a saliency-guided video encoding framework that reduces the energy consumption over frame-by-frame SOD by increasing the detection interval and applying the proposed region-of-interest (ROI) tracking between successive detections. The computational complexity of our ROI tracking technique is kept low by predicting object movements from motion vectors, which are inherently calculated during encoding. Our experimental results demonstrate that the proposed ROI tracking solution saves energy by 86-95% and attains 84-94% accuracy over frame-by-frame SOD. Correspondingly, integrating our proposal into the complete saliency-guided video coding scheme reduces energy consumption on CPU by 79-82% at a cost of weighted PSNR of less than 5%. These findings indicate that our solution has significant potential for low-cost and low-power streaming media applications.","PeriodicalId":48827,"journal":{"name":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","volume":"15 1","pages":"44-57"},"PeriodicalIF":3.7000,"publicationDate":"2025-01-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10820524","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems","FirstCategoryId":"5","ListUrlMain":"https://ieeexplore.ieee.org/document/10820524/","RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
引用次数: 0

Abstract

The significant growth in global video data traffic can be mitigated by saliency-based video coding schemes that seek to increase coding efficiency without any loss of objective visual quality by compressing salient video regions less heavily than non-salient regions. However, conducting salient object detection (SOD) on every video frame before encoding tends to lead to substantial complexity and energy consumption overhead, especially if state-of-the-art deep learning techniques are used in saliency detection. This work introduces a saliency-guided video encoding framework that reduces the energy consumption over frame-by-frame SOD by increasing the detection interval and applying the proposed region-of-interest (ROI) tracking between successive detections. The computational complexity of our ROI tracking technique is kept low by predicting object movements from motion vectors, which are inherently calculated during encoding. Our experimental results demonstrate that the proposed ROI tracking solution saves energy by 86-95% and attains 84-94% accuracy over frame-by-frame SOD. Correspondingly, integrating our proposal into the complete saliency-guided video coding scheme reduces energy consumption on CPU by 79-82% at a cost of weighted PSNR of less than 5%. These findings indicate that our solution has significant potential for low-cost and low-power streaming media applications.
面向实时应用的高能效显著性视频编码框架
全球视频数据流量的显著增长可以通过基于显著性的视频编码方案来缓解,该方案通过压缩显著视频区域而不是非显著视频区域来寻求在不损失客观视觉质量的情况下提高编码效率。然而,在编码之前对每个视频帧进行显著目标检测(SOD)往往会导致大量的复杂性和能耗开销,特别是在显著性检测中使用最先进的深度学习技术时。这项工作引入了一个显著性引导的视频编码框架,通过增加检测间隔和在连续检测之间应用提议的感兴趣区域(ROI)跟踪来减少逐帧SOD的能量消耗。我们的ROI跟踪技术通过从运动向量预测物体运动来保持较低的计算复杂度,运动向量在编码过程中固有地计算。实验结果表明,所提出的ROI跟踪方案在逐帧SOD上节能86-95%,准确率达到84-94%。相应地,将我们的建议集成到完整的显著性引导视频编码方案中,以加权PSNR小于5%为代价,减少了79-82%的CPU能耗。这些发现表明,我们的解决方案在低成本和低功耗的流媒体应用中具有巨大的潜力。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
CiteScore
8.50
自引率
2.20%
发文量
86
期刊介绍: The IEEE Journal on Emerging and Selected Topics in Circuits and Systems is published quarterly and solicits, with particular emphasis on emerging areas, special issues on topics that cover the entire scope of the IEEE Circuits and Systems (CAS) Society, namely the theory, analysis, design, tools, and implementation of circuits and systems, spanning their theoretical foundations, applications, and architectures for signal and information processing.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信