MRIM: Lightweight saliency-based mixed-resolution imaging for low-power pervasive vision

IF 3 3区 计算机科学 Q2 COMPUTER SCIENCE, INFORMATION SYSTEMS
Ji-Yan Wu, Vithurson Subasharan, Tuan Tran, Kasun Gamlath, Archan Misra
{"title":"MRIM: Lightweight saliency-based mixed-resolution imaging for low-power pervasive vision","authors":"Ji-Yan Wu,&nbsp;Vithurson Subasharan,&nbsp;Tuan Tran,&nbsp;Kasun Gamlath,&nbsp;Archan Misra","doi":"10.1016/j.pmcj.2023.101858","DOIUrl":null,"url":null,"abstract":"<div><p><span><span>While many pervasive computing applications increasingly utilize real-time context extracted from a vision sensing infrastructure, the high energy overhead of DNN-based vision sensing pipelines remains a challenge for sustainable in-the-wild deployment. One common approach to reducing such energy overheads is the capture and transmission of lower-resolution images to an edge node (where the </span>DNN inferencing task is executed), but this results in an accuracy-vs-energy tradeoff, as the DNN inference accuracy typically degrades with a drop in resolution. In this work, we introduce </span><em>MRIM</em>, a simple but effective framework to tackle this tradeoff. Under <em>MRIM</em><span>, the vision sensor platform first executes a lightweight preprocessing step to determine the saliency of different sub-regions within a single captured image frame, and then performs a saliency-aware non-uniform downscaling of individual sub-regions to produce a “mixed-resolution” image. We describe two novel low-complexity algorithms that the sensor platform can use to quickly compute suitable resolution choices for different regions under different energy/accuracy constraints. Experimental studies, involving object detection tasks evaluated traces from two benchmark urban monitoring datasets as well as a prototype Raspberry Pi-based </span><em>MRIM</em> implementation, demonstrate <em>MRIM’s</em> efficacy: even with an unoptimized embedded platform, <em>MRIM</em><span> can provide system energy conservation of </span><span><math><mrow><mn>35</mn><mo>+</mo><mtext>%</mtext></mrow></math></span> (<span><math><mo>∼</mo></math></span>80% in high accuracy regimes) or increase task accuracy by <span><math><mrow><mn>8</mn><mo>+</mo><mtext>%</mtext></mrow></math></span><span>, over conventional baselines of uniform resolution downscaling or image encoding, while supporting high throughput. On a low power ESP32 vision board, </span><em>MRIM</em><span> continues to provide 60+% energy savings over uniform downscaling while maintaining high detection accuracy. We further introduce an automated data-driven technique for determining a close-to-optimal number of </span><em>MRIM</em> sub-regions (for differential resolution adjustment), across different deployment conditions. We also show the generalized use of <em>MRIM</em><span> by considering an additional license plate recognition (LPR) task: while alternative approaches suffer 35%–40% loss in accuracy, </span><em>MRIM</em> suffers only a modest recognition loss of <span><math><mo>∼</mo></math></span>10% even when the transmission data is reduced by over 50%.</p></div>","PeriodicalId":49005,"journal":{"name":"Pervasive and Mobile Computing","volume":null,"pages":null},"PeriodicalIF":3.0000,"publicationDate":"2023-10-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Pervasive and Mobile Computing","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S1574119223001165","RegionNum":3,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0

Abstract

While many pervasive computing applications increasingly utilize real-time context extracted from a vision sensing infrastructure, the high energy overhead of DNN-based vision sensing pipelines remains a challenge for sustainable in-the-wild deployment. One common approach to reducing such energy overheads is the capture and transmission of lower-resolution images to an edge node (where the DNN inferencing task is executed), but this results in an accuracy-vs-energy tradeoff, as the DNN inference accuracy typically degrades with a drop in resolution. In this work, we introduce MRIM, a simple but effective framework to tackle this tradeoff. Under MRIM, the vision sensor platform first executes a lightweight preprocessing step to determine the saliency of different sub-regions within a single captured image frame, and then performs a saliency-aware non-uniform downscaling of individual sub-regions to produce a “mixed-resolution” image. We describe two novel low-complexity algorithms that the sensor platform can use to quickly compute suitable resolution choices for different regions under different energy/accuracy constraints. Experimental studies, involving object detection tasks evaluated traces from two benchmark urban monitoring datasets as well as a prototype Raspberry Pi-based MRIM implementation, demonstrate MRIM’s efficacy: even with an unoptimized embedded platform, MRIM can provide system energy conservation of 35+% (80% in high accuracy regimes) or increase task accuracy by 8+%, over conventional baselines of uniform resolution downscaling or image encoding, while supporting high throughput. On a low power ESP32 vision board, MRIM continues to provide 60+% energy savings over uniform downscaling while maintaining high detection accuracy. We further introduce an automated data-driven technique for determining a close-to-optimal number of MRIM sub-regions (for differential resolution adjustment), across different deployment conditions. We also show the generalized use of MRIM by considering an additional license plate recognition (LPR) task: while alternative approaches suffer 35%–40% loss in accuracy, MRIM suffers only a modest recognition loss of 10% even when the transmission data is reduced by over 50%.

mrm:用于低功耗普适视觉的轻量级显著性混合分辨率成像
虽然许多普然计算应用越来越多地利用从视觉传感基础设施中提取的实时上下文,但基于dnn的视觉传感管道的高能量开销仍然是可持续野外部署的一个挑战。减少这种能量开销的一种常见方法是捕获和传输低分辨率图像到边缘节点(DNN推理任务执行的地方),但这会导致精度与能量的权衡,因为DNN推理精度通常会随着分辨率的下降而降低。在这项工作中,我们介绍了一个简单但有效的框架,以解决这种权衡。在mrm下,视觉传感器平台首先执行轻量级预处理步骤,以确定单个捕获图像帧内不同子区域的显着性,然后对单个子区域进行显着性感知的非均匀降尺度,以产生“混合分辨率”图像。我们描述了两种新颖的低复杂度算法,传感器平台可以使用它们在不同能量/精度约束下快速计算不同区域的合适分辨率选择。通过对两个基准城市监测数据集和基于树莓派的原型MRIM实现的目标检测任务进行评估的实验研究,证明了MRIM的有效性:即使使用未优化的嵌入式平台,与统一分辨率降尺度或图像编码的传统基线相比,MRIM也可以提供35%以上的系统节能(在高精度情况下为80%)或将任务精度提高8%以上,同时支持高吞吐量。在低功耗ESP32视觉板上,通过均匀缩小,mrm继续提供60%以上的节能,同时保持高检测精度。我们进一步介绍了一种自动化数据驱动技术,用于在不同部署条件下确定接近最优的mrm子区域数量(用于差分分辨率调整)。我们还通过考虑额外的车牌识别(LPR)任务展示了MRIM的广泛使用:尽管其他方法的准确性会损失35%-40%,但即使传输数据减少50%以上,MRIM也只会遭受10%的适度识别损失。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
Pervasive and Mobile Computing
Pervasive and Mobile Computing COMPUTER SCIENCE, INFORMATION SYSTEMS-TELECOMMUNICATIONS
CiteScore
7.70
自引率
2.30%
发文量
80
审稿时长
68 days
期刊介绍: As envisioned by Mark Weiser as early as 1991, pervasive computing systems and services have truly become integral parts of our daily lives. Tremendous developments in a multitude of technologies ranging from personalized and embedded smart devices (e.g., smartphones, sensors, wearables, IoTs, etc.) to ubiquitous connectivity, via a variety of wireless mobile communications and cognitive networking infrastructures, to advanced computing techniques (including edge, fog and cloud) and user-friendly middleware services and platforms have significantly contributed to the unprecedented advances in pervasive and mobile computing. Cutting-edge applications and paradigms have evolved, such as cyber-physical systems and smart environments (e.g., smart city, smart energy, smart transportation, smart healthcare, etc.) that also involve human in the loop through social interactions and participatory and/or mobile crowd sensing, for example. The goal of pervasive computing systems is to improve human experience and quality of life, without explicit awareness of the underlying communications and computing technologies. The Pervasive and Mobile Computing Journal (PMC) is a high-impact, peer-reviewed technical journal that publishes high-quality scientific articles spanning theory and practice, and covering all aspects of pervasive and mobile computing and systems.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术官方微信