Mozart: A Mobile ToF System for Sensing in the Dark through Phase Manipulation

Zhiyuan Xie, Xiaomin Ouyang, Lilin Pan, Wenrui Lu, Guoliang Xing, Xiaoming Liu
{"title":"Mozart: A Mobile ToF System for Sensing in the Dark through Phase Manipulation","authors":"Zhiyuan Xie, Xiaomin Ouyang, Lilin Pan, Wenrui Lu, Guoliang Xing, Xiaoming Liu","doi":"10.1145/3581791.3596840","DOIUrl":null,"url":null,"abstract":"Sensing in low-light and dark environments has a wide range of applications. However, existing sensing technologies suffer several major challenges, such as excessive noise and low resolution. This paper proposes Mozart - a new mobile sensing system that leverages off-the-shelf Time-of-Flight (ToF) depth cameras to generate high-resolution and rich-in-texture maps for applications in dark scenarios. The design of Mozart is based on our key observation that the phase components of ToF measurements can be manipulated to expose texture information. Through in-depth analysis of the physical reflection model, we show that the textures can be exposed and enhanced using highly compute-efficient phase manipulation functions. By exploiting the physics texture models, we propose an autoencoder-based unsupervised learning approach that can automatically learn efficient representations from phase components to generate high-resolution maps. We implemented Mozart on several Android smartphone models1, and an edge testbed with standalone ToF camera platforms for various applications in the dark. The results show that Mozart can work in real time and delivers significant improvement over existing sensing technologies. Therefore, Mozart offers a low-cost, high-performance sensing technology for next-generation applications in the dark.","PeriodicalId":195110,"journal":{"name":"Proceedings of the 21st Annual International Conference on Mobile Systems, Applications and Services","volume":"5 1","pages":"0"},"PeriodicalIF":0.0000,"publicationDate":"2023-06-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Proceedings of the 21st Annual International Conference on Mobile Systems, Applications and Services","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1145/3581791.3596840","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
引用次数: 0

Abstract

Sensing in low-light and dark environments has a wide range of applications. However, existing sensing technologies suffer several major challenges, such as excessive noise and low resolution. This paper proposes Mozart - a new mobile sensing system that leverages off-the-shelf Time-of-Flight (ToF) depth cameras to generate high-resolution and rich-in-texture maps for applications in dark scenarios. The design of Mozart is based on our key observation that the phase components of ToF measurements can be manipulated to expose texture information. Through in-depth analysis of the physical reflection model, we show that the textures can be exposed and enhanced using highly compute-efficient phase manipulation functions. By exploiting the physics texture models, we propose an autoencoder-based unsupervised learning approach that can automatically learn efficient representations from phase components to generate high-resolution maps. We implemented Mozart on several Android smartphone models1, and an edge testbed with standalone ToF camera platforms for various applications in the dark. The results show that Mozart can work in real time and delivers significant improvement over existing sensing technologies. Therefore, Mozart offers a low-cost, high-performance sensing technology for next-generation applications in the dark.
莫扎特:通过相位操纵在黑暗中感应的移动ToF系统
低光暗环境下的传感具有广泛的应用前景。然而,现有的传感技术面临着一些主要的挑战,如噪声过大和分辨率低。本文提出了Mozart——一种新的移动传感系统,它利用现成的飞行时间(ToF)深度相机来生成高分辨率和丰富纹理的地图,用于黑暗场景的应用。莫扎特的设计是基于我们的关键观察,即ToF测量的相位分量可以被操纵以暴露纹理信息。通过对物理反射模型的深入分析,我们表明可以使用高效的相位操作函数来暴露和增强纹理。通过利用物理纹理模型,我们提出了一种基于自编码器的无监督学习方法,该方法可以自动学习相位分量的有效表示以生成高分辨率地图。我们在几个Android智能手机型号1上实现了Mozart,并在一个带有独立ToF相机平台的边缘测试平台上实现了在黑暗中的各种应用。结果表明,莫扎特可以实时工作,并且比现有的传感技术有了显著的改进。因此,莫扎特为下一代在黑暗中的应用提供了低成本、高性能的传感技术。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 求助全文
来源期刊
自引率
0.00%
发文量
0
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
确定
请完成安全验证×
copy
已复制链接
快去分享给好友吧!
我知道了
右上角分享
点击右上角分享
0
联系我们:info@booksci.cn Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。 Copyright © 2023 布克学术 All rights reserved.
京ICP备2023020795号-1
ghs 京公网安备 11010802042870号
Book学术文献互助
Book学术文献互助群
群 号:604180095
Book学术官方微信
小红书