Jun-Tian Ye, Yi Sun, Wenwen Li, Jian-Wei Zeng, Yu Hong, Zheng-Ping Li, Xin Huang, Xianghui Xue, Xin Yuan, Feihu Xu, Xiankang Dou, Jian-Wei Pan
{"title":"Real-time non-line-of-sight computational imaging using spectrum filtering and motion compensation.","authors":"Jun-Tian Ye, Yi Sun, Wenwen Li, Jian-Wei Zeng, Yu Hong, Zheng-Ping Li, Xin Huang, Xianghui Xue, Xin Yuan, Feihu Xu, Xiankang Dou, Jian-Wei Pan","doi":"10.1038/s43588-024-00722-4","DOIUrl":null,"url":null,"abstract":"<p><p>Non-line-of-sight (NLOS) imaging aims at recovering the shape and albedo of hidden objects. Despite recent advances, real-time video of complex and dynamic scenes remains a major challenge owing to the weak signal of multiply scattered light. Here we propose and demonstrate a framework of spectrum filtering and motion compensation to realize high-quality NLOS video for room-sized scenes. Spectrum filtering leverages a wave-based model for denoising and deblurring in the frequency domain, enabling computational image reconstruction with a small number of sampling points. Motion compensation tailored with an interleaved scanning scheme can compute high-resolution live video during the acquisition of low-quality image sequences. Together, we demonstrate live NLOS videos at 4 fps for a variety of dynamic real-life scenes. The results mark a substantial stride toward real-time, large-scale and low-power NLOS imaging and sensing applications.</p>","PeriodicalId":74246,"journal":{"name":"Nature computational science","volume":null,"pages":null},"PeriodicalIF":12.0000,"publicationDate":"2024-11-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Nature computational science","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1038/s43588-024-00722-4","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS","Score":null,"Total":0}
引用次数: 0
Abstract
Non-line-of-sight (NLOS) imaging aims at recovering the shape and albedo of hidden objects. Despite recent advances, real-time video of complex and dynamic scenes remains a major challenge owing to the weak signal of multiply scattered light. Here we propose and demonstrate a framework of spectrum filtering and motion compensation to realize high-quality NLOS video for room-sized scenes. Spectrum filtering leverages a wave-based model for denoising and deblurring in the frequency domain, enabling computational image reconstruction with a small number of sampling points. Motion compensation tailored with an interleaved scanning scheme can compute high-resolution live video during the acquisition of low-quality image sequences. Together, we demonstrate live NLOS videos at 4 fps for a variety of dynamic real-life scenes. The results mark a substantial stride toward real-time, large-scale and low-power NLOS imaging and sensing applications.