Wentao Chao;Fuqing Duan;Xuechun Wang;Yingqian Wang;Ke Lu;Guanghui Wang
{"title":"OccCasNet:光场深度估计的光场感知级联成本体积","authors":"Wentao Chao;Fuqing Duan;Xuechun Wang;Yingqian Wang;Ke Lu;Guanghui Wang","doi":"10.1109/TCI.2024.3488563","DOIUrl":null,"url":null,"abstract":"Depth estimation using the Light Field (LF) technique is an essential task with a wide range of practical applications. While mainstream approaches based on multi-view stereo techniques can attain exceptional accuracy by creating finer cost volumes, they are resource-intensive, time-consuming, and often overlook occlusion during cost volume construction. To address these issues and strike a better balance between accuracy and efficiency, we propose an occlusion-aware cascade cost volume for LF depth (disparity) estimation. Our cascaded strategy reduces the sampling number while maintaining a constant sampling interval, enabling the construction of a finer cost volume. We also introduce occlusion maps to enhance accuracy in constructing the occlusion-aware cost volume. Specifically, we first generate a coarse disparity map through a coarse disparity estimation network. Then, we warp the sub-aperture images (SAIs) of adjacent views to the center view based on the coarse disparity map to generate occlusion maps for each SAI by photo-consistency constraints. Finally, we seamlessly incorporate occlusion maps into cascade cost volume to construct an occlusion-aware refined cost volume, allowing the refined disparity estimation network to yield a more precise disparity map. Extensive experiments demonstrate the effectiveness of our method. Compared with the state-of-the-art techniques, our method achieves a superior balance between accuracy and efficiency, ranking first in the Q25 metric on the HCI 4D benchmark.","PeriodicalId":56022,"journal":{"name":"IEEE Transactions on Computational Imaging","volume":"10 ","pages":"1680-1691"},"PeriodicalIF":4.2000,"publicationDate":"2024-10-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"OccCasNet: Occlusion-Aware Cascade Cost Volume for Light Field Depth Estimation\",\"authors\":\"Wentao Chao;Fuqing Duan;Xuechun Wang;Yingqian Wang;Ke Lu;Guanghui Wang\",\"doi\":\"10.1109/TCI.2024.3488563\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Depth estimation using the Light Field (LF) technique is an essential task with a wide range of practical applications. While mainstream approaches based on multi-view stereo techniques can attain exceptional accuracy by creating finer cost volumes, they are resource-intensive, time-consuming, and often overlook occlusion during cost volume construction. To address these issues and strike a better balance between accuracy and efficiency, we propose an occlusion-aware cascade cost volume for LF depth (disparity) estimation. Our cascaded strategy reduces the sampling number while maintaining a constant sampling interval, enabling the construction of a finer cost volume. We also introduce occlusion maps to enhance accuracy in constructing the occlusion-aware cost volume. Specifically, we first generate a coarse disparity map through a coarse disparity estimation network. Then, we warp the sub-aperture images (SAIs) of adjacent views to the center view based on the coarse disparity map to generate occlusion maps for each SAI by photo-consistency constraints. Finally, we seamlessly incorporate occlusion maps into cascade cost volume to construct an occlusion-aware refined cost volume, allowing the refined disparity estimation network to yield a more precise disparity map. Extensive experiments demonstrate the effectiveness of our method. Compared with the state-of-the-art techniques, our method achieves a superior balance between accuracy and efficiency, ranking first in the Q25 metric on the HCI 4D benchmark.\",\"PeriodicalId\":56022,\"journal\":{\"name\":\"IEEE Transactions on Computational Imaging\",\"volume\":\"10 \",\"pages\":\"1680-1691\"},\"PeriodicalIF\":4.2000,\"publicationDate\":\"2024-10-30\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Transactions on Computational Imaging\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10738443/\",\"RegionNum\":2,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"ENGINEERING, ELECTRICAL & ELECTRONIC\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Computational Imaging","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10738443/","RegionNum":2,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
OccCasNet: Occlusion-Aware Cascade Cost Volume for Light Field Depth Estimation
Depth estimation using the Light Field (LF) technique is an essential task with a wide range of practical applications. While mainstream approaches based on multi-view stereo techniques can attain exceptional accuracy by creating finer cost volumes, they are resource-intensive, time-consuming, and often overlook occlusion during cost volume construction. To address these issues and strike a better balance between accuracy and efficiency, we propose an occlusion-aware cascade cost volume for LF depth (disparity) estimation. Our cascaded strategy reduces the sampling number while maintaining a constant sampling interval, enabling the construction of a finer cost volume. We also introduce occlusion maps to enhance accuracy in constructing the occlusion-aware cost volume. Specifically, we first generate a coarse disparity map through a coarse disparity estimation network. Then, we warp the sub-aperture images (SAIs) of adjacent views to the center view based on the coarse disparity map to generate occlusion maps for each SAI by photo-consistency constraints. Finally, we seamlessly incorporate occlusion maps into cascade cost volume to construct an occlusion-aware refined cost volume, allowing the refined disparity estimation network to yield a more precise disparity map. Extensive experiments demonstrate the effectiveness of our method. Compared with the state-of-the-art techniques, our method achieves a superior balance between accuracy and efficiency, ranking first in the Q25 metric on the HCI 4D benchmark.
期刊介绍:
The IEEE Transactions on Computational Imaging will publish articles where computation plays an integral role in the image formation process. Papers will cover all areas of computational imaging ranging from fundamental theoretical methods to the latest innovative computational imaging system designs. Topics of interest will include advanced algorithms and mathematical techniques, model-based data inversion, methods for image and signal recovery from sparse and incomplete data, techniques for non-traditional sensing of image data, methods for dynamic information acquisition and extraction from imaging sensors, software and hardware for efficient computation in imaging systems, and highly novel imaging system design.