Multi-vision-based displacement monitoring using global-local deep deblurring and Rauch-Tung-Striebel smoother

Abstract

Measuring structural vibrations help assess dynamic performances of civil structures and infrastructure. Although conventional displacement sensors have been widely adopted, they are contact-based methods which lack scalability. Recently, computer vision (CV) has been applied as a noncontact method to measure displacements. However, fast speed of structural vibration (e.g., in shake table tests) can inevitably cause motion blur that imposes challenges in all image-based object/feature detections, especially for normal portable cameras (without high-speed shutters). To address such issue, the study proposed a multi-vision, full-field sensing framework with affordable cameras using a novel global–local detection and deblurring (GLDD) module, which was designed with a generative adversarial network (GAN)-based deblurring model to enhance detection efficiency and accuracy by restoring blemished videos from multiple perspectives. Rauch-Tung-Striebel (RTS) smoother was studied for data fitting using incomplete observations caused due to severe motion-induced blurs. A shake table test was conducted on an aluminum frame with cameras and conventional sensors monitoring the structural vibrations. Fiducial markers were used to track the movement of the key locations on the structure. Results showed that the proposed method is satisfactory to monitor shake table tests when compared to conventional measurements with root-mean-square errors of 0.51–0.95 mm. The proposed deblurring module restored misdetection by 92.1 %, 50.6 %, and 25.2 % for mild-, medium-, and severe-level motion blurs, respectively. Smoother-based data fitting outperformed filter-based one when dealing with highly blemished images. The proposed monitoring system with GLDD and RTS smoother-based data fitting provides a robust measurement solution when dealing with motion blurs.