Vision-based real-time structural vibration measurement through deep-learning-based detection and tracking methods

• Published in: Engineering structures Vol. 281; p. 115676
• Main Authors: Pan, Xiao, Yang, T.Y., Xiao, Yifei, Yao, Hongcan, Adeli, Hojjat
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 15.04.2023
• Subjects: Computer vision; Deep learning; Object detection; Structural vibration measurement; Deep learning; Computer vision; Structural vibration measurement; Object detection
• ISSN: 0141-0296
• DOI: 10.1016/j.engstruct.2023.115676

•Real-time vision-based methods are proposed for structural vibration measurement.•The YOLOv3-tiny-KLT method achieves higher accuracy over the YOLOv3-tiny method.•The YOLOv3-tiny-KLT method achieves higher robustness over the KLT method.•The proposed methods address illumination changes and background noise. Structural vibration measurement is crucial in structural health monitoring and structural laboratory tests. Traditional contact type sensors are usually required to be attached to the test specimens, which may be difficult to install, and may affect the structural properties and response. Non-contact type wireless sensors are usually expensive and require specialized workers to install and operate. In recent years, vision-based tracking methods for structural vibration measurement have gained increasing interests due to their high accuracy, non-contact feature and low cost. However, traditional vision-based tracking algorithms are susceptible to external environmental conditions such as illumination and background noise. In this paper, two real-time methods, YOLOv3-tiny and YOLOv3-tiny-KLT, are proposed to track structural motions. In the first method, YOLOv3-tiny is established based on the YOLOv3 architecture to localize customized markers where structural displacements are directly determined from the bounding boxes generated. The second method, YOLOv3-tiny-KLT, is a more advanced method which combines the YOLOv3-tiny detector and the traditional KLT tracking algorithm. The pretrained YOLOv3-tiny is deployed to localize the targets automatically, which will then be tracked by Kanade‐Lucas‐Tomasi algorithm. YOLOv3-tiny is intended to provide baseline vibration measurement when the KLT tracking gets lost. The proposed methods were implemented for the videos of shake table tests on a two-storey steel structure. Parametric studies were conducted for the YOLOv3-tiny-KLT method to examine its sensitivity to the tracking parameters. The results show that the proposed method is capable of achieving real-time speed and high accuracy, when compared with the traditional displacement sensors including linear variable differential transducer (LVDT) and String Pots. It is also found that the combined YOLOv3-tiny-KLT approach achieves higher accuracy than YOLOv3-tiny only method, and higher robustness than KLT only method against illumination changes and background noise.