Top > JRM > Real-Time Vibration Visualization Using GPU-Based High-Speed V ...

single-rb.php

JRM Vol.34 No.5 pp. 1011-1023
doi: 10.20965/jrm.2022.p1011
(2022)

Paper:

Views over last 60 days: 442

Real-Time Vibration Visualization Using GPU-Based High-Speed Vision

Feiyue Wang*, Shaopeng Hu*, Kohei Shimasaki**, and Idaku Ishii*

*Graduate School of Advanced Science and Engineering, Hiroshima University
1-4-1 Kagamiyama, Higashi-hiroshima, Hiroshima 739-8527, Japan

**Digital Monozukuri (Manufacturing) Education and Research Center, Hiroshima University
3-10-32 Kagamiyama, Higashi-hiroshima, Hiroshima 739-0046, Japan

Received:
April 7, 2022
Accepted:
July 1, 2022
Published:
October 20, 2022
Creative Commons License
Keywords:
vibration monitoring, high-speed vision, digital image correlation, short-time Fourier transform
Abstract

In this study, we developed a real-time vibration visualization system that can estimate and display vibration distributions at all frequencies in real time through parallel implementation of subpixel digital image correlation (DIC) computations with short-time Fourier transforms on a GPU-based high-speed vision platform. To help operators intuitively monitor high-speed motion, we introduced a two-step framework of high-speed video processing to obtain vibration distributions at hundreds of hertz and video conversion processing for the visualization of vibration distribution at dozens of hertz. The proposed system can estimate the full-field vibration displacements of 1920 × 1080 images in real time at 1000 fps and display their frequency responses in the range of 0–500 Hz on a computer at dozens of frames per second by accelerating phase-only DICs for full-field displacement measurement and video conversion. The effectiveness of this system for real-time vibration monitoring and visualization was demonstrated by conducting experiments on objects vibrating at dozens or hundreds of hertz.

Real-time visualization of vibration using high-speed vision

Real-time visualization of vibration using high-speed vision

Cite this article as:
F. Wang, S. Hu, K. Shimasaki, and I. Ishii, “Real-Time Vibration Visualization Using GPU-Based High-Speed Vision,” J. Robot. Mechatron., Vol.34 No.5, pp. 1011-1023, 2022.
Data files:
References
  1. [1] D. Goyal and B. S. Pabla, “The vibration monitoring methods and signal processing techniques for structural health monitoring: a review,” Arch. Comput. Methods Eng., Vol.23, No.4, pp. 585-594, 2016.
  2. [2] A. P. Daga and L. Garibaldi, “Machine vibration monitoring for diagnostics through hypothesis testing,” Information, Vol.10, No.6, 204, 2019.
  3. [3] X. Lei and Y. Wu, “Research on mechanical vibration monitoring based on wireless sensor network and sparse Bayes,” EURASIP J. Wireless Commun. Networking, Vol.2020, 225, 2020.
  4. [4] Q. Huang, B. Tang, and L. Deng, “Development of high synchronous acquisition accuracy wireless sensor network for machine vibration monitoring,” Measurement, Vol.66, pp. 35-44, 2015.
  5. [5] E. Caetano, S. Silva, and J. Bateira, “A vision system for vibration monitoring of civil engineering structures,” Exp. Tech., Vol.35, No.4, pp. 74-82, 2011.
  6. [6] E. Balms, M. Basseville, F. Bourquin, L. Mevel, H. Nasser, and F. Treyssede, “Merging sensor data from multiple temperature scenarios for vibration monitoring of civil structures,” Struct. Health Monit., Vol.7, No.2, pp. 129-142, 2008.
  7. [7] A. Zona, “Vision-based vibration monitoring of structures and infrastructures: An overview of recent applications,” Infrastructures, Vol.6, No.1, 4, 2021.
  8. [8] A. Cigada, P. Mazzoleni, and E. Zappa, “Vibration monitoring of multiple bridge points by means of a unique vision-based measuring system,” Exp. Mech., Vol.54, No.2, pp. 255-271, 2014.
  9. [9] T. C. Huynh, J. H. Park, and J. T. Kim, “Structural identification of cable-stayed bridge under back-to-back typhoons by wireless vibration monitoring,” Measurement, Vol.88, pp. 385-401, 2016.
  10. [10] P. S. Harvey Jr and G. Elisha, “Vision-based vibration monitoring using existing cameras installed within a building,” Struct. Contr. Health Monit., Vol.25, No.11, e2235, 2018.
  11. [11] N. Nakata and R. Snieder, “Monitoring a building using deconvolution interferometry. II: Ambient-vibration analysis,” Bull. Seismolological Soc. Am., Vol.104, No.1, pp. 204-213, 2014.
  12. [12] H. W. Paschold and A. G. Mayton, “Whole-body vibration: building awareness in SH and E,” Prof. Saf., Vol.56, No.04, pp. 30-35, 2011.
  13. [13] F. L. M. dos Santos, B. Peeters, J. Lau et al, “The use of strain gauges in vibration-based damage detection,” J. Phys: Conf. Series IOP Publishing, Vol.628, No.1, 012119, 2015.
  14. [14] M. Y. Cheng, K. W. Liao, Y. F. Chiu, Y. W. Wu, S. H. Yeh, and T. C. Lin, “Automated mobile vibration measurement and signal analysis for bridge scour prevention and warning,” Automat. Construct., Vol.134, 104063, 2022.
  15. [15] X. Meng, A. H. Dodson, and G. W. Roberts, “Detecting bridge dynamics with GPS and triaxial accelerometers,” Eng. Struct., Vol.29, No.11, pp. 3178-3184, 2007.
  16. [16] H. Nguyen, Z. Wang, P. Jones, and B. Zhao, “3D shape, deformation, and vibration measurements using infrared Kinect sensors and digital image correlation,” Appl. Opt., Vol.56, No.32, pp. 9030-9037, 2017.
  17. [17] L. Wu, Y. Su, Z. Chen, S. Chen, S. Cheng, and P. Lin, “Six-degree-of-freedom generalized displacements measurement based on binocular vision,” Struct. Contr. Health Monit., Vol.27, No.2, e2458, 2020.
  18. [18] M. Jiang, Q. Gu, T. Aoyama, T. Takaki, and I. Ishii, “Real-time vibration source tracking using high-speed vision,” IEEE Sens. J., Vol.17, No.5, pp. 1513-1527, 2017.
  19. [19] K. Shimasaki, T. Okamura, M. Jiang, T. Takaki, and I. Ishii, “Real-time high-speed vision-based vibration spectrum imaging,” Proc. IEEE/SICE Int. Symp. Syst. Integr., pp. 474-477, 2019.
  20. [20] Q. Hu, S. He, S. Wang, Y. Liu, Z. Zhang, L. He, F. Wang, Q. Cai, R. Shi, and Y. Yang, “A high-speed target-free vision-based sensor for bus rapid transit viaduct vibration measurements using CMT and ORB algorithms,” Sensors, Vol.17, No.6, 1305, 2017.
  21. [21] H. S. Park, H. Y. Lee, S. W. Choi, and Y. Kim, “A practical monitoring system for the structural safety of mega-trusses using wireless vibrating wire strain gauges,” Sensors, Vol.13, No.12, pp. 17346-17361, 2013.
  22. [22] D. C. Kammer and M. L. Tinker, “Optimal placement of triaxial accelerometers for modal vibration tests,” Mech. Syst. Signal Process., Vol.18, No.1, pp. 29-41, 2004.
  23. [23] A. Umeda, M. Onoe, K. Sakata, T. Fukushia, K. Kanari, H. Iioka, and T. Kobayashi, “Calibration of three-axis accelerometers using a three-dimensional vibration generator and three laser interferometers,” Sens. Actuators A: Phys., Vol.114, No.1, pp. 93-101, 2004.
  24. [24] A. Sabato, C. Niezrecki, and G. Fortino, “Wireless MEMS-based accelerometer sensor boards for structural vibration monitoring: a review,” IEEE Sens. J., Vol.17, No.2, pp. 226-235, 2016.
  25. [25] S. Kalaiselvi, L. Sujatha, and R. Sundar, “Fabrication of MEMS accelerometer for vibration sensing in gas turbine,” Proc. 2018 IEEE SENSORS, pp. 1-4, 2018.
  26. [26] X. Zhang, Q. Shen, and X. Liu, “A High Sensitivity MEMS-based Accelerometer with Reduced Cross-axis Coupling for Vibration Detection,” Proc. IEEE Int. Conf. Unmanned Syst., pp. 951-954, 2019.
  27. [27] S. J. Rothberg, M. S. Allen, P. Castellini, D. D. Maio, J. J. J. Dirckx, D. J. Ewins, B. J. Halkon, P. Muyshondt, N. Paone, T. Ryan, H. Steger, E. P. Tomasini, S. Vanlanduit, and J. F. Vignola, “An international review of laser Doppler vibrometry: Making light work of vibration measurement,” Opt. Lasers Eng., Vol.99, pp. 11-22, 2017.
  28. [28] N. Hasheminejad, C. Vuye, J. Dirckx, and S. Vanlanduit, “A comparative study of laser Doppler vibrometers for vibration measurements on pavement materials,” Infrastructures, Vol.3, No.4, 47, 2018.
  29. [29] H. Khalil, D. Kim, J. Nam, and K. Park, “Accuracy and noise analyses of 3D vibration measurements using laser Doppler vibrometer,” Measurement, Vol.94, pp. 883-892, 2016.
  30. [30] M. Kalybek, M. Bocian, and N. Nikitas, “Performance of optical structural vibration monitoring systems in experimental modal analysis,” Sensors, Vol.21, No.4, 1239, 2021.
  31. [31] Z. Qiu, X. Wang, X. M. Zhang, and J. Liu, “A novel vibration measurement and active control method for a hinged flexible two-connected piezoelectric plate,” Mech. Syst. Signal Process, Vol.107, pp. 357-395, 2018.
  32. [32] J. Luo, B. Liu, P. Yang, and X. Fan, “High-speed vision measurement of vibration based on an improved ZNSSD template matching algorithm,” Syst. Sci. Contr. Eng., Vol.10, No.1, pp. 43-54, 2022.
  33. [33] Y. Wang, J. Brownjohn, J. A. J. Capilla, K. Dai, W. Lu, and K. Y. Koo, “Vibration investigation for telecom structures with smartphone camera: case studies,” J. Civil Struct. Health Monit., Vol.11, No.3, pp. 757-766, 2021.
  34. [34] D. H. Diamond, P. S. Heyns, and A. J. Oberholster, “Accuracy evaluation of sub-pixel structural vibration measurements through optical flow analysis of a video sequence,” Measurement, Vol.95, pp. 166-172, 2017.
  35. [35] C. Z. Dong, O. Celik, F. N. Catbas, E. J. O’Brien, and S. Taylor, “Structural displacement monitoring using deep learning-based full field optical flow methods,” Struct. Infrastruct. Eng., Vol.16, No.1, pp. 51-71, 2020.
  36. [36] T. J. Beberniss and D. A. Ehrhardt, “High-speed 3D digital image correlation vibration measurement: Recent advancements and noted limitations,” Mech. Syst. Signal Process., Vol.86, pp. 35-48, 2017.
  37. [37] P. L. Reu, D. P. Rohe, and L. D. Jacobs, “Comparison of DIC and LDV for practical vibration and modal measurements,” Mech Syst. Signal Process., Vol.86, pp. 2-16, 2017.
  38. [38] B. Pan, K. Qian, H. Xie, and A. Asundi, “Two-dimensional digital image correlation for in-plane displacement and strain measurement: a review,” Meas. Sci. Tech., Vol.20, No.6, 062001, 2009.
  39. [39] G. Y. Jeong, A. Zink-Sharp, and D. P. Hindman, “Tensile properties of earlywood and latewood from loblolly pine (Pinus taeda) using digital image correlation,” Wood Fiber Sci,, Vol.41, No.1, pp. 51-63, 2009.
  40. [40] R. Fedele, B. Raka, F. Hild, and S. Raux, “Identification of adhesive properties in GLARE assemblies using digital image correlation,” J. Mech. Phys. Solids, Vol.57, No.7, pp. 1003-1016, 2009.
  41. [41] S. Yoneyama and H. Ueda, “Bridge deflection measurement using digital image correlation with camera movement correction,” Mater. Trans., Vol.53, No.2, pp. 285-290, 2012.
  42. [42] W. H. Peters and W. F. Ranson, “Digital imaging techniques in experimental stress analysis,” Opt. Eng., Vol.21, No.3, pp. 427-431, 1982.
  43. [43] A. Giachetti, “Matching techniques to compute image motion,” Image Vision Comput., Vol.18, No.3, pp. 247-260, 2000.
  44. [44] B. Pan, Z. Wang, and H. Xie, “Generalized spatial-gradient-based digital image correlation for displacement and shape measurement with subpixel accuracy,” J. Strain Anal. Eng. Des., Vol.44, No.8, pp. 659-669, 2009.
  45. [45] W. Tong, “An evaluation of digital image correlation criteria for strain mapping applications,” Strain, Vol.41, No.4, pp. 167-175, 2005.
  46. [46] B. Pan, “Recent progress in digital image correlation,” Exp. Mech., Vol.51, No.7, pp. 1223-1235, 2011.
  47. [47] M. A. Sutton, C. Mingqi, W. H. Peters, Y. J. Chao, and S. R. McNeill, “Application of an optimized digital correlation method to planar deformation analysis,” Image Vision Comput., Vol.4, No.3, pp. 143-150, 1986.
  48. [48] D. J. Chen, F. P. Chiang, Y. S. Tan, and H. S. Don, “Digital speckle-displacement measurement using a complex spectrum method,” Appl. Opt., Vol.32, No.11, pp. 1839-1849, 1993.
  49. [49] H. W. Schreier, J. R. Braasch, and M. A. Sutton, “Systematic errors in digital image correlation caused by intensity interpolation,” Opt. Eng., Vol.39, No.11, pp. 2915-2921, 2000.
  50. [50] J. Zhang, G. Jin, S. Ma, and L. Meng, “Application of an improved subpixel registration algorithm on digital speckle correlation measurement,” Opt. Laser Tech., Vol.35, No.7, pp. 533-542, 2003.
  51. [51] H. Jin and H. A. Bruck, “Pointwise digital image correlation using genetic algorithms,” Exp. Tech., Vol.29, No.1, pp. 36-39, 2005.
  52. [52] B. Pan, K. Li, and W. Tong, “Fast, robust and accurate digital image correlation calculation without redundant computations,” Exp. Mech., Vol.53, No.7, pp. 1277-1289, 2013.
  53. [53] L. Luu, Z. Wang, M. Vo, T. Hoang, and J. Ma, “Accuracy enhancement of digital image correlation with B-spline interpolation,” Opt. Lett., Vol.36, No.16, pp. 3070-3072, 2011.
  54. [54] M. Ren, J. Liang, Z. Tang, X. Guo, and L. G. Li, “Optimized interpolation filter for digital image correlation methods,” J. Xian Jiaotong Univ., pp. 65-70, 2014.
  55. [55] M. Ren, J. Liang, and B. Wei, “Accurate B-spline-based 3-D interpolation scheme for digital volume correlation,” Rev. Sci. Instr., Vol.87, No.12, 125114, 2016.
  56. [56] Y. Zhou, C. Sun, and J. Chen, “Adaptive subset offset for systematic error reduction in incremental digital image correlation,” Opt. Lasers Eng., Vol.55, pp. 5-11, 2014.
  57. [57] D. Wang, Y. Jiang, W. Wang, and Y. Wang, “Bias reduction in sub-pixel image registration based on the anti-symmetric feature,” Meas. Sci. Tech., Vol.27, No.3, 035206, 2016.
  58. [58] B. Pan, “Bias error reduction of digital image correlation using Gaussian pre-filtering,” Opt. Lasers Eng., Vol.51, No.10, pp. 1161-1167, 2013.
  59. [59] I. Ishii, T. Tatebe, Q. Gu, Y. Moriue, T. Takaki, and K. Tajima, “2000 fps real-time vision system with high-frame-rate video recording,” Proc. IEEE Int. Conf. Robot. Automat., pp. 1536-1541, 2010.
  60. [60] M. Hirabayashi, Y. Saito, K. Murakami, A. Ohsato, S. Kato, and M. Edahiro, “Vision-Based Sensing Systems for Autonomous Driving: Centralized or Decentralized?,” J. Robot. Mechatron., Vol.33, No.3, pp. 686-697, 2021.
  61. [61] Y. Nie, T. Takaki, I. Ishii, and H. Matsuda, “Algorithm for automatic behavior quantification of laboratory mice using high-frame-rate videos,” SICE J. Contr. Meas. Syst. Integr., Vol.4, No.5, pp. 322-331, 2011.
  62. [62] Y. Yoshimoto and H. Tamukoh, “FPGA Implementation of a Binarized Dual Stream Convolutional Neural Network for Service Robots,” J. Robot. Mechatron., Vol.33, No.2, pp. 386-399, 2021.
  63. [63] X. Jiang, Q. Gu, T. Aoyama, T. Takaki, and I. Ishii, “A High-Speed Vision System with Multithread Automatic Exposure Control for High-Dynamic-Range Imaging,” J. Robot. Mechatron., Vol.30, No.1, pp. 117-127, 2018.
  64. [64] J. Takei, S. Kagami, and K. Hashimoto, “3,000-fps 3-D shape measurement using a high-speed camera-projector system,” Proc. IEEE/RSJ Int. Conf. Intelli. Robot. Syst., pp. 3211-3216, 2007.
  65. [65] Y. Watanabe, T. Komuro, S. Kagami, and M. Ishikawa, “Real-time visual measurements using high-speed vision,” Proc. Mach. Vision Optomechatr, Appl., pp. 234-242, 2004.
  66. [66] I. Ishii, T. Ichida, Q. Gu, and T. Takaki, “500-fps face tracking system,” J. Real-time Image Process., Vol.8, No.4, pp. 379-388, 2013.
  67. [67] Q. Gu, T. Takaki, and I. Ishii, “Fast FPGA-based multiobject feature extraction,” IEEE Trans. Circ. Syst. Video Tech., Vol.23, No.1, pp. 30-45, 2012.
  68. [68] I. Ishii, T. Tatebe, Q. Gu, and T. Takaki, “Color-histogram-based tracking at 2000 fps,” J. Elecr. Imaging, Vol.21, No.1, 013010, 2012.
  69. [69] I. Ishii, T. Taniguchi, K. Yamamoto, and T. Takaki, “High-framerate optical flow system,” IEEE Trans. Circ. Syst. Video Tech., Vol.22, No.1, pp. 105-112, 2012.
  70. [70] M. Jiang, T. Aoyama, T. Takaki, and I. Ishii, “Pixel-level and robust vibration source sensing in high-frame-rate video analysis,” Sensors, Vol.16, No.11, 1842, 2016.
  71. [71] M. Jiang, Q. Gu, T. Aoyama, T. Takaki, and I. Ishii, “Real-time vibration source tracking using high-speed vision,” IEEE Sens. J., Vol.17, No.5, pp. 1513-1527, 2017.
  72. [72] K. Shimasaki, M. Jiang, T. Takaki, I. Ishii, and K. Yamamoto, “HFR-video-based honeybee activity sensing,” IEEE Sens. J., Vol.20, No.10, pp. 5575-5587, 2020.
  73. [73] K. Shimasaki, N. Fujiwara, S. Hu, T. Senoo, and I. Ishii, “High-frame-rate Video-based Multicopter Tracking System Using Pixel-level Short-time Fourier Transform,” J. Intelli. Robot. Syst., Vol.103, No.2, 36, 2021.
  74. [74] C. D. Kuglin and D. C. Hines, “The phase correlation image alignment method,” Proc. Int. Conf. Cybern. Soc., pp. 163-165, 1975.

Creative Commons License  This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Apr. 18, 2024