Paper:

# Simultaneous Dynamics-Based Visual Inspection Using Modal Parameter Estimation

## Hua Yang, Takeshi Takaki, and Idaku Ishii

Robotics Laboratory, Graduate School of Engineering, Hiroshima University, 1-4-1 Kagamiyama, Higashi-hiroshima, Hiroshima 739-8527, Japan

In this study, we introduce the concept of dynamicsbased visual inspection with High-Frame-Rate (HFR) video analysis as a novel non-destructive active sensing method for verifying dynamic properties of a vibrating object. The HFR video is used for determining the structural dynamic properties of an object, such as its resonant frequencies and mode shapes, which can be estimated as modal parameters by modal analysis only when the object is excited. By improving and implementing a fast output-only modal parameter estimation algorithm on a real-time 2000-fps vision platform, the modal parameters of an excited object are simultaneously estimated as its input-invariant dynamic properties for dynamics-based visual inspection evenwhen the objects undergo different excitation conditions. Our simultaneous 2000-fps visual inspection system can facilitate non-destructive and longterm monitoring of the structures of beam-shaped objects vibrating at dozens or hundreds of hertz, and it can detect small changes in the dynamic properties of these objects caused by internal defects such as fatigue cracks in real time, even when their static appearances are similar. To demonstrate the performance of the proposed 2000-fps simultaneous dynamics-based visual inspection approach, the resonant frequencies and mode shapes for beam-shaped cantilevers with different artificial cracks and weights, excited by human finger tapping, were estimated in real time.

*J. Robot. Mechatron.*, Vol.23, No.1, pp. 180-195, 2011.

- [1] J. He and Z. F. Fu, “Modal analysis, Butterworth-Heinemann,” Oxford, 2001.
- [2] C. R. Pickrel, “A practical approach to modal pretest design,” Mechanical Systems and Signal Processing, Vol.13, No.2, pp. 271-295, 1999.
- [3] H. Korsch, A. Dafnis, and H. G. Reimerdes, “Dynamic qualification of the HIRENASD elastic wing model,” Aerospace Science and Technology, Vol.13, No.2-3, pp. 130-138, 2009.
- [4] P. Bonello and P. M. Hai, “A receptance harmonic balance technique for the computation of the vibration of a whole aero-engine model with nonlinear bearings,” J. of Sound and Vibration, Vol.324, No.1-2, pp. 221-242, 2009.
- [5] M. Okuma, “Correction of finite element models using experimental modal data for vibration analysis,” Finite Elements in Analysis and Design, Vol.14, No.2-3, pp. 153-162, 1993.
- [6] L. Hermans and H. Van der Auweraer, “Modal testing and analysis of structures under operational conditions: industrial applications,” Mechanical Systems and Signal Processing, Vol.13, No.2, pp. 193-216, 1999.
- [7] T. Yamaguchi, Y. Kurosawa, and H. Enomoto, “Damped vibration analysis using finite element method with approximated modal damping for automotive double walls with a porous material,” J. of Sound and Vibration, Vol.325, No.1-2, pp. 436-450, 2009.
- [8] A. N. Damir, A. Elkhatib, and G. Nassef, “Prediction of fatigue life using modal analysis for grey and ductile cast iron,” Int. J. of Fatigue, Vol.29, No.3, pp. 499-507, 2007.
- [9] E. Reynders and G. D. Roeck, “Reference-based combined deterministic-stochastic subspace identification for experimental and operational modal analysis,” Mechanical Systems and Signal Processing, Vol.22, No.3, pp. 617-637, 2008.
- [10] C. Devriendt, G. D. Sitter, S. Vanlanduit, and P. Guillaume, “Operational modal analysis in the presence of harmonic excitations by the use of transmissibility measurements,” Mechanical Systems and Signal Processing, Vol.23, No.3, pp. 621-635, 2009.
- [11] Y. F. Ji and C. C. Chang, “Nontarget stereo vision technique for spatiotemporal response measurement of line-like structures,” J. of Engineering Mechanics, Vol.134, No.6, pp. 466-474, 2008.
- [12] P. Kohut and P. Kurowski, “Application of modal analysis supported by 3D vision-based measurements,” J. of Theoretical and Applied Mechanics, Vol.47, No.4, pp. 855-870, 2009.
- [13] J. Morlier and G. Michon, “Virtual vibration measurement using KLT motion tracking algorithm,” J. of Dynamic Systems, Measurement, and Control, Vol.132, No.1, pp. 011003-011011, 2010.
- [14] T. M. Bernard, B. Y. Zavidovique, and F. J. Devos, “A programmable artificial retina,” IEEE J. of Solid-State Circuits, Vol.28, No.7, pp. 789-797, 1993.
- [15] J. E. Eklund, C. Svensson, and A. Astrom, “VLSI implementation of a focal plane image processor – A realization of the near-sensor image processing concept,” IEEE Trans. on VLSI Systems, Vol.4, No.3, pp. 322-335, 1996.
- [16] T. Komuro, S. Kagami, and M. Ishikawa, “A Dynamically Reconfigurable SIMD Processor for a Vision Chip,” IEEE J. of Solid-State Circuits, Vol.39, No.1, pp. 265-268, 2004.
- [17] I. Ishii, K. Yamamoto, and M. Kubozono, “Higher order autocorrelation vision chip,” IEEE Trans. on Electron Devices, Vol.53, No.8, pp. 1797-1804, 2006.
- [18] Y. Watanabe, T. Komuro, and M. Ishikawa, “955-fps real-time shape measurement of a moving/deforming object using high-speed vision for numerous-point analysis,” Proc. of IEEE Int. Conf. on Robotics and Automation, pp. 3192-3197, 2007.
- [19] S. Hirai, M. Zakoji, A. Masubuchi, and T. Tsuboi, “Realtime FPGA-based vision system,” J. of Robotics and Mechatronics, Vol.17, No.4, pp. 401-409, 2005.
- [20] I. Ishii, T. Taniguchi, R. Sukenobe, and K. Yamamoto, “Development of high-speed and real-time vision platform, H3 Vision,” Proc. of IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 3671-3678, 2009.
- [21] I. Ishii, T. Tatebe, Q. Gu, Y. Moriue, T. Takaki, and K. Tajima, “2000 fps real-time vision system with high-frame-rate video recording,” Proc. of IEEE Int. Conf. on Robotics and Automation, pp. 1536-1541, 2010.
- [22] I. Ishii, Y. Nakabo, and M. Ishikawa, “Target tracking algorithm for 1ms visual feedback system using massively parallel processing,” Proc. of IEEE Int. Conf. on Robotics and Automation, pp. 2309-2314, 1996.
- [23] A. Namiki, Y. Imai, M. Ishikawa, and M. Kaneko, “Development of a high-speed multifingered hand system and its application to catching,” Proc. of IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 2666-2671, 2003.
- [24] Y. Nakamura, K. Kishi, and H. Kawakami, “Heartbeat synchronization for robotic cardiac surgery,” Proc. of IEEE Int. Conf. on Robotics and Automation, pp. 2014-2019, 2001.
- [25] Y. Nie, I. Ishii, K. Yamamoto, K. Orito, and H.Matsuda, “Real-time scratching behavior quantification system for laboratory mice using high-speed vision,” J. of Real-Time Image Processing, Vol.4, No.2, pp. 181-190, 2009.
- [26] L. Mihaylova, T. Lefebvre, H. Bruyninckx, K. Gadeyne, and J. D. Schutter, “Active sensing for robotics – a survey,” Proc. of 5th Int. Conf. on Numerical Methods and Applications, pp. 316-324, 2002.
- [27] Y. Aloimonous, I.Weiss, and A. Bandopadhay, “Active vision,” Int. J. of Computer Vision, Vol.1, No.4, pp. 333-356, 1987.
- [28] D. Ballard, “Animate vision,” Artificial Intelligence, Vol.48, No.1, pp. 57-86, 1991.
- [29] T. Matsuyama, S. Hiura, T. Wada, K. Murase, and A. Toshioka, “Dynamic memory: architecture for real time integration of visual perception, camera action, and network communication,” Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 728-735, 2000.
- [30] R. Bajcsy, “Real-time obstacle avoidance algorithm for visual navigation,” Proc. of the 3rd Workshop on Computer Vision: Representation and Control, pp. 55-59, 1985.
- [31] G. N. DeSouza and A. Kak, “Vision for mobile robot navigation: a survey,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.24, No.2, pp. 237-267, 2002.
- [32] A. Nuchter and H. Surmann, “6D SLAM with an application in autonomous mine mapping,” Proc. of IEEE Int. Conf. on Robotics and Automation, pp. 1998-2003, 2004.
- [33] R. S. Desai and R. A. Volz, “Identification and verification of termination conditions in fine motion in presence of sensor errors and geometric uncertainties,” Proc. of IEEE Int. Conf. on Robotics and Automation, pp. 800-807, 1989.
- [34] K. Nagata, M. Ooki, and M. Kakikur, “Feature detection with an image based compliant tactile sensor,” Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 838-843, 1992.
- [35] R. D. Howe and M. R. Cutkosky, “Dynamic tactile sensing: perception of fine surface features with stress rate sensing,” IEEE Trans. on Robotics and Automation, Vol.9, No.2, pp. 140-151, 1993.
- [36] R. A. Russell, “Using tactile whiskers to measure surface contours,” Proc. of IEEE Int. Conf. on Robotics and Automation, pp. 1295-1299, 1992.
- [37] M. Kaneko, N. Kanayama, and T. Tsuji, “Vision based active sensor using a flexible beam,” IEEE/ASME Trans. onMechatronics, Vol.6, No.1, pp. 7-16, 2001.
- [38] T. N. Clements and C. D. Rahn, “Three-dimensional contact imaging with an actuated whisker,” IEEE Trans. on Robotics and Automation, Vol.22, No.4, pp. 844-848, 2006.
- [39] M. Kaneko, K. Tokuda, and T. Kawahara, “Dynamic sensing of human eye,” Proc. of IEEE Int. Conf. on Robotics and Automation, pp. 2882-2887, 2005.
- [40] T. Kawahara, S. Tanaka, and M. Kaneko, “Non-contact stiffness imager,” Int. J. of Robotics Research, Vol.25, No.5-6, pp. 537-549, 2006.
- [41] N. Tanaka and M. Kaneko, “Skin surface shock wave,” Proc. of IEEE Engineering In Medicine and Biology Annual Conference, pp. 4123-4126, 2006.
- [42] J. S. Bendat and A. G. Piersol, “Engineering Applications of Correlation and Spectral Analysis,” 2nd edition, JohnWiley & Sons, New York, 1993.
- [43] A. Felber, “Development of hybrid bridge evaluation system,” Ph.D. thesis, University of British Columbia, Vancouver, Canada, 1993.
- [44] R. Brincker, L. Zhang, and P. Andersen, “Modal identification of output-only systems using frequency domain decomposition,” Smart Materials and Structures, Vol.10, No.3, pp. 441-445, 2001.
- [45] R. Brincker, C. Ventura, and P. Andersen, “Damping estimation by frequency domain decomposition,” Proc. of SPIE, Vol.4359, pp. 698-703, 2001.
- [46] H. A. Cole, “On-the-line analysis of random vibrations,” AIAA Paper, No.68-288, 1968.
- [47] S. R. Ibrahim and E. C. Mikulcik, “A method for the direct identification of vibration parameters from the free response,” The Shock and Vibration Bulletin, Vol.47, No.4, pp. 183-98, 1977.
- [48] F. Deblauwe, R. J. Allemang, and D. L. Brown, “The polyreference time domain technique,” Proc. of Int. Modal Analysis Conf., pp. 832-845, 1987.
- [49] P. E. Gautier, C. Gontier, and M. Smail, “Robustness of an ARMA identification method for modal analysis of mechanical systems in the presence of noise,” J. of Sound and Vibration, Vol.179, No.2, pp. 227-242, 1995.
- [50] J. N. Juang and R. S. Pappa, “An eigensystem realization algorithm for modal parameter identification and model reduction,” J. of Guidance, Control, and Dynamics, Vol.8, No.5, pp. 620-627, 1985.
- [51] A. Benveniste and J. J. Fuchs, “Single sample modal identification of a nonstationary stochastic process,” IEEE Trans. on Automatic Control, Vol.30, No.1, pp. 66-74, 1985.
- [52] P. V. Overschee and B. D. Moor, “Subspace algorithm for the stochastic identification problem,” Automatica, Vol.29, No.3, pp. 649-660, 1993.
- [53] B. Peeters and G. D. Roeck, “Reference-based stochastic subspace identification for output-only modal analysis,” Mechanical Systems and Signal Processing, Vol.13, No.6, pp. 855-878, 1999.
- [54] G. Mercere, L. Bako, and S. Lecuche, “Propagator-based methods for recursive subspace model identification,” Signal Processing, Vol.88, No.3, pp. 468-491, 2008.
- [55] J. Munier and G. Y. Delisle, “Spatial analysis using new properties of the cross spectral matrix,” IEEE Trans. on Signal Processing, Vol.39, No.3. pp. 746-749, 1991.
- [56] B. Yang, “Projection approximation subspace tracking,” IEEE Trans. on Signal Processing, Vol.43, No.1, pp. 95-107, 1995.
- [57] T. Gustafsson, “Recursive system identification using instrumental variable subspace tracking,” Proc. of 11th IFAC Symposium on System Identification, 1997.
- [58] G. Mercere, S. Lecoeuche, and M. Lovera “Recursive subspace identification based on instrumental variable unconstrained quadratic optimization,” Int. J. of Adaptive Control and Signal Processing, Vol.18, No.9-10, pp. 771-797, 2004.

This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

Copyright© 2011 by Fuji Technology Press Ltd. and Japan Society of Mechanical Engineers. All right reserved.