Paper:

# Online Estimation of Image Jacobian Matrix with Time-Delay Compensation

## Xinmei Wang^{*}, Wu Wei^{**}, Feng Liu^{*}, Longsheng Wei^{*}, and Zhihui Liu^{***}

^{*}School of Automation, China University of Geosciences

Wuhan, Hubei 430074, China

^{**}College of Automation Science and Engineering, South China University of Technology

Guangzhou, Guangdong 510640, China

^{***}School of Mathematics and Physics, China University of Geosciences

Wuhan, Hubei 430074, China

Time delay exists in image-based visual servoing system. To compensate for the impact of time delay, the feature point image and image Jacobian matrix with time-delay compensation is discussed in this paper. Firstly, the current position and velocity estimation of the feature point in the image space is based on Kalman filtering, but Markov chain model is applied in the description of the measurement noise, then the cross-correlation between the process noise and measurement noise is produced, the traditional Kalman filtering algorithm is restricted, by introducing a filtering revision matrix, the process equation and measurement equation are redefined, under the mathematical properties of the noise in Kalman filtering algorithm, the filtering revision matrix can be obtained for the elimination of the cross-correlation, a robust Kalman filtering model can be constructed. Secondly, for the measurement vectors which cannot be obtained during time delay in the robust Kalman filtering model, a polynomial fitting method is proposed in which the selection of the polynomial includes the position, the velocity and the acceleration of the feature point which impact the feature point trajectory. Finally, from the current predicted position and velocity of the feature point in the image space, the current accurate image Jacobian matrix with time-delay compensation can be obtained. Simulation and experimental results verify the feasibility and superiority of this method.

- [1] J. A. Piepmeier and H. Lipkin, “Uncalibrated eye-in-hand visual servoing,” Int. J. of Robotics Research, Vol.22, No.10, pp. 805-819, 2003.
- [2] A. Shademan, A. M. Farahmand, and M. Jagersand, “Robust Jacobian estimation for uncalibrated visual servoing,” 2010 IEEE Int. Conf. on Robotics & Automation, pp. 5564-5569, 2010.
- [3] X. J. Zeng, X. H. Huang, and M. Wang, “Vision servoing based on online estimation of image Jacobian matrix of Broyden,” J. of Huazhong University of Science and Technology (Natural Science Ed.), Vol.36, No.9, pp. 17-20, 2008.
- [4] S. Kumar and L. Beharal, “Implementation of a neural network based visual motor control algorithm for A 7 DOF redundant manipulator,” IEEE Int. Joint Conf. on Neural Networks, pp. 1344-1351, 2008.
- [5] Y. X. Li, Z. Y. Mao, and L. F.Tian, “Visual servoing of 4DOF-robot using image moments and neural network,” Control Theory & Application, Vol.26, No.10, pp. 1162-1166, 2009.
- [6] A. M. Farahmand, A. Shademan, and M. Jagersand, “Global visual-motor estimation for uncalibrated visual servoing,” IEEE/ RSJ Int. Conf. on Intelligent Robots and Systems, pp. 1969-1974, 2007.
- [7] M. Zoran, M. Marko, L. Mihailo, B. Bojan, “Neural network reinforcement learning for visual control of robot manipulators,” Expert Systems with Applications, Vol.40, No.5, pp. 1721-1736, 2013.
- [8] D. I. Kosmopoulos, “Robust Jacobian matrix estimation for image-based visual servoing,” Robotics and Computer-Integrated Manufacturing, Vol.27, No.1, pp. 82-87, 2011.
- [9] J. Qian and J. Su, “Online estimation of image Jacobian matrix by Kalman-Bucy filter for uncalibrated stereo vision feedback,” Proc. of the 2002 IEEE Int. Conf. on Robotics and Automation, pp. 562-567, 2002.
- [10] F. Janabi-Sharifi and M. Marey, “A Kalman-filter-based method for pose estimation in visual servoing,” IEEE Trans. on Robotics, Vol.26, No.5, pp. 939-947, 2010.
- [11] V. Vaidehi, N. Chitra, M. Chokkalingam, and C. N. Krishnan, “Neural network aided Kalman filtering for multitarget tracking applications,” Computers & Electrical Engineering, Vol.27, No.3, pp. 217-228, 2001.
- [12] J. Zhang and D. Liu, “Online estimation of image Jacobian Matrix based on robust information filter,” J. of Xi’an University of Science and Technology, Vol.27, No.2, pp. 133-138, 2011.
- [13] H. X. Li, Y. H. Shi, and G. R.Wang, “Visual guidance of welding robot using SVR-Jacobian Estimator,” J. of South China University of Technology (Natural Science Ed.), Vol.41, No.7, pp. 19-25, 2013.
- [14] X. G. Zhong, X. Y. Zhong, and X. F. Peng, “Robots visual servo control with features constraint employing Kalman-neural-network filtering scheme,” Neurocomputing, Vol.151, No.1, pp. 268-277, 2015.
- [15] D. Nishio, M. Nakamura, S. Komada, and J. Hirai, “Tracking of moving object by manipulator using estimated image feature and its error correction on image planes,” The 8
^{th}IEEE Int. Workshop on Advanced Motion Control, pp. 653-657, 2004. - [16] M. Nakadokoro, S. Komada, and T. Hori, “Stereo visual servo of robot manipulators by estimated image features without 3D reconstruction,” IEEE Conf. on Systems, Man and Cybernetics, pp. 571-576, 1999.
- [17] Z. D. Gao and J. B. Su, “The estimation of image Jacobian matrix with time-delay compensation for uncalibrated visual servoing,” Control Theory & Applications, Vol.26, No.1, pp. 23-27, 2009.
- [18] W. F. Liu, Z. G. Bing, S. L. Lu, and H. L. Lu, “Online estimation of image Jacobian matrix with time-delay compensation,” Computer Engineering and Applications, Vol.46, No.21, pp. 181-184, 2010.