Top > JRM > Gesture-World Environment Technology for Mobile Manipulation ...

single-rb.php

JRM Vol.24 No.1 pp. 180-190
doi: 10.20965/jrm.2012.p0180
(2012)

Paper:

Views over last 60 days: 940

Gesture-World Environment Technology for Mobile Manipulation – Remote Control System of a Robot with Hand Pose Estimation –

Kiyoshi Hoshino*, Takuya Kasahara*, Motomasa Tomida**,
and Takanobu Tanimoto*

*Graduate School of Systems and Information Engineering, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki 305-8573, Japan

**Crescent, Inc., 1-1-1 Tennodai, Tsukuba, Ibaraki 305-8573, Japan

Received:
May 7, 2011
Accepted:
August 31, 2011
Published:
February 20, 2012
Creative Commons License
Keywords:
3D hand pose estimation, two cameras installed at position of loosely orthogonal relationship, 3D shape reconstruction of a hand from a 2D image, remote control of a robot
Abstract
The purpose of this paper is to propose a remotecontrolled robot system capable of accurate highspeed performance of the same operation strictly conforming to human operator movement without sensors or special control means. We specifically intend to implement high-precision high-speed 3D hand pose estimation enabling a remote-controlled robot to be operated using two cameras installed loosely orthogonally using one ordinary PC. The two cameras have their own database. Once sequential hand images are shot at high speed, the system starts selecting one database with bigger size of hand region in each recorded image. Coarse screening then proceeds based on proportional hand image information roughly corresponding to wrist rotation or thumb or finger extension. Finally, a detailed search is done for similarity among selected candidates. Experiments show that mean and standard deviation scores of errors in estimated angles at the proximal interphalangeal (PIP) index are 0.45 ± 14.57 and at the carpometacarpal (CM) thumb 4.7 ± 10.82, respectively, indicating it as a high-precision 3D hand pose estimation. Remote control of a robot with the proposed vision system shows high performance as well.
Cite this article as:
K. Hoshino, T. Kasahara, M. Tomida, and T. Tanimoto, “Gesture-World Environment Technology for Mobile Manipulation – Remote Control System of a Robot with Hand Pose Estimation –,” J. Robot. Mechatron., Vol.24 No.1, pp. 180-190, 2012.
Data files:
References
  1. [1] J.M. Rehg and T. Kanade, “Visual tracking of high DOF articulated structures: an application to human hand tracking,” European Conf. Computer Vision, pp. 35-46, 1994.
  2. [2] M. H. Jeong, Y. Kuno, N. Shimada, and Y. Shirai, “Recognition of shape-changing hand gestures,” IEICE Trans. on Information and Systems, Vol.E85-D, pp. 1678-1687, 2002.
  3. [3] S. Lu, D. Metaxas, D. Samaras, and J. Oliensis, “Using multiple cues for hand tracking and model refinement,” Proc. CVPR2003, Vol.2, pp. 443-450, 2003.
  4. [4] E. Ueda, Y. Matsumoto, M. Imai, and T. Ogasawara, “Hand pose estimation for vision-based human interface,” IEEE Trans. on Industrial Electronics, Vol.50, No.4, pp. 676-684, 2003.
  5. [5] M. H. Jeong, Y. Kuno, N. Shimada, and Y. Shirai, “Recognition of two-hand gestures using coupled switching linear model,” IEICE Trans. on In-formation and Systems, Vol.E86-D, pp. 1416-1425, 2003.
  6. [6] T. Gumpp, P. Azad, K. Welke, E. Oztop, R. Dillmann, and G. Cheng, “Unconstrained real-time markerless hand tracking for humanoid interaction,” Proc. IEEE-RAS Int. Conf. on Humanoid Robots, CD-ROM, 2006.
  7. [7] V. Athitos and S. Scarloff, “An appearance-based framework for 3D hand shape classification and camera viewpoint estimation,” Proc. Automatic Face and Gesture Recognition, pp. 40-45, 2002.
  8. [8] K. Hoshino and T. Tanimoto, “Real time search for similar hand images from database for robotic hand control,” IEICE Trans. on Fundamentals of Electronics, Communications and Computer Sciences, Vol.E88-A, pp. 2514-2520, 2005.
  9. [9] Y.Wu, J. Lin, and T. S. Huang, “Analyzing and capturing articulated hand motion in image sequences,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.27, pp. 1910-1922, 2005.
  10. [10] K. Hoshino, E. Tamaki, and E. Tanimoto, “Copycat hand – Robot hand imitating human motions at high speed and with high accuracy,” Advanced Robotics, Vol.21, pp. 1743-1761, 2007.
  11. [11] K. Hoshino and T. Tanimoto, “Realtime hand posture estimation with Self-Organizing Map for stable robot control,” IEICE Trans. on Information and Systems, Vol.E89-D, No.6, pp. 1813-1819, 2006.
  12. [12] K. Hoshino and M. Tomida, “3D hand pose estimation using a single camera for unspecified users,” J. of Robotics and Mechatronics, Vol.21, No.6, pp. 749-757, 2009.
  13. [13] N. Otsu and T. Kurita, “A new scheme for practical, flexible and intelligent vision systems,” Proc. IAPR. Workshop on Computer Vision, pp. 431-435, 1998.
  14. [14]
    Supporting Online Materials:[a] Z. Zhang, “A Flexible New Technique for Camera Calibration,”
    http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.35.6725&rep=rep1&type=pdf
  15. [15] [b] Gesture-Driven Robot Arm System,
    http://www.youtube.com/watch?v=UjbZYN1Db14

Creative Commons License  This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

Copyright© 2012 by Fuji Technology Press Ltd. and Japan Society of Mechanical Engineers. All right reserved.

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Apr. 24, 2025