JRM Vol.20 No.4 pp. 515-525
doi: 10.20965/jrm.2008.p0515


Target Identification Through Human Pointing Gesture Based on Human-Adaptive Approach

Yusuke Tamura, Masao Sugi, Tamio Arai, and Jun Ota

Research into Artifact, Center for Engineering (RACE), The University of Tokyo, 5-1-5 Kashiwanoha, Kashiwa-shi, Chiba 277-8568, Japan

January 31, 2008
June 24, 2008
August 20, 2008
pointing, context, epistemic action, human-robot interface

We propose a human-adaptive approach for calculating human pointing targets, integrating (1) calculating the user’s subjective pointing direction from finger direction, (2) integrating sensory information obtained from user pointing and contextual information such as user action sequences, and (3) arranging target candidates based on the user’s characteristics of pointing and action sequences. The user’s subjective pointing direction is approximated by the linear function with the finger direction. Integration of sensory and contextual information using a probabilistic model enables the system to calculate the target accurately. Using a force-directed approach, we obtained good placement in which false estimations are decreased and not moved much from initial placement. Experimental results demonstrate the usefulness of our proposal.

  1. [1] T. Sato, Y. Nishida, and H. Mizoguchi, “Robotic room: symbiosis with human through behavior media,” Robotics and Autonomous Systems, Vol.18, pp. 185-194, 1996.
  2. [2] R. A. Brooks, “The intelligent room project,” Proc. of the 2nd Int. Cognitive Technology Conf., pp. 271-278, 1997.
  3. [3] M. Sugi, Y. Tamura, J. Ota, T. Arai, K. Takamasu, K. Kotani, H. Suzuki, and Y. Sato, “Attentive Workbench: an intelligent production cell supporting human workers,” 7th Int. Symposium on Distributed Autonomous Robotic Systems, pp. 441-450, 2004.
  4. [4] K. Oka, Y. Sato, and H. Koike, “Real-time fingertip tracking and gesture recognition,” IEEE Computer Graphics and Applications, Vol.22, No.6, pp. 64-71, 2002.
  5. [5] B. A. Sawyer, “Magnetic positioning device,” US patent, 3,457,482, 1969.
  6. [6] K. Kotani, I. Hidaka, Y. Yamamoto, and S. Ozono, “Analysis of respiratory sinus arrhythmia with respect to respiratory phase,” Method of Information in Medicine, Vol.39, pp. 153-156, 2000.
  7. [7] R. A. Bolt, “ “Put-that-there”: voice and gesture at the graphics interface,” Proc. of the 7th Annual Conf. on Computer Graphics and Interactive Techniques, pp. 262-270, 1980.
  8. [8] K. Tsukada and M. Yasumura, “Ubi-Finger: gesture input device for mobile use,” Proc. of the 5th Asia-Pacific Conf. on Computer-Human Interaction, Vol.1, pp. 388-400, 2002.
  9. [9] R. Cipolla and N. J. Hollinghurst, “Human-robot interface by pointing with uncalibrated stereo vision,” Image and Vision Computing, Vol.14, pp. 171-178, 1996.
  10. [10] S. Sato and S. Sakane, “A human-robot interface using an interactive hand pointer that projects a mark in the real work space,” Proc. of the 2000 IEEE Int. Conf. on Robotics and Automation, pp. 589-595, 2000.
  11. [11] R. E. Kahn, M. J. Swain, P. N. Prokopowicz, and R. J. Firby, “Gesture recognition using the perseus architecture,” Proc. of the 1996 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, pp. 734-741, 1996.
  12. [12] M. S. Lee, D. Weinshall, E. Cohen-Solal, A. Colmenarez, and D. Lyons, “A computer vision system for on-screen item selection by finger pointing,” Proc. of the 2001 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, Vol.1, pp. 1026-1033, 2001.
  13. [13] M. Hild, M. Hashimoto, and K. Yoshida, “Object recognition via recognition of finger pointing actions,” Proc. of the 12th Int. Conf. on Image Analysis and Processing, pp. 88-93, 2003.
  14. [14] C. Colombo, A. D. Bimbo, and A. Valli, “Visual capture and understanding of hand pointing actions in a 3-D environment,” IEEE Transactions on Systems, Man, and Cybernetics – Part B: Cybernetics, Vol.33, No.4, pp. 677-686, 2003.
  15. [15] M. Fukumoto, Y. Suenaga, and K. Mase, “ “Finger-pointer”: pointing interface by image processing,” Computers & Graphics, Vol.18, No.5, pp. 633-642, 1994.
  16. [16] T. Mashita, Y. Iwai, and M. Yachida, “Detecting indicated object from head-mouted omnidirectional images by pointing gesture,” Proc. of SICE Annual Conf., pp. 3230-3235, 2003.
  17. [17] B. A. Po, B. D. Fisher, and K. S. Booth, “Pointing and visual feedback for spatial interaction in large-screen display environments,” Proc. of the 3rd Int. Symposium on Smart Graphics 2003, pp. 22-38, 2003.
  18. [18] T. Miyasato and F. Kishino, “An Evaluation of Virtual Space Teleconferencing System Based on Detection of Objects Pointed through a Virtual Space,” The IEICE Transactions on Information and Systems, Vol.J80-D-II, No.5, pp. 1221-1230, 1997 (in Japanese).
  19. [19] T. Imai, D. Sekiguchi, N. Kawakami, and S. Tachi, “Measuring Accuracy of Nonverbal Information Perception of Humans: Measurement of Pointing Gesture Perception,” Transactions of the Virtual Reality Society of Japan, Vol.9, No.1, pp. 89-96, 2004 (in Japanese).
  20. [20] O. Sugiyama, T. Kanda, M. Imai, H. Ishiguro, and N. Hagita, “Three-layered draw-attention model for humanoid robots with gestures and verbal cues,” Proc. of the 2005 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 2140-2145, 2005.
  21. [21] E. Gumbel, J. Greenwood, and D. Durand, “The circular normal distribution: theory and tables,” Journal of the American Statistical Association, Vol.48, No.261, pp. 131-152, 1953.
  22. [22] T. Kamada and S. Kawai, “An algorithm for drawing general undirected graphs,” Information Processing Letters, Vol.31, No.1, pp. 7-15, 1989.
  23. [23] T. M. Fruchterman and E. M. Reingold, “Graph drawing by forcedirected placement,” Software – Practice and Experience, Vol.21, pp. 1129-1161, 1991.
  24. [24] N. R. Quinn and M. A. Breuer, “A forced directed component placement procedure for printed circuit boards,” IEEE Transactions on Circuits and Systems, Vol.26, No.6, pp. 377-388, 1979.
  25. [25] D. Kirsh, “The intelligent use of space,” Artificial Intelligence, Vol.73, pp. 31-68, 1995.
  26. [26] P. Maglio and D. Kirsh, “Epistemic action increases with skill,” Proc. of the 18th Annual Conf. of the Cognitive Science Society, pp. 391-396, 1996.

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, IE9,10,11, Opera.

Last updated on Feb. 21, 2018