Target Identification Through Human Pointing Gesture Based on Human-Adaptive Approach

Yusuke Tamura; Masao Sugi; Tamio Arai; Jun Ota

doi:10.20965/jrm.2008.p0515

single-rb.php

« previous

JRM Vol.20 No.4 pp. 515-525

doi: 10.20965/jrm.2008.p0515

(2008)

Paper:

Views over last 60 days: 1,595

Target Identification Through Human Pointing Gesture Based on Human-Adaptive Approach

Yusuke Tamura, Masao Sugi, Tamio Arai, and Jun Ota

Research into Artifact, Center for Engineering (RACE), The University of Tokyo, 5-1-5 Kashiwanoha, Kashiwa-shi, Chiba 277-8568, Japan

Received:

January 31, 2008

Accepted:

June 24, 2008

Published:

August 20, 2008

Keywords:

pointing, context, epistemic action, human-robot interface

Abstract

We propose a human-adaptive approach for calculating human pointing targets, integrating (1) calculating the user’s subjective pointing direction from finger direction, (2) integrating sensory information obtained from user pointing and contextual information such as user action sequences, and (3) arranging target candidates based on the user’s characteristics of pointing and action sequences. The user's subjective pointing direction is approximated by the linear function with the finger direction. Integration of sensory and contextual information using a probabilistic model enables the system to calculate the target accurately. Using a force-directed approach, we obtained good placement in which false estimations are decreased and not moved much from initial placement. Experimental results demonstrate the usefulness of our proposal.

Cite this article as:

Y. Tamura, M. Sugi, T. Arai, and J. Ota, “Target Identification Through Human Pointing Gesture Based on Human-Adaptive Approach,” J. Robot. Mechatron., Vol.20 No.4, pp. 515-525, 2008.

Data files:

References

[1] T. Sato, Y. Nishida, and H. Mizoguchi, “Robotic room: symbiosis with human through behavior media,” Robotics and Autonomous Systems, Vol.18, pp. 185-194, 1996.
[2] R. A. Brooks, “The intelligent room project,” Proc. of the 2nd Int. Cognitive Technology Conf., pp. 271-278, 1997.
[3] M. Sugi, Y. Tamura, J. Ota, T. Arai, K. Takamasu, K. Kotani, H. Suzuki, and Y. Sato, “Attentive Workbench: an intelligent production cell supporting human workers,” 7th Int. Symposium on Distributed Autonomous Robotic Systems, pp. 441-450, 2004.
[4] K. Oka, Y. Sato, and H. Koike, “Real-time fingertip tracking and gesture recognition,” IEEE Computer Graphics and Applications, Vol.22, No.6, pp. 64-71, 2002.
[5] B. A. Sawyer, “Magnetic positioning device,” US patent, 3,457,482, 1969.
[6] K. Kotani, I. Hidaka, Y. Yamamoto, and S. Ozono, “Analysis of respiratory sinus arrhythmia with respect to respiratory phase,” Method of Information in Medicine, Vol.39, pp. 153-156, 2000.
[7] R. A. Bolt, “ “Put-that-there”: voice and gesture at the graphics interface,” Proc. of the 7th Annual Conf. on Computer Graphics and Interactive Techniques, pp. 262-270, 1980.
[8] K. Tsukada and M. Yasumura, “Ubi-Finger: gesture input device for mobile use,” Proc. of the 5th Asia-Pacific Conf. on Computer-Human Interaction, Vol.1, pp. 388-400, 2002.
[9] R. Cipolla and N. J. Hollinghurst, “Human-robot interface by pointing with uncalibrated stereo vision,” Image and Vision Computing, Vol.14, pp. 171-178, 1996.
[10] S. Sato and S. Sakane, “A human-robot interface using an interactive hand pointer that projects a mark in the real work space,” Proc. of the 2000 IEEE Int. Conf. on Robotics and Automation, pp. 589-595, 2000.
[11] R. E. Kahn, M. J. Swain, P. N. Prokopowicz, and R. J. Firby, “Gesture recognition using the perseus architecture,” Proc. of the 1996 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, pp. 734-741, 1996.
[12] M. S. Lee, D. Weinshall, E. Cohen-Solal, A. Colmenarez, and D. Lyons, “A computer vision system for on-screen item selection by finger pointing,” Proc. of the 2001 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, Vol.1, pp. 1026-1033, 2001.
[13] M. Hild, M. Hashimoto, and K. Yoshida, “Object recognition via recognition of finger pointing actions,” Proc. of the 12th Int. Conf. on Image Analysis and Processing, pp. 88-93, 2003.
[14] C. Colombo, A. D. Bimbo, and A. Valli, “Visual capture and understanding of hand pointing actions in a 3-D environment,” IEEE Transactions on Systems, Man, and Cybernetics – Part B: Cybernetics, Vol.33, No.4, pp. 677-686, 2003.
[15] M. Fukumoto, Y. Suenaga, and K. Mase, “ “Finger-pointer”: pointing interface by image processing,” Computers & Graphics, Vol.18, No.5, pp. 633-642, 1994.
[16] T. Mashita, Y. Iwai, and M. Yachida, “Detecting indicated object from head-mouted omnidirectional images by pointing gesture,” Proc. of SICE Annual Conf., pp. 3230-3235, 2003.
[17] B. A. Po, B. D. Fisher, and K. S. Booth, “Pointing and visual feedback for spatial interaction in large-screen display environments,” Proc. of the 3rd Int. Symposium on Smart Graphics 2003, pp. 22-38, 2003.
[18] T. Miyasato and F. Kishino, “An Evaluation of Virtual Space Teleconferencing System Based on Detection of Objects Pointed through a Virtual Space,” The IEICE Transactions on Information and Systems, Vol.J80-D-II, No.5, pp. 1221-1230, 1997 (in Japanese).
[19] T. Imai, D. Sekiguchi, N. Kawakami, and S. Tachi, “Measuring Accuracy of Nonverbal Information Perception of Humans: Measurement of Pointing Gesture Perception,” Transactions of the Virtual Reality Society of Japan, Vol.9, No.1, pp. 89-96, 2004 (in Japanese).
[20] O. Sugiyama, T. Kanda, M. Imai, H. Ishiguro, and N. Hagita, “Three-layered draw-attention model for humanoid robots with gestures and verbal cues,” Proc. of the 2005 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 2140-2145, 2005.
[21] E. Gumbel, J. Greenwood, and D. Durand, “The circular normal distribution: theory and tables,” Journal of the American Statistical Association, Vol.48, No.261, pp. 131-152, 1953.
[22] T. Kamada and S. Kawai, “An algorithm for drawing general undirected graphs,” Information Processing Letters, Vol.31, No.1, pp. 7-15, 1989.
[23] T. M. Fruchterman and E. M. Reingold, “Graph drawing by forcedirected placement,” Software – Practice and Experience, Vol.21, pp. 1129-1161, 1991.
[24] N. R. Quinn and M. A. Breuer, “A forced directed component placement procedure for printed circuit boards,” IEEE Transactions on Circuits and Systems, Vol.26, No.6, pp. 377-388, 1979.
[25] D. Kirsh, “The intelligent use of space,” Artificial Intelligence, Vol.73, pp. 31-68, 1995.
[26] P. Maglio and D. Kirsh, “Epistemic action increases with skill,” Proc. of the 18th Annual Conf. of the Cognitive Science Society, pp. 391-396, 1996.

This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

[1] [1] T. Sato, Y. Nishida, and H. Mizoguchi, “Robotic room: symbiosis with human through behavior media,” Robotics and Autonomous Systems, Vol.18, pp. 185-194, 1996.

[2] [2] R. A. Brooks, “The intelligent room project,” Proc. of the 2nd Int. Cognitive Technology Conf., pp. 271-278, 1997.

[3] [3] M. Sugi, Y. Tamura, J. Ota, T. Arai, K. Takamasu, K. Kotani, H. Suzuki, and Y. Sato, “Attentive Workbench: an intelligent production cell supporting human workers,” 7th Int. Symposium on Distributed Autonomous Robotic Systems, pp. 441-450, 2004.

[4] [4] K. Oka, Y. Sato, and H. Koike, “Real-time fingertip tracking and gesture recognition,” IEEE Computer Graphics and Applications, Vol.22, No.6, pp. 64-71, 2002.

[5] [5] B. A. Sawyer, “Magnetic positioning device,” US patent, 3,457,482, 1969.

[6] [6] K. Kotani, I. Hidaka, Y. Yamamoto, and S. Ozono, “Analysis of respiratory sinus arrhythmia with respect to respiratory phase,” Method of Information in Medicine, Vol.39, pp. 153-156, 2000.

[7] [7] R. A. Bolt, “ “Put-that-there”: voice and gesture at the graphics interface,” Proc. of the 7th Annual Conf. on Computer Graphics and Interactive Techniques, pp. 262-270, 1980.

[8] [8] K. Tsukada and M. Yasumura, “Ubi-Finger: gesture input device for mobile use,” Proc. of the 5th Asia-Pacific Conf. on Computer-Human Interaction, Vol.1, pp. 388-400, 2002.

[9] [9] R. Cipolla and N. J. Hollinghurst, “Human-robot interface by pointing with uncalibrated stereo vision,” Image and Vision Computing, Vol.14, pp. 171-178, 1996.

[10] [10] S. Sato and S. Sakane, “A human-robot interface using an interactive hand pointer that projects a mark in the real work space,” Proc. of the 2000 IEEE Int. Conf. on Robotics and Automation, pp. 589-595, 2000.

[11] [11] R. E. Kahn, M. J. Swain, P. N. Prokopowicz, and R. J. Firby, “Gesture recognition using the perseus architecture,” Proc. of the 1996 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, pp. 734-741, 1996.

[12] [12] M. S. Lee, D. Weinshall, E. Cohen-Solal, A. Colmenarez, and D. Lyons, “A computer vision system for on-screen item selection by finger pointing,” Proc. of the 2001 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, Vol.1, pp. 1026-1033, 2001.

[13] [13] M. Hild, M. Hashimoto, and K. Yoshida, “Object recognition via recognition of finger pointing actions,” Proc. of the 12th Int. Conf. on Image Analysis and Processing, pp. 88-93, 2003.

[14] [14] C. Colombo, A. D. Bimbo, and A. Valli, “Visual capture and understanding of hand pointing actions in a 3-D environment,” IEEE Transactions on Systems, Man, and Cybernetics – Part B: Cybernetics, Vol.33, No.4, pp. 677-686, 2003.

[15] [15] M. Fukumoto, Y. Suenaga, and K. Mase, “ “Finger-pointer”: pointing interface by image processing,” Computers & Graphics, Vol.18, No.5, pp. 633-642, 1994.

[16] [16] T. Mashita, Y. Iwai, and M. Yachida, “Detecting indicated object from head-mouted omnidirectional images by pointing gesture,” Proc. of SICE Annual Conf., pp. 3230-3235, 2003.

[17] [17] B. A. Po, B. D. Fisher, and K. S. Booth, “Pointing and visual feedback for spatial interaction in large-screen display environments,” Proc. of the 3rd Int. Symposium on Smart Graphics 2003, pp. 22-38, 2003.

[18] [18] T. Miyasato and F. Kishino, “An Evaluation of Virtual Space Teleconferencing System Based on Detection of Objects Pointed through a Virtual Space,” The IEICE Transactions on Information and Systems, Vol.J80-D-II, No.5, pp. 1221-1230, 1997 (in Japanese).

[19] [19] T. Imai, D. Sekiguchi, N. Kawakami, and S. Tachi, “Measuring Accuracy of Nonverbal Information Perception of Humans: Measurement of Pointing Gesture Perception,” Transactions of the Virtual Reality Society of Japan, Vol.9, No.1, pp. 89-96, 2004 (in Japanese).

[20] [20] O. Sugiyama, T. Kanda, M. Imai, H. Ishiguro, and N. Hagita, “Three-layered draw-attention model for humanoid robots with gestures and verbal cues,” Proc. of the 2005 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 2140-2145, 2005.

[21] [21] E. Gumbel, J. Greenwood, and D. Durand, “The circular normal distribution: theory and tables,” Journal of the American Statistical Association, Vol.48, No.261, pp. 131-152, 1953.

[22] [22] T. Kamada and S. Kawai, “An algorithm for drawing general undirected graphs,” Information Processing Letters, Vol.31, No.1, pp. 7-15, 1989.

[23] [23] T. M. Fruchterman and E. M. Reingold, “Graph drawing by forcedirected placement,” Software – Practice and Experience, Vol.21, pp. 1129-1161, 1991.

[24] [24] N. R. Quinn and M. A. Breuer, “A forced directed component placement procedure for printed circuit boards,” IEEE Transactions on Circuits and Systems, Vol.26, No.6, pp. 377-388, 1979.

[25] [25] D. Kirsh, “The intelligent use of space,” Artificial Intelligence, Vol.73, pp. 31-68, 1995.

[26] [26] P. Maglio and D. Kirsh, “Epistemic action increases with skill,” Proc. of the 18th Annual Conf. of the Cognitive Science Society, pp. 391-396, 1996.