JRM Vol.27 No.6 pp. 691-697
doi: 10.20965/jrm.2015.p0691


Human Detection and Face Recognition Using 3D Structure of Head and Face Surfaces Detected by RGB-D Sensor

Michio Tanaka, Hiroki Matsubara, and Takashi Morie

Kyushu Institute of Technology
2-4 Hibikino, Wakamatsu-ku, Kitakyushu 808-0196, Japan

May 20, 2015
September 2, 2015
December 20, 2015
home service robot, image processing, human detection and recognition, RGB-D sensor
Summary of proposed method

Home service robots must possess the ability to communicate with humans, for which human detection and recognition methods are particularly important. This paper proposes methods for human detection and face recognition that are based on image processing, and are suitable for home service robots. For the human detection method, we combine the method proposed by Xia et al. based on the use of head shape with the results of region segmentation based on depth information, and use the positional relations of the detected points. We obtained a detection rate of 98.1% when the method was evaluated for various postures and facing directions. We demonstrate the robustness of the proposed method against postural changes such as stretching the arms, resting the chin on one’s hands, and drinking beverages. For the human recognition method, we combine the elastic bunch graph matching method proposed by Wiskott et al. with Face Tracking SDK to extract facial feature points, and use the 3D information in the deformation computation; we obtained a recognition rate of 93.6% during evaluation.

Cite this article as:
M. Tanaka, H. Matsubara, and T. Morie, “Human Detection and Face Recognition Using 3D Structure of Head and Face Surfaces Detected by RGB-D Sensor,” J. Robot. Mechatron., Vol.27, No.6, pp. 691-697, 2015.
Data files:
  1. [1] N. Dalal, B. Triggs, and C. Schmid, “Human Detection Using Oriented Histograms of Flow and Appearance,” Proc. of the 9th European Conf. on Computer Vision, Vol.2, pp. 428-441, 2006.
  2. [2] H. C. Zeng, S. H. Huang, and S. H. Lai, “Real-Time Video Surveillance Based on Combining Foreground Extraction and Human Detection,” Advances in Multimedia Modeling Lecture Notes in Computer Science, Vol.4930, pp. 70-79, 2008.
  3. [3] Y. Kanzawa, T. Naito, and Y. Kimura, “Human Skin Detection by Visible and Near-Infrared Imaging,” IAPR Conf. on Machine Vision Applications, pp. 503-507, 2011.
  4. [4] S. Jia, J. Sheng, D. Chugo, and K. Takase, “Human Recognition Using RFID Technology and Stereo Vision,” J. Robotics and Mechatronics, Vol.21, No.1, pp. 28-35, 2009.
  5. [5] T. Aoki, K. Ito, T. Shibahara, and S. Nagashima, “High-Accuracy Machine Vision Using Phase-Only Correlation – Toward Image Sensing Technology Breaking the Limit of Pixel Resolution –” IEICE ESS Fundamentals Review, Vol.1, No.1, pp. 30-40, 2007.
  6. [6] K. Ito, A. Hayasaka, T. Aoki, H. Nakajimam, and K. Kobayashi, “A Study of Combination of 2D/3D Face Recognition Algorithms,” Symp. on Cryptography and Information Security, pp. 1-6, 2009.
  7. [7] X. Wang, Q. Ruan, and Y. Ming, “3D Face Recognition Using Corresponding Point Direction Measure and Depth Local Features,” IEEE 10th Int. Conf. on Signal Processing, pp. 86-89, 2010.
  8. [8] K. C. Kwak, W. Pedrycz, H. J. Go, and M. G. Chun, “Fuzzy Aggregation Method Using Fisherface and Wavelet Decomposition for Face Recognition,” J. of Advanced Computational Intelligence and Intelligent Informatics, Vol.8, No.4, pp. 379-384, 2004.
  9. [9] L. Xia, C. C. Chen, and J. K. Aggarwal, “Human Detection Using Depth Information by Kinect,” IEEE Computer Society Conf. on Computer Vision and Pattern Recognition Workshops, pp. 15-22, 2011.
  10. [10] L. Wiskott, J. M. Fellous, N. Kuiger, and C. v. d. Malsburg, “Face Recognition by Elastic Bunch Graph Matching,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.19, No.7, pp. 775-779, 1997.
  11. [11] J. G. Daugman, “Uncertainty Relation for Resolution in Space, Spatial Frequency, and Orientation Optimized by Two-Dimensional Visual Cortical Filters,” J. Optical Society of America A, Vol.2, No.7, pp. 1160-1169, 1985.
  12. [12] T. Morie, J. Umezawa, and A. Iwata, “Gabor-Type Filtering Using Transient States of Cellular Neural Networks,” Intelligent Automation and Soft Computing, Vol.10, No.2, pp. 95-104, 2004.
  13. [13] T. Morie, J. Umezawa, and A. Iwata, “A Pixel-Parallel Image Processor for Gabor Filtering Based on Merged Analog-Digital Architecture,” Symp. on VLSI Circuits, pp. 212-213, 2004.

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, IE9,10,11, Opera.

Last updated on Nov. 12, 2018