Human Detection and Face Recognition Using 3D Structure of Head and Face Surfaces Detected by RGB-D Sensor

Michio Tanaka; Hiroki Matsubara; Takashi Morie

doi:10.20965/jrm.2015.p0691

single-rb.php

« previous

JRM Vol.27 No.6 pp. 691-697

doi: 10.20965/jrm.2015.p0691

(2015)

Paper:

Views over last 60 days: 2,829

Human Detection and Face Recognition Using 3D Structure of Head and Face Surfaces Detected by RGB-D Sensor

Michio Tanaka, Hiroki Matsubara, and Takashi Morie

Kyushu Institute of Technology
2-4 Hibikino, Wakamatsu-ku, Kitakyushu 808-0196, Japan

Received:

May 20, 2015

Accepted:

September 2, 2015

Published:

December 20, 2015

Keywords:

home service robot, image processing, human detection and recognition, RGB-D sensor

Abstract

Summary of proposed method

Home service robots must possess the ability to communicate with humans, for which human detection and recognition methods are particularly important. This paper proposes methods for human detection and face recognition that are based on image processing, and are suitable for home service robots. For the human detection method, we combine the method proposed by Xia et al. based on the use of head shape with the results of region segmentation based on depth information, and use the positional relations of the detected points. We obtained a detection rate of 98.1% when the method was evaluated for various postures and facing directions. We demonstrate the robustness of the proposed method against postural changes such as stretching the arms, resting the chin on one’s hands, and drinking beverages. For the human recognition method, we combine the elastic bunch graph matching method proposed by Wiskott et al. with Face Tracking SDK to extract facial feature points, and use the 3D information in the deformation computation; we obtained a recognition rate of 93.6% during evaluation.

Cite this article as:

M. Tanaka, H. Matsubara, and T. Morie, “Human Detection and Face Recognition Using 3D Structure of Head and Face Surfaces Detected by RGB-D Sensor,” J. Robot. Mechatron., Vol.27 No.6, pp. 691-697, 2015.

Data files:

References

[1] N. Dalal, B. Triggs, and C. Schmid, “Human Detection Using Oriented Histograms of Flow and Appearance,” Proc. of the 9^th European Conf. on Computer Vision, Vol.2, pp. 428-441, 2006.
[2] H. C. Zeng, S. H. Huang, and S. H. Lai, “Real-Time Video Surveillance Based on Combining Foreground Extraction and Human Detection,” Advances in Multimedia Modeling Lecture Notes in Computer Science, Vol.4930, pp. 70-79, 2008.
[3] Y. Kanzawa, T. Naito, and Y. Kimura, “Human Skin Detection by Visible and Near-Infrared Imaging,” IAPR Conf. on Machine Vision Applications, pp. 503-507, 2011.
[4] S. Jia, J. Sheng, D. Chugo, and K. Takase, “Human Recognition Using RFID Technology and Stereo Vision,” J. Robotics and Mechatronics, Vol.21, No.1, pp. 28-35, 2009.
[5] T. Aoki, K. Ito, T. Shibahara, and S. Nagashima, “High-Accuracy Machine Vision Using Phase-Only Correlation – Toward Image Sensing Technology Breaking the Limit of Pixel Resolution –” IEICE ESS Fundamentals Review, Vol.1, No.1, pp. 30-40, 2007.
[6] K. Ito, A. Hayasaka, T. Aoki, H. Nakajimam, and K. Kobayashi, “A Study of Combination of 2D/3D Face Recognition Algorithms,” Symp. on Cryptography and Information Security, pp. 1-6, 2009.
[7] X. Wang, Q. Ruan, and Y. Ming, “3D Face Recognition Using Corresponding Point Direction Measure and Depth Local Features,” IEEE 10^th Int. Conf. on Signal Processing, pp. 86-89, 2010.
[8] K. C. Kwak, W. Pedrycz, H. J. Go, and M. G. Chun, “Fuzzy Aggregation Method Using Fisherface and Wavelet Decomposition for Face Recognition,” J. of Advanced Computational Intelligence and Intelligent Informatics, Vol.8, No.4, pp. 379-384, 2004.
[9] L. Xia, C. C. Chen, and J. K. Aggarwal, “Human Detection Using Depth Information by Kinect,” IEEE Computer Society Conf. on Computer Vision and Pattern Recognition Workshops, pp. 15-22, 2011.
[10] L. Wiskott, J. M. Fellous, N. Kuiger, and C. v. d. Malsburg, “Face Recognition by Elastic Bunch Graph Matching,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.19, No.7, pp. 775-779, 1997.
[11] J. G. Daugman, “Uncertainty Relation for Resolution in Space, Spatial Frequency, and Orientation Optimized by Two-Dimensional Visual Cortical Filters,” J. Optical Society of America A, Vol.2, No.7, pp. 1160-1169, 1985.
[12] T. Morie, J. Umezawa, and A. Iwata, “Gabor-Type Filtering Using Transient States of Cellular Neural Networks,” Intelligent Automation and Soft Computing, Vol.10, No.2, pp. 95-104, 2004.
[13] T. Morie, J. Umezawa, and A. Iwata, “A Pixel-Parallel Image Processor for Gabor Filtering Based on Merged Analog-Digital Architecture,” Symp. on VLSI Circuits, pp. 212-213, 2004.

This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

[1] [1] N. Dalal, B. Triggs, and C. Schmid, “Human Detection Using Oriented Histograms of Flow and Appearance,” Proc. of the 9^th European Conf. on Computer Vision, Vol.2, pp. 428-441, 2006.

[2] [2] H. C. Zeng, S. H. Huang, and S. H. Lai, “Real-Time Video Surveillance Based on Combining Foreground Extraction and Human Detection,” Advances in Multimedia Modeling Lecture Notes in Computer Science, Vol.4930, pp. 70-79, 2008.

[3] [3] Y. Kanzawa, T. Naito, and Y. Kimura, “Human Skin Detection by Visible and Near-Infrared Imaging,” IAPR Conf. on Machine Vision Applications, pp. 503-507, 2011.

[4] [4] S. Jia, J. Sheng, D. Chugo, and K. Takase, “Human Recognition Using RFID Technology and Stereo Vision,” J. Robotics and Mechatronics, Vol.21, No.1, pp. 28-35, 2009.

[5] [5] T. Aoki, K. Ito, T. Shibahara, and S. Nagashima, “High-Accuracy Machine Vision Using Phase-Only Correlation – Toward Image Sensing Technology Breaking the Limit of Pixel Resolution –” IEICE ESS Fundamentals Review, Vol.1, No.1, pp. 30-40, 2007.

[6] [6] K. Ito, A. Hayasaka, T. Aoki, H. Nakajimam, and K. Kobayashi, “A Study of Combination of 2D/3D Face Recognition Algorithms,” Symp. on Cryptography and Information Security, pp. 1-6, 2009.

[7] [7] X. Wang, Q. Ruan, and Y. Ming, “3D Face Recognition Using Corresponding Point Direction Measure and Depth Local Features,” IEEE 10^th Int. Conf. on Signal Processing, pp. 86-89, 2010.

[8] [8] K. C. Kwak, W. Pedrycz, H. J. Go, and M. G. Chun, “Fuzzy Aggregation Method Using Fisherface and Wavelet Decomposition for Face Recognition,” J. of Advanced Computational Intelligence and Intelligent Informatics, Vol.8, No.4, pp. 379-384, 2004.

[9] [9] L. Xia, C. C. Chen, and J. K. Aggarwal, “Human Detection Using Depth Information by Kinect,” IEEE Computer Society Conf. on Computer Vision and Pattern Recognition Workshops, pp. 15-22, 2011.

[10] [10] L. Wiskott, J. M. Fellous, N. Kuiger, and C. v. d. Malsburg, “Face Recognition by Elastic Bunch Graph Matching,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.19, No.7, pp. 775-779, 1997.

[11] [11] J. G. Daugman, “Uncertainty Relation for Resolution in Space, Spatial Frequency, and Orientation Optimized by Two-Dimensional Visual Cortical Filters,” J. Optical Society of America A, Vol.2, No.7, pp. 1160-1169, 1985.

[12] [12] T. Morie, J. Umezawa, and A. Iwata, “Gabor-Type Filtering Using Transient States of Cellular Neural Networks,” Intelligent Automation and Soft Computing, Vol.10, No.2, pp. 95-104, 2004.

[13] [13] T. Morie, J. Umezawa, and A. Iwata, “A Pixel-Parallel Image Processor for Gabor Filtering Based on Merged Analog-Digital Architecture,” Symp. on VLSI Circuits, pp. 212-213, 2004.