Top > JACIII > Emotion Recognition Based on ECG Signals for Service Robots in ...

single-jc.php

JACIII Vol.15 No.5 pp. 582-591
doi: 10.20965/jaciii.2011.p0582
(2011)

Paper:

Views over last 60 days: 924

Emotion Recognition Based on ECG Signals for Service Robots in the Intelligent Space During Daily Life

Kanlaya Rattanyu* and Makoto Mizukawa**

*Graduate School of Functional Control Systems Engineering, Shibaura Institute of Technology, 3-7-5 Toyosu, Koto-ku, Tokyo 135-8548, Japan

**Department of Electrical Engineering, Shibaura Institute of Technology, 3-7-5 Toyosu, Koto-ku, Tokyo 135-8548, Japan

Received:
November 20, 2010
Accepted:
March 7, 2011
Published:
July 20, 2011
Creative Commons License
Keywords:
emotion recognition, intelligent space, ECG, ANOVA, LDA
Abstract
This paper presents our approach for emotion recognition based on Electrocardiogram (ECG) signals. We propose to use the ECG’s inter-beat features together with within-beat features in our recognition system. In order to reduce the feature space, post hoc tests in the Analysis of Variance (ANOVA) were employed to select the set of eleven most significant features. We conducted experiments on twelve subjects using the International Affective Picture System (IAPS) database. RF-ECG sensors were attached to the subject’s skin to monitor the ECG signal via wireless connection. Results showed that our eleven feature approach outperforms the conventional three feature approach. For simultaneous classification of six emotional states: anger, fear, disgust, sadness, neutral, and joy, the Correct Classification Ratio (CCR) showed significant improvement from 37.23% to over 61.44%. Our system was able to monitor human emotion wirelessly without affecting the subject’s activities. Therefore it is suitable to be integrated with service robots to provide assistive and healthcare services.
Cite this article as:
K. Rattanyu and M. Mizukawa, “Emotion Recognition Based on ECG Signals for Service Robots in the Intelligent Space During Daily Life,” J. Adv. Comput. Intell. Intell. Inform., Vol.15 No.5, pp. 582-591, 2011.
Data files:
References
  1. [1] R. B. Adler and R. Rodman, “Understanding Human Communication,” New York: Oxford University Press, Inc., 2006.
  2. [2] P. Jackson and I. Moulinier, “Natural Language Processing for Online Application,” John Benjamins Publishing Co., Amsterdam, 2002.
  3. [3] P. Ekman, R. W. Levenson, and W. V. Friesen, “Autonomic Nervous System Activity Distinguishes Among Emotions,” Science, New Series, Vol.221, No.4616, pp. 1208-1220, 1983.
  4. [4] C.-H. Yang, J.-L. Wang, K.-L. Lin, Y.-H. Kuo, and K.-S. Cheng, “Negative Emotion Detection Using the Heart Rate Recovery and Time for Twelve-Beats Heart Rate Decay After Exercise Stress Test,” In Int. Joint Conf. on Neural Networks (IJCNN), pp. 1-6. IEEE, 2010.
  5. [5] J. Thayer and G. Siegle, “Neurovisceral integration in cardiac and emotional regulation,” Engineering in Medicine and Biology Magazine, IEEE, Vol.21, No.4, pp. 24-29, 2002.
  6. [6] W. Wu and J. Lee, “Improvement of HRV Methodology for Positive/Negative Emotion Assessment,” In Collaborative Computing: Networking, Application and Worksharing 2009, CollaborateCom 2009, 5th Int. Conf. on, pp. 1-6. IEEE, 2009.
  7. [7] C. Lee and S. Yoo, “ECG-based Biofeedback Chair for Selfemotion Management at Home,” In Int. Conf. on Consumer Electronics 2008 (ICCE 2008), Digest of Technical Papers, pp. 1-2, IEEE, 2008.
  8. [8] J. D. Rodriguez and L. Santos, “Comparative Analysis Using the 80-Lead Body Surface Map and 12-Lead ECGWith Exercise Stress Echocardiograms,” J. of Diagnostic Medical Sonography, Vol.22, No.5, pp. 308-316, 2006.
  9. [9] S. R. Vrana, B. N. Cuthbert, and P. J. Lang, “Fear Imagery and text processing,” The Int. J. of the Society for Psychophysiological Research (Psychophysiology), Vol.23, No.3, pp. 247-253, 1986.
  10. [10] B. L. Fredrickson, R. A. Mancuso, C. Branigan, and M. M. Tugade, “The Undoing Effect of Positive Emotion,” Motivation and Emotion, Vol.24, No.4, pp. 237-258, 2000.
  11. [11] I. C. Christie, “Multivariate Discrimination of Emotion-Specific Autonomic Nervous System Activity,” Master’s thesis, Virginia Polytechnic Institute and State University, 2002.
  12. [12] G. Chanel, K. Ansari-Asl, and T. Pun, “Using Neural Network to Recognize Human Emotions from Heart Rate Variability and Skin Resistance,” In Int. Conf. of the IEEE-Engineering in Medicine and Biology Society (IEEE-EMBS), pp. 5523-5525, IEEE, 2005.
  13. [13] J. Kim and E. Andre, “Emotion Recognition Based on Physiological Changes in Music Listening,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.30, No.12, pp. 2067-2083, 2008.
  14. [14] P. Montoya, J. J. Campos, and R. Schandry, “See red? Turn pale? Unveiling Emotions Cardiovascular and Hemodynamic Change,” Spanish J. of Psychology, Vol.8, No.001, pp. 79-85, 2005.
  15. [15] R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and J. G. Taylor, “Emotion Recognition in Human-Computer Interaction,” IEEE Signal Processing Magazine, pp. 32-80, January 2001.
  16. [16] C. D. Katsis, N. Katertsidis, G. Ganiatsas, and D. I. Fotiadis, “Toward Emotion Recognition in Car-Racing Drivers: A Biosignal Processing Approach,” IEEE Trans. on System, Man, and Cybernetics-Part A: Systems and Humans, Vol.38, No.3, pp. 502-512, 2008.
  17. [17] R. W. Picard, “Toward Computers that Recognize and Respond to User Emotion,” IBM System Journal, Vol.39, No.3-4, pp. 705-719, 2000.
  18. [18] M. B. Ammar, M. Neji, and A. M. Alimi, “The Integration of an Emotional System in the Intelligent System,” In Computer Systems and Applications 2005, The 3rd ACS/IEEE Int. Conf. on. IEEE, 2005.
  19. [19] B. Takacs, “Special Education and Rehabilitation: Teaching and Healing with Interactive Graphics,” Computer Graphics and Applications, IEEE, Vol.25, No.5, pp. 40-48, 2005.
  20. [20] S. Cherry, “Anger Management: The next time you raise your voice to a phone-in service, you may well be heard by a computer,” IEEE Spectrum, p. 16, April 2005.
  21. [21] L. B. Theng and H. K. Shi, “A Mobile Real Time Interactive Communication Assistant for Cerebral Palsy,” Int. J. of Computing and ICT Research, Vol.3, No.2, pp. 31-40, 2009.
  22. [22] C. M. Lee and S. S. Narayanan, “Toward Detecting Emotions in Spoken Dialogs,” IEEE Trans. on Audio, Speech, and Audio Processing, Vol.13, No.2, p. 293, 2005.
  23. [23] N. D. Cook, T. X. Fujisawa, and K. Takami, “Evaluation of the Affective Valence of Speech Using Pitch Substructure,” IEEE Trans. on Audio, Speech, and Language Processing, Vol.14, No.1, pp. 142-151, 2006.
  24. [24] M. Pantic and L. Rothkrantz, “Toward an Affect-Sensitive Multimodal Human-Computer Interaction,” Proc. of IEEE, Vol.91, No.9, pp. 1370-1390, 2003.
  25. [25] C. Busso et al., “Analysis of emotion recognition using facial expressions, speech and multimodal information,” In Proc. of the 6th Int. Conf. on Multimodal Interfaces (ICMI), pp. 205-211, ACM, 2004.
  26. [26] R. Cowie et al., “An intelligent system for facial emotion recognition,” In IEEE Int. Conf. on Multimedia and Expo 2005 (ICME 2005), p. 4. IEEE, 2005.
  27. [27] I. Kotsia and I. Pitas, “Facial Expression Recognition in Image Sequences Using Geometric Deformation Features and Support Vector Machines,” IEEE Trans. on Image Processing, Vol.16, No.1, p. 172, 2007.
  28. [28] C.-T. Tu and J.-J. Lien, “Automatic Location of Facial Feature Points and Synthesis of Facial Sketches Using Direct Combined Model,” IEEE Trans. on Systems, Man, and Cybernetics, Part B: Cybernetics, Vol.40, No.4, pp. 1158-1169, 2010.
  29. [29] E. L. Pritchett, M. C. Schulte, D. Schnell, S. R. Marcello, W. E. Wilkinson, R. L. Page, and S. J. Connolly, “Effects of azimilide on heart rate and ECG conduction intervals during sinus rhythm in patients with a history of atrial fibrillation,” J. of Clinical Pharmacology, Vol.42, pp. 388-394, 2002.
  30. [30] N. P. Utama, A. Takemoto, K. Nakamura, and Y. Koike, “Singletrial EEG data to classify type and intensity of facial emotion from P100 and N170,” In Int. Joint Conf. on Neural Networks (IJCNN), pp. 3156-3163. IEEE, 2009.
  31. [31] J. Malmivuo and R. Plonsey, “Bioelectromagnetism – Principles and Applications of Bioelectric and Biomagnetic Fields,” Oxford University Press, Inc., New York, 1995.
  32. [32] P. J. Lang, M. M. Bradley, and B. N. Cuthbert, “International affective picture system (IAPS): Affective ratings of pictures and instruction manual,” Technical Report A-8, University of Florida, Gainesville, FL, 2008.
  33. [33] A. J. Izenman, “Modern Multivariate Statistical Techniques: Regression, Classification, and Manifold Learning,” Springer Netherlands, 1st edition, 2008.
  34. [34] P. S. Addison, “Wavelet Transforms and the ECG: a review,” Physiological Measurement, Vol.26, pp. 155-199, 2005.
  35. [35] W. Kanlaya, L. Dung, and M. Mizukawa, “Virtual Object for Evaluating Adaptable K-Nearest Neighbor Method Solving Various Conditions of Object Recognition,” In ICROS-SICE Int. Joint Conf. 2009 (ICCAS-SICE 2009), pp. 4338-4342, IEEE, 2009.

Creative Commons License  This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Apr. 22, 2024