single-rb.php

JRM Vol.35 No.5 pp. 1331-1339
doi: 10.20965/jrm.2023.p1331
(2023)

Paper:

Speech-Driven Avatar Robot System with Changing Complexion for the Visualization of an Interactive Atmosphere

Yoshihiro Sejima* ORCID Icon, Liheng Yang**, Saki Inagaki**, and Daiki Morita**

*Faculty of Informatics, Kansai University
2-1-1 Ryozenji-cho, Takatsuki, Osaka 569-1095, Japan

**Graduate School of Informatics, Kansai University
2-1-1 Ryozenji-cho, Takatsuki, Osaka 569-1095, Japan

Received:
January 24, 2023
Accepted:
July 3, 2023
Published:
October 20, 2023
Keywords:
nonverbal communication, human-robot interaction, interactive atmosphere, affective display
Abstract

Smooth interactions between talkers can be realized by transmitting and receiving mutual video images and voices in remote communication. However, in such remote communication, it is difficult to generate a sense of unity and an interactive atmosphere because humans recognize screens as a boundary of the physical space. Therefore, it is essential to develop a communication system that can generate and share an interactive atmosphere and interaction-activated communication even if talkers are in remote places. In this study, we developed a speech-driven avatar robot system incorporating an estimation model that simulates the degree of activated communication based on the talker’s speech. The developed avatar robot system can visualize an interactive atmosphere while changing the complexion based on an estimated value. The effectiveness of the developed system was demonstrated by means of sensory evaluations.

Talker can grasp the atmosphere from the complexion of the avatar robot during remote communication

Talker can grasp the atmosphere from the complexion of the avatar robot during remote communication

Cite this article as:
Y. Sejima, L. Yang, S. Inagaki, and D. Morita, “Speech-Driven Avatar Robot System with Changing Complexion for the Visualization of an Interactive Atmosphere,” J. Robot. Mechatron., Vol.35 No.5, pp. 1331-1339, 2023.
Data files:
References
  1. [1] S. Dash, S. Samadder, A. Srivastava, R. Meena, and P. Ranjan, “Review of online teaching platforms in the current period of COVID-19 pandemic,” Indian J. of Surgery, Vol.84, Suppl.1, pp. 12-17, 2022. https://doi.org/10.1007/s12262-021-02962-4
  2. [2] T. Watanabe, “Human-entrained embodied interaction and communication technology for human-connected IoT design,” J. of Advanced Mechanical Design, Systems, and Manufacturing, Vol.14, No.2, Article No.19-00322, 2020. https://doi.org/10.1299/jamdsm.2020jamdsm0025
  3. [3] T. Watanabe, “Human-entrained embodied interaction and communication technology,” S. Fukuda (Ed.), “Emotional Engineering: Service Development,” pp. 161-177, Springer, 2011. https://doi.org/10.1007/978-1-84996-423-4_9
  4. [4] Y. Sejima, T. Watanabe, and M. Jindai, “Estimation model of interaction-activated communication based on the heat conduction equation,” J. of Advanced Mechanical Design, Systems, and Manufacturing, Vol.10, No.9, Article No.15-00548, 2016. https://doi.org/10.1299/jamdsm.2016jamdsm0103
  5. [5] F. P. Incropera and D. P. DeWitt, “Introduction to Heat Transfer,” 4th Edition, John Wiley & Sons, 2002.
  6. [6] Y. Sejima, Y. Ishii, and T. Watanabe, “An embodied communication system with avatar-shadow’s color expressions based on an interaction-activated communication model in voice communication,” Trans. of the JSME, Vol.85, No.873, Article No.18-00074, 2019 (in Japanese). https://doi.org/10.1299/transjsme.18-00074
  7. [7] K. Takeuchi, Y. Yamazaki, and K. Yoshifuji, “Avatar work: Telework for disabled people unable to go outside by using avatar robots,” Companion of the 2020 ACM/IEEE Int. Conf. on Human-Robot Interaction (HRI’20), pp. 53-60, 2020. https://doi.org/10.1145/3371382.3380737
  8. [8] R. Yamazaki, H. Kase, S. Nishio, and H. Ishiguro, “Anxiety reduction through close communication with robotic media in dementia patients and healthy older adults,” J. of Robot. Mechatron., Vol.32, No.1, pp. 32-42, 2020. https://doi.org/10.20965/jrm.2020.p0032
  9. [9] T. Ito and T. Watanabe, “Natural involvement to video conference through ARM-COMS,” S. Yamamoto and H. Mori (Eds.), “Human Interface and the Management of Information: Applications in Complex Technological Environments,” pp. 238-246, Springer, 2022. https://doi.org/10.1007/978-3-031-06509-5_17
  10. [10] S. Yamamoto, J. Woo, W. H. Chin, K. Matsumura, and N. Kubota, “Interactive information support by robot partners based on informationally structured space,” J. Robot. Mechatron., Vol.32, No.1, pp. 236-243, 2020. https://doi.org/10.20965/jrm.2020.p0236
  11. [11] M. Shiomi, H. Sumioka, and H. Ishiguro, “Survey of social touch interaction between humans and robots,” J. Robot. Mechatron., Vol.32, No.1, pp. 128-135, 2020. https://doi.org/10.20965/jrm.2020.p0128
  12. [12] Y. Seong, “Design research of wearable soft avatar robot for interactive social presence,” J. Robot. Mechatron., Vol.34, No.2, pp. 325-327, 2022. https://doi.org/10.20965/jrm.2022.p0325
  13. [13] Y. Inoue and M. Kitazaki, “Virtual mirror and beyond: The psychological basis for avatar embodiment via a mirror,” J. Robot. Mechatron., Vol.33, No.5, pp. 1004-1012, 2021. https://doi.org/10.20965/jrm.2021.p1004
  14. [14] A. M. Soccini, A. Clocchiatti, and T. Inamura, “Effects of frequent changes in extended self-avatar movements on adaptation performance,” J. Robot. Mechatron., Vol.34, No.4, pp. 756-766, 2021. https://doi.org/10.20965/jrm.2022.p0756
  15. [15] K. Higashi, N. Isoyama, N. Sakata, and K. Kiyokawa, “Manipulating sense of participation in multipartite conversations by manipulating head attitude and gaze direction,” J. Robot. Mechatron., Vol.33, No.5, pp. 1013-1028, 2021. https://doi.org/10.20965/jrm.2021.p1013
  16. [16] J. N. Bailenson, “Nonverbal overload: A theoretical argument for the causes of Zoom fatigue,” Technology, Mind, and Behavior, Vol.2, No.1, 2021. https://doi.org/10.1037/tmb0000030
  17. [17] T. Obo and K. Takizawa, “Analysis of timing and effect of visual cue on turn-taking in human-robot interaction,” J. Robot. Mechatron., Vol.34, No.1, pp. 55-63, 2022. https://doi.org/10.20965/jrm.2022.p0055
  18. [18] L. Yang, S. Inagaki, D. Morita, and Y. Sejima, “Development of an avatar-robot with interactive atmosphere in online communication,” The Proc. of 2022 JSME Annual Conf. on Robotics and Mechatronics (Robomec), Session ID 2A1-H07, 2022 (in Japanese). https://doi.org/10.1299/jsmermd.2022.2A1-H07
  19. [19] V. M. Reid, K. Dunn, R. J. Young, J. Amu, T. Donovan, and N. Reissland, “The human fetus preferentially engages with face-like visual stimuli,” Current Biology, Vol.27, No.12, pp.1825-1828, 2017. https://doi.org/10.1016/j.cub.2017.05.044
  20. [20] Y. Sejima, S. Egawa, R. Maeda, Y. Sato, and T. Watanabe, “A speech-driven pupil response robot synchronized with burst-pause of utterance,” 2017 26th IEEE Int. Symp. on Robot and Human Interactive Communication (RO-MAN), pp. 437-442, 2017. https://doi.org/10.1109/ROMAN.2017.8172339
  21. [21] T. Watanabe, M. Ogikubo, and Y. Ishii, “Visualization of respiration in the embodied virtual communication system and its evaluation,” Int. J. of Human-Computer Interaction, Vol.17, No.1, pp. 89-102, 2004. https://doi.org/10.1207/s15327590ijhc1701_7
  22. [22] I. Inagaki, D. Morita, and Y. Sejima, “Development of a communication system that expresses interactive atmosphere with changing the background color using interaction-activated communication model,” Proc. of the Symp. on Human Interface 2022, pp. 533-537, 2022 (in Japanese).
  23. [23] Y. Sejima, Y. Sato, T. Watanabe, and M. Jindai, “Speech-driven embodied entrainment character system with pupillary response,” Mechanical Engineering J., Vol.3, No.4, Article No.15-00314, 2016. https://doi.org/10.1299/mej.15-00314
  24. [24] R. D. Luce, “Individual choice behaviour: A theoretical analysis,” J. Wiley, 1959.
  25. [25] P. Ekman, R. W. Levenson, and W. V. Friesen, “Autonomic nervous system activity distinguishes among emotions,” Science, Vol.221, No.4616, pp. 1208-1210, 1983. https://doi.org/10.1126/science.6612338
  26. [26] T. Yamada and T. Watanabe, “An average facial color image avatar system for the analysis by synthesis of affect display by dynamic facial color and expression,” Trans. of the Japan Society of Mechanical Engineers, Series C, Vol.78, No.791, pp. 2526-2535, 2012 (in Japanese). https://doi.org/10.1299/kikaic.78.2526
  27. [27] D. McDuff and E. M. Nowara, “‘Warm bodies’: A post-processing technique for animating dynamic blood flow on photos and avatars,” Proc. of the 2021 Conf. on Human Factors in Computing Systems (CHI’21), Article No.579, 2021. https://doi.org/10.1145/3411764.3445719
  28. [28] M. Hori, Y. Tsuruda, H. Yoshimura, and Y. Iwai, “Expression transmission using exaggerated animation for Elfoid,” Frontiers in Psychology, Vol.6, Article No.1219, 2015. https://doi.org/10.3389/fpsyg.2015.01219

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Jun. 03, 2024