Smooth interactions between talkers can be realized by transmitting and receiving mutual video images and voices in remote communication. However, in such remote communication, it is difficult to generate a sense of unity and an interactive atmosphere because humans recognize screens as a boundary of the physical space. Therefore, it is essential to develop a communication system that can generate and share an interactive atmosphere and interaction-activated communication even if talkers are in remote places. In this study, we developed a speech-driven avatar robot system incorporating an estimation model that simulates the degree of activated communication based on the talker’s speech. The developed avatar robot system can visualize an interactive atmosphere while changing the complexion based on an estimated value. The effectiveness of the developed system was demonstrated by means of sensory evaluations.

1. [1] S. Dash, S. Samadder, A. Srivastava, R. Meena, and P. Ranjan, “Review of online teaching platforms in the current period of COVID-19 pandemic,” Indian J. of Surgery, Vol.84, Suppl.1, pp. 12-17, 2022. https://doi.org/10.1007/s12262-021-02962-4
2. [2] T. Watanabe, “Human-entrained embodied interaction and communication technology for human-connected IoT design,” J. of Advanced Mechanical Design, Systems, and Manufacturing, Vol.14, No.2, Article No.19-00322, 2020. https://doi.org/10.1299/jamdsm.2020jamdsm0025
3. [3] T. Watanabe, “Human-entrained embodied interaction and communication technology,” S. Fukuda (Ed.), “Emotional Engineering: Service Development,” pp. 161-177, Springer, 2011. https://doi.org/10.1007/978-1-84996-423-4_9
4. [4] Y. Sejima, T. Watanabe, and M. Jindai, “Estimation model of interaction-activated communication based on the heat conduction equation,” J. of Advanced Mechanical Design, Systems, and Manufacturing, Vol.10, No.9, Article No.15-00548, 2016. https://doi.org/10.1299/jamdsm.2016jamdsm0103
5. [5] F. P. Incropera and D. P. DeWitt, “Introduction to Heat Transfer,” 4th Edition, John Wiley & Sons, 2002.
6. [6] Y. Sejima, Y. Ishii, and T. Watanabe, “An embodied communication system with avatar-shadow’s color expressions based on an interaction-activated communication model in voice communication,” Trans. of the JSME, Vol.85, No.873, Article No.18-00074, 2019 (in Japanese). https://doi.org/10.1299/transjsme.18-00074
7. [7] K. Takeuchi, Y. Yamazaki, and K. Yoshifuji, “Avatar work: Telework for disabled people unable to go outside by using avatar robots,” Companion of the 2020 ACM/IEEE Int. Conf. on Human-Robot Interaction (HRI’20), pp. 53-60, 2020. https://doi.org/10.1145/3371382.3380737
8. [8] R. Yamazaki, H. Kase, S. Nishio, and H. Ishiguro, “Anxiety reduction through close communication with robotic media in dementia patients and healthy older adults,” J. of Robot. Mechatron., Vol.32, No.1, pp. 32-42, 2020. https://doi.org/10.20965/jrm.2020.p0032
9. [9] T. Ito and T. Watanabe, “Natural involvement to video conference through ARM-COMS,” S. Yamamoto and H. Mori (Eds.), “Human Interface and the Management of Information: Applications in Complex Technological Environments,” pp. 238-246, Springer, 2022. https://doi.org/10.1007/978-3-031-06509-5_17
10. [10] S. Yamamoto, J. Woo, W. H. Chin, K. Matsumura, and N. Kubota, “Interactive information support by robot partners based on informationally structured space,” J. Robot. Mechatron., Vol.32, No.1, pp. 236-243, 2020. https://doi.org/10.20965/jrm.2020.p0236
11. [11] M. Shiomi, H. Sumioka, and H. Ishiguro, “Survey of social touch interaction between humans and robots,” J. Robot. Mechatron., Vol.32, No.1, pp. 128-135, 2020. https://doi.org/10.20965/jrm.2020.p0128
12. [12] Y. Seong, “Design research of wearable soft avatar robot for interactive social presence,” J. Robot. Mechatron., Vol.34, No.2, pp. 325-327, 2022. https://doi.org/10.20965/jrm.2022.p0325
13. [13] Y. Inoue and M. Kitazaki, “Virtual mirror and beyond: The psychological basis for avatar embodiment via a mirror,” J. Robot. Mechatron., Vol.33, No.5, pp. 1004-1012, 2021. https://doi.org/10.20965/jrm.2021.p1004
14. [14] A. M. Soccini, A. Clocchiatti, and T. Inamura, “Effects of frequent changes in extended self-avatar movements on adaptation performance,” J. Robot. Mechatron., Vol.34, No.4, pp. 756-766, 2021. https://doi.org/10.20965/jrm.2022.p0756
15. [15] K. Higashi, N. Isoyama, N. Sakata, and K. Kiyokawa, “Manipulating sense of participation in multipartite conversations by manipulating head attitude and gaze direction,” J. Robot. Mechatron., Vol.33, No.5, pp. 1013-1028, 2021. https://doi.org/10.20965/jrm.2021.p1013
16. [16] J. N. Bailenson, “Nonverbal overload: A theoretical argument for the causes of Zoom fatigue,” Technology, Mind, and Behavior, Vol.2, No.1, 2021. https://doi.org/10.1037/tmb0000030
17. [17] T. Obo and K. Takizawa, “Analysis of timing and effect of visual cue on turn-taking in human-robot interaction,” J. Robot. Mechatron., Vol.34, No.1, pp. 55-63, 2022. https://doi.org/10.20965/jrm.2022.p0055
18. [18] L. Yang, S. Inagaki, D. Morita, and Y. Sejima, “Development of an avatar-robot with interactive atmosphere in online communication,” The Proc. of 2022 JSME Annual Conf. on Robotics and Mechatronics (Robomec), Session ID 2A1-H07, 2022 (in Japanese). https://doi.org/10.1299/jsmermd.2022.2A1-H07
19. [19] V. M. Reid, K. Dunn, R. J. Young, J. Amu, T. Donovan, and N. Reissland, “The human fetus preferentially engages with face-like visual stimuli,” Current Biology, Vol.27, No.12, pp.1825-1828, 2017. https://doi.org/10.1016/j.cub.2017.05.044
20. [20] Y. Sejima, S. Egawa, R. Maeda, Y. Sato, and T. Watanabe, “A speech-driven pupil response robot synchronized with burst-pause of utterance,” 2017 26th IEEE Int. Symp. on Robot and Human Interactive Communication (RO-MAN), pp. 437-442, 2017. https://doi.org/10.1109/ROMAN.2017.8172339
21. [21] T. Watanabe, M. Ogikubo, and Y. Ishii, “Visualization of respiration in the embodied virtual communication system and its evaluation,” Int. J. of Human-Computer Interaction, Vol.17, No.1, pp. 89-102, 2004. https://doi.org/10.1207/s15327590ijhc1701_7
22. [22] I. Inagaki, D. Morita, and Y. Sejima, “Development of a communication system that expresses interactive atmosphere with changing the background color using interaction-activated communication model,” Proc. of the Symp. on Human Interface 2022, pp. 533-537, 2022 (in Japanese).
23. [23] Y. Sejima, Y. Sato, T. Watanabe, and M. Jindai, “Speech-driven embodied entrainment character system with pupillary response,” Mechanical Engineering J., Vol.3, No.4, Article No.15-00314, 2016. https://doi.org/10.1299/mej.15-00314
24. [24] R. D. Luce, “Individual choice behaviour: A theoretical analysis,” J. Wiley, 1959.
25. [25] P. Ekman, R. W. Levenson, and W. V. Friesen, “Autonomic nervous system activity distinguishes among emotions,” Science, Vol.221, No.4616, pp. 1208-1210, 1983. https://doi.org/10.1126/science.6612338
26. [26] T. Yamada and T. Watanabe, “An average facial color image avatar system for the analysis by synthesis of affect display by dynamic facial color and expression,” Trans. of the Japan Society of Mechanical Engineers, Series C, Vol.78, No.791, pp. 2526-2535, 2012 (in Japanese). https://doi.org/10.1299/kikaic.78.2526
27. [27] D. McDuff and E. M. Nowara, “‘Warm bodies’: A post-processing technique for animating dynamic blood flow on photos and avatars,” Proc. of the 2021 Conf. on Human Factors in Computing Systems (CHI’21), Article No.579, 2021. https://doi.org/10.1145/3411764.3445719
28. [28] M. Hori, Y. Tsuruda, H. Yoshimura, and Y. Iwai, “Expression transmission using exaggerated animation for Elfoid,” Frontiers in Psychology, Vol.6, Article No.1219, 2015. https://doi.org/10.3389/fpsyg.2015.01219