Recent advances in artificial intelligence and robotics have accelerated the deployment of service robots in daily environments. However, many systems still lack adaptive responsiveness to the nonverbal behaviors of users. This study proposes a gaze-based user analysis system integrated into a mixed reality (MR) smart home environment to support attentional intention-aware human–system interactions. Rather than directly estimating emotional states, the proposed approach infers attentional intentions of users based on gaze behavior as an operational proxy for emotional attunement. Using gaze data collected through HoloLens 2, we develop a machine learning model based on a long short-term memory network combined with a mixture density network to predict future gaze coordinates in a three-dimensional space. The predicted gaze information is shared with a robotic partner to enable proactive context-aware information support. The proposed system demonstrates the feasibility of leveraging gaze prediction to anticipate user focus and provide adaptive support in MR-based smart environments.

1. [1] M. C.-T. Tai, “The impact of artificial intelligence on human society and bioethics,” Tzu Chi Medical J., Vol.32, No.4, pp. 339-343, 2020. https://doi.org/10.4103/tcmj.tcmj_71_20
2. [2] T. Fong, I. Nourbakhsh, and K. Dautenhahn, “A survey of socially interactive robots,” Robotics and Autonomous Systems, Vol.42, Nos.3-4, pp. 143-166, 2003. https://doi.org/10.1016/S0921-8890(02)00372-X
3. [3] C. Bartneck, T. Kanda, O. Mubin, and A. Al Mahmud, “Does the design of a robot influence its animacy and perceived intelligence?,” Int. J. of Social Robotics, Vol.1, No.2, pp. 195-204, 2009. https://doi.org/10.1007/s12369-009-0013-7
4. [4] G. Bardaro, A. Antonini, and E. Motta, “Robots for elderly care in the home: A landscape analysis and co-design toolkit,” Int. J. of Social Robotics, Vol.14, No.3, pp. 657-681, 2022. https://doi.org/10.1007/s12369-021-00816-3
5. [5] L. C. Cesário, P. Barbosa, P. A. C. Miguel, and G. H. Mendes, “Service robots in caring for older adults: Uncovering the current conceptual and intellectual structures and future research agenda,” Archives of Gerontology and Geriatrics, Vol.131, Article No.105755, 2025. https://doi.org/10.1016/j.archger.2025.105755
6. [6] J. Abdi, A. Al-Hindawi, T. Ng, and M. P. Vizcaychipi, “Scoping review on the use of socially assistive robot technology in elderly care,” BMJ Open, Vol.8, No.2, Article No.e018815, 2018. https://doi.org/10.1136/bmjopen-2017-018815
7. [7] Y. Sone, C. Matsumoto, J. Woo, and Y. Ohyama, “Development of a control support system for smart homes using the analysis of user interests based on mixed reality,” ROBOMECH J., Vol.12, Article No.2, 2025. https://doi.org/10.1186/s40648-024-00288-w
8. [8] I. Leite, A. Pereira, S. Mascarenhas, C. Martinho, R. Prada, and A. Paiva, “The influence of empathy in human–robot relations,” Int. J. of Human-Computer Studies, Vol.71, No.3, pp. 250-260, 2013. https://doi.org/10.1016/j.ijhcs.2012.09.005
9. [9] M. Coeckelbergh, “Artificial companions: Empathy and vulnerability mirroring in human-robot relations,” Studies in Ethics, Law, and Technology, Vol.4, No.3, 2010. https://doi.org/10.2202/1941-6008.1126
10. [10] M. Spezialetti, G. Placidi, and S. Rossi, “Emotion recognition for human-robot interaction: Recent advances and future perspectives,” Frontiers in Robotics and AI, Vol.7, Article No.532279, 2020. https://doi.org/10.3389/frobt.2020.532279
11. [11] M. Tomasello, “Origins of Human Communication,” MIT Press, 2010.
12. [12] M. Tomasello, “A Natural History of Human Thinking,” Harvard University Press, 2014. https://doi.org/10.4159/9780674726369
13. [13] S. Balzarotti, L. Piccini, G. Andreoni, and R. Ciceri, ““I know that you know how I feel”: Behavioral and physiological signals demonstrate emotional attunement while interacting with a computer simulating emotional intelligence,” J. of Nonverbal Behavior, Vol.38, No.3, pp. 283-299, 2014. https://doi.org/10.1007/s10919-014-0180-6
14. [14] C. Matsumoto, Y. Sone, J. Woo, and Y. Ohyama, “Development of a gesture analysis system for human system interaction,” 2024 Joint 13th Int. Conf. on Soft Computing and Intelligent Systems and 25th Int. Symp. on Advanced Intelligent Systems (SCIS&ISIS), 2024. https://doi.org/10.1109/SCISISIS61014.2024.10760070
15. [15] Y. Sone and J. Woo, “Design of a human-centric robotic system for user support based on gaze information,” J. Adv. Comput. Intell. Intell. Inform., Vol.29, No.4, pp. 796-802, 2025. https://doi.org/10.20965/jaciii.2025.p0796
16. [16] J. Woo and J. Hu, “System for analyzing user interest based on eye gaze responses to enhance empathy with users,” J. Adv. Comput. Intell. Intell. Inform., Vol.29, No.3, pp. 641-648, 2025. https://doi.org/10.20965/jaciii.2025.p0641
17. [17] P. Balakrishnan and H.-J. Guo, “Hololens 2 technical evaluation as mixed reality guide,” Int. Conf. on Human-Computer Interaction, pp. 145-165, 2024. https://doi.org/10.1007/978-3-031-61041-7_10
18. [18] C. Matsumoto, Y. Sone, J. Woo, and Y. Ohyama, “Development of a user-friendly interface for a smart home system based on IoT technology,” The 20th World Congress of the Int. Fuzzy Systems Association (IFSA 2023), 2023.
19. [19] J. Woo, Y. Ohyama, and N. Kubota, “Robot partner development platform for human-robot interaction based on a user-centered design approach,” Applied Sciences, Vol.10, No.22, Article No.7992, 2020. https://doi.org/10.3390/app10227992
20. [20] J. W. Kelly, T. A. Doty, M. Ambourn, and L. A. Cherep, “Distance perception in the Oculus Quest and Oculus Quest 2,” Frontiers in Virtual Reality, Vol.3, Article No.850471, 2022. https://doi.org/10.3389/frvir.2022.850471
21. [21] C. S. Rosales, G. Pointon, H. Adams, J. Stefanucci, S. Creem-Regehr, W. B. Thompson, and B. Bodenheimer, “Distance judgments to on-and off-ground objects in augmented reality,” 2019 IEEE Conf. on Virtual Reality and 3D User Interfaces (VR), pp. 237-243, 2019. https://doi.org/10.1109/VR.2019.8798095