JRM Vol.36 No.1 pp. 168-180
doi: 10.20965/jrm.2024.p0168


Proposal of Learning Support Model for Teacher-Type Robot Supporting Learning According to Learner’s Perplexed Facial Expressions

Kohei Okawa*1 ORCID Icon, Felix Jimenez*2 ORCID Icon, Shuichi Akizuki*3, and Tomohiro Yoshikawa*4

*1Graduate School of Information Science and Technology, Aichi Prefectural University
1522-3 Ibaragabasama, Nagakute-shi, Aichi 480-1198, Japan

*2School of Information Science and Technology, Aichi Prefectural University
1522-3 Ibaragabasama, Nagakute-shi, Aichi 480-1198, Japan

*3School of Engineering, Chukyo University
101-2 Yagoto Honmachi, Showa-ku, Nagoya, Aichi 466-8666, Japan

*4Faculty of Medical Engineering, Suzuka University of Medical Science
1001-1 Kishioka, Suzuka-city, Mie 510-0293, Japan

June 12, 2023
October 5, 2023
February 20, 2024
educational support robot, collaborative learning, deep learning, human-robot interaction

The introduction of ICT into education in educational settings has become increasingly common. Among these, the research and development of educational support robots have attracted attention. Conventional robots provide academic support through button operation by the learner. However, it has been reported that excessive requests for academic support can be problematic in environments in which learners can freely request academic support. To solve this problem, we developed a perplexion estimation method that estimates the state of perplexion from the learner’s facial expression. When educational support is provided using the proposed method, the robot can autonomously provide academic support. Simulation experiments demonstrated that the perplexion estimation method has the potential to accurately estimate the learner’s state. However, there is still no dedicated model for educational support robots that utilizes the perplexion estimation method. Therefore, this paper proposes an apprenticeship promotion model that integrates a learning support method based on cognitive apprenticeship theory and our perplexion estimation method. We then verified the learning effects of the robot equipped with the proposed model on the learner through participant experiments. The experimental results suggest that the robot equipped with the proposed model not only provides the same learning effect to university students as a conventional robot, but also can autonomously provide academic support at the optimal timing.

Robot autonomously providing learning assistance

Robot autonomously providing learning assistance

Cite this article as:
K. Okawa, F. Jimenez, S. Akizuki, and T. Yoshikawa, “Proposal of Learning Support Model for Teacher-Type Robot Supporting Learning According to Learner’s Perplexed Facial Expressions,” J. Robot. Mechatron., Vol.36 No.1, pp. 168-180, 2024.
Data files:
  1. [1] T. Belpaeme et al., “Social robots for education: A review,” Science Robotics, Vol.3, Issue 21, 2018.
  2. [2] W.-F. Hsieh et al., “Investigation of Robot Expression Style in Human-Robot Interaction,” J. Robot. Mechatron., Vol.32, No.1, pp. 224-235, 2020.
  3. [3] T. Obo and K. Takizawa, “Analysis of Timing and Effect of Visual Cue on Turn-Taking in Human-Robot Interaction,” J. Robot. Mechatron., Vol.34, No.1, pp. 55-63, 2022.
  4. [4] K. Kadota, “Development of Communication Robot for STEM Education by Using Digital Fabrication,” J. Robot. Mechatron., Vol.29, No.6, pp. 944-951, 2017.
  5. [5] R. Yoshizawa et al., “Proposal of a Behavioral Model for Robots Supporting Learning According to Learners’ Learning Performance,” J. Robot. Mechatron., Vol.32, No.4, pp. 769-779, 2020.
  6. [6] A. Collins et al., “Cognitive Apprenticeship: Teaching the Craft of Reading, Writing, and Mathematics,” L. Resnick (Ed.), “Knowing, Learning, and Instruction: Essays in Honor of Robert Glaser,” Routledge, 1989.
  7. [7] S. P. Lajoie and A. Lesgold, “Apprenticeship Training in the Workplace: Computer-Coached Practice Environment as a New Form of Apprenticeship,” Machine-Mediated Learning, Vol.3, Issue 1, pp. 7-28, 1989.
  8. [8] V. Aleven and K. R. Koedinger, “Limitations of student control: Do students know when they need help?,” Proc. of the Int. Conf. on Intelligent Tutoring Systems (ITS 2000), pp. 292-303, 2000.
  9. [9] J. A. Walonoski and N. T. Heffernan, “Detection and Analysis of Off-Task Gaming Behavior in Intelligent Tutoring Systems,” Proc. of the Int. Conf. on Intelligent Tutoring Systems (ITS 2006), pp. 382-391, 2006.
  10. [10] F. Jimenez and M. Kanoh, “Change in Learning Ability Using Scaffold in EFL Vocabulary Learning System,” J. of Japan Society for Fuzzy Theory and Intelligent Informatics, Vol.25, No.5, pp. 880-888, 2013 (in Japanese).
  11. [11] I. Roll et al., “Improving students’ help-seeking skills using metacognitive feedback in an intelligent tutoring system,” Learning and Instruction, Vol.21, No.2, pp. 267-280, 2011.
  12. [12] A. Kanemitsu et al., “The Current Status of Social Robots for Education,” Bulletin of Tokyo Metropolitan Institute of Technology, No.14, pp. 91-96, 2020.
  13. [13] F. Shimono et al., “Method for assessment of mental stress during high-tension and monotonous tasks using heart rate, respiration and blood pressure,” The Japanese J. of Ergonomics, Vol.34, No.3, pp. 107-115, 1998 (in Japanese).
  14. [14] K. Miyauchi et al., “Learning Effects of Robots Teaching Based on Cognitive Apprenticeship Theory,” J. Adv. Comput. Intell. Intell. Inform., Vol.24, No.1, pp. 101-112, 2020.
  15. [15] T. Matsui, “Estimation of Learners’ Physiological Information and Learners’ Mental States by Machine Learning and its Application for Learning Support,” Trans. of Japanese Society for Information and Systems in Education, Vol.36, No.2, pp. 76-83, 2019.
  16. [16] P. Ekman and W. V. Friesen, “Unmasking the face,” Seishin Shobou, 1987 (in Japanese).
  17. [17] T. Nishime et al., “Feature Acquisition from Facial Expression Image Using Convolutional Neural Networks,” J. of Robotics, Networking and Artificial Life, Vol.3, No.1, pp. 9-12, 2016.
  18. [18] O. Arriaga et al., “Realtime Convolutional Neural Networks for Emotion and Gender Classification,” arXiv:1710.07557, 2017.
  19. [19] A. Gulli and S. Pal, “Chokkan deep learning: Python × Keras de aidea wo katachi ni suru reshipi,” O’Reilly Japan, 2018 (in Japanese).
  20. [20] I. J. Goodfellow et al., “Challenges in representation learning: A report on three machine learning contests,” Proc. of Int. Conf. on Neural Information Processing (ICONIP 2013), pp. 117-124, 2013.
  21. [21] T. Oyama, “Historical background and the present status of reaction time studies,” The Japanese J. of Ergonomics, Vol.21, No.2, pp. 57-64, 1985.
  22. [22] T. Nagai, “Jukuren jutsusya no Trifecta wo kakyuteki ni sokonawanaitameno kyouiku sisutemu no kouchiku,” Japanese J. of Endourology, Vol.31, No.1, pp. 27-31, 2018.
  23. [23] A. Kashihara et al., “Cognitive Apprenticeship Approach to Developing Meta-Cognitive Skill with Cognitive Tool for Web-Based Navigational Learning,” Proc. of the 7th IASTED Int. Conf. on Web-Based Education, 2008.
  24. [24] F. Jimenez et al., “An Emotional Expression Model for Educational-Support Robots,” J. of Artificial Intelligence and Soft Computing Research, Vol.5, No.1, pp. 51-57, 2015.
  25. [25] Y. Tanizaki et al., “Impression Effects of Educational Support Robots Using Sympathy Expressions Method by Body Movement and Facial Expression,” J. of Japan Society for Fuzzy Theory and Intelligent Informatics, Vol.30, No.5, pp. 700-708, 2018.
  26. [26] D. Lakens, “Equivalence Tests: A Practical Primer for t Tests, Correlations, and Meta-Analyses,” Social Psychological and Personality Science, Vol.8, No.4, pp. 355-362, 2017.
  27. [27] E. Saeki, “Executive control processes underpinning cognitive flexibility: Findings from task-switching studies,” Japanese Psychological Review, Vol.58, No.1, pp. 34-51, 2015.

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Apr. 05, 2024