single-au.php

IJAT Vol.17 No.3 pp. 248-261
doi: 10.20965/ijat.2023.p0248
(2023)

Research Paper:

Effect of Perspective and Visuo-Tactile Feedback in Virtual Reality-Based Posture Learning

Ryo Hanashima*,**, Takumi Tokuda*,***, Masaaki Mochimaru*,*** ORCID Icon, and Junji Ohyama*,**,† ORCID Icon

*National Institute of Advanced Industrial Science and Technology (AIST)
Kashiwa 2nd Campus, The University of Tokyo, 6-2-3 Kashiwa, Chiba 277-0882, Japan

**University of Tsukuba
Tsukuba, Japan

***The University of Tokyo
Kashiwa, Japan

Corresponding author

Received:
October 24, 2022
Accepted:
February 6, 2023
Published:
May 5, 2023
Keywords:
motor learning, virtual reality, avatar, perspective, tactile sensation
Abstract

Posture learning is required in rehabilitation and in sports such as yoga and martial arts. Virtual reality (VR) systems are being used to learn posture by superimposing the postures of the learner and instructor in cyberspace using avatars. In this study, we examined whether the presented perspective of the avatar (first-person vs. third-person perspective) and visuo-tactile feedback (tactile correct feedback + visual feedback vs. tactile incorrect feedback + visual feedback vs. visual only feedback) are effective for the posture learning. The results of an experiment (N = 24) suggested that use of the third-person perspective may result in accurate learning of the head position as compared with first-person perspective. Visuo-tactile feedback was found to improve the subjective rating on the ease of learning, while the presentation method in which tactile feedback is given when body position is correct was found to be more effective than tactile feedback given when body position is incorrect. The sense of agency was maintained at a high level under all conditions for perspective and visuo-tactile feedback and may have improved the learning accuracy of posture. The findings of this study are expected to contribute to the design of effective perspective and tactile presentation in VR-based motor learning.

Cite this article as:
R. Hanashima, T. Tokuda, M. Mochimaru, and J. Ohyama, “Effect of Perspective and Visuo-Tactile Feedback in Virtual Reality-Based Posture Learning,” Int. J. Automation Technol., Vol.17 No.3, pp. 248-261, 2023.
Data files:
References
  1. [1] R. A. Schmidt, T. D. Lee, T. D. Carolee, G. Wulf, and H. Zelaznik, “Motor Control and Learning (sixth edition),” Human Kinetics, Champaign, 2018.
  2. [2] G. Wulf, C. Shea, and R. Lewthwaite, “Motor skill learning and performance: A review of influential factors,” Med. Educ., Vol.44, No.1, pp. 75-84, 2010. https://doi.org/10.1111/j.1365-2923.2009.03421.x
  3. [3] R. Sigrist, G. Rauter, R. Riener, and P. Wolf, “Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review,” Psychon. Bull. Rev., Vol.20, No.1, pp. 21-53, 2013. https://doi.org/10.3758/s13423-012-0333-8
  4. [4] D. L. Neumann, R. L. Moffitt, P. R. Thomas, K. Loveday, D. P. Watling, C. L. Lombard, S. Antonova, and M. A. Tremeer, “A systematic review of the application of interactive virtual reality to sport,” Virtual Reality, Vol.22, No.3, pp. 183-198, 2018. https://doi.org/10.1007/s10055-017-0320-5
  5. [5] A. Asadzadeh, T. Samad-Soltani, Z. Salahzadeh, and P. Rezaei-Hachesu, “Effectiveness of virtual reality-based exercise therapy in rehabilitation: A scoping review,” Inf. Med. Unlocked, Vol.24, 100562, 2021. https://doi.org/10.1016/j.imu.2021.100562
  6. [6] S. K. Renganayagalu, S. C. Mallam, and S. Nazir, “Effectiveness of VR Head Mounted Displays in Professional Training: A Systematic Review,” Technology, Knowledge and Learning, Vol.26, pp. 999-1041, 2021. https://doi.org/10.1007/s10758-020-09489-9
  7. [7] M. K. Holden and E. Todorov, “Use of Virtual Environments in Motor Learning and Rehabilitation,” M. K. Stanny (Ed.), “Handbook of Virtual Environments: Design, Implementation, and Applications,” Lawrence Erlbaum Associates, pp. 1-35, 2002.
  8. [8] T. Shank, “Measuring Mental Representations,” G. Tenenbaum, R. C. Eklund, and A. Kamata (Eds.), “Measurement in sport and exercise psychology,” Human Kinetics, Champaign, 2012.
  9. [9] T. Schack and F. Mechsner, “Representation of motor skills in human long-term memory,” Neurosci. Lett., Vol.391, No.3, pp. 77-81, 2006. https://doi.org/10.1016/j.neulet.2005.10.009
  10. [10] T. N. Hoang, M. Reinoso, F. Vetere, and E. Tanin, “Onebody: Remote posture guidance system using first person view in virtual environment,” Proc. of the 9th Nordic Conf. on Human-Computer Interaction (NordiCHI ’16), pp. 23-27, 2016. https://doi.org/10.1109/VR.2003.1191125
  11. [11] D. S. Kaesler, R. B. Mellifont, P. D. Kelly, and D. R, Taaffe, “A novel balance exercise program for postural stability in older adults: A pilot study,” J. Bodyw. Mov. Ther., Vol.11, No.1, pp. 37-43, 2007. https://doi.org/10.1016/j.jbmt.2006.05.003
  12. [12] P. T. Chua, R. Crivella, B. Daly, N. Hu, R. Schaaf, D. Ventura, T. Camill, J. Hodgins, and R. Pausch, “Training for physical tasks in virtual environments: Tai Chi,” IEEE Virtual Reality, pp. 87-94, 2003. https://doi.org/10.1109/VR.2003.1191125
  13. [13] T. Kojima, A. Hiyama, T. Miura, and M. Hirose, “Training Archived Physical Skill through Immersive Virtual Environment,” S. Yamamoto (Ed.), “Human Interface and the Management of Information. Information and Knowledge in Applications and Services,” HIMI 2014. Lecture Notes in Computer Science, Vol.8522, pp. 51-58, 2014. https://doi.org/10.1007/978-3-319-07863-2_6
  14. [14] F. Hülsmann, C. Frank, I. Senna, M. O. Ernst, T. Schack, and M. Botsch, “Superimposed Skilled Performance in a Virtual Mirror Improves Motor Performance and Cognitive Representation of a Full Body Motor Action,” Front. Rob. AI, Vol.6, 43, 2019. https://doi.org/10.3389/frobt.2019.00043
  15. [15] T. Le Naour, L. Hamon, and J.-P. Bresciani, “Superimposing 3D Virtual Self + Expert Modeling for Motor Learning: Application to the Throw in American Football,” Front. ICT, Vol.6, 16, 2019. https://doi.org/10.3389/fict.2019.00016
  16. [16] T. Waltemate, I. Senna, F. Hülsmann, M. Rohde, S. Kopp, M. Ernst, and M. Botsch, “The impact of latency on perceptual judgments and motor performance in closed-loop interaction in virtual reality,” Proc. of the 22nd ACM Conf. on Virtual Reality Software and Technology (VRST ’16), pp. 27-35, 2016. https://doi.org/10.1145/2993369.2993381
  17. [17] S. Gallagher, “Philosophical conceptions of the self: implications for cognitive science,” Trends Cognit. Sci., Vol.4, No.1, pp. 14-21, 2000. https://doi.org/10.1016/S1364-6613(99)01417-5
  18. [18] M. Tsakiris, G. Prabhu, and P. Haggard, “Having a body versus moving your body: How agency structures body-ownership,” Conscious Cogn., Vol.15, No.2, pp. 423-432, 2006. https://doi.org/10.1016/j.concog.2005.09.004
  19. [19] O. Blanke and T. Metzinger, “Full-body illusions and minimal phenomenal selfhood,” Trends Cognit. Sci., Vol.13, No.1, pp. 7-13, 2009. https://doi.org/10.1016/j.tics.2008.10.003
  20. [20] K. Kilteni, R. Groten, and M. Slater, “The Sense of embodiment in virtual reality,” Presence (Camb), Vol.22, No.1, pp. 373-387, 2013. https://doi.org/10.1162/PRES_a_00124
  21. [21] T. Asai, “Feedback control of one’s own action: Self-other sensory attribution in motor control,” Conscious Cogn., Vol.38, pp. 118-129, 2015. https://doi.org/10.1016/j.concog.2015.11.002
  22. [22] D. Burin, K. Kilteni, M. Rabuffetti, and L. Pia, “Body ownership increases the interference between observed and executed movements,” PLoS One, Vol.14, No.1, e0209899, 2019. https://doi.org/10.1371/journal.pone.0209899
  23. [23] K. Grechuta, J. Guga, G. Maffei, B. R. Balleater, and P. F. M. J. Verschure, “Visuotactile integration modulates motor performance in a perceptual decision-making task,” Sci. Rep., Vol.7, 3333, 2017. https://doi.org/10.1038/s41598-017-03488-0
  24. [24] R. Newport, R. Pearce, and C. Preston, “Fake hands in action: Embodiment and control of supernumerary limbs,” Exp. Brain Res., Vol.204, No.3, pp. 385-395, 2010. https://doi.org/10.1007/s00221-009-2104-y
  25. [25] R. Zopf, S. Truong, M. Finkbeiner, J. Friedman, and M. A. Williams, “Viewing and feeling touch modulates hand position for reaching,” Neuropsychologia, Vol.49, No.5, pp. 1287-1293, 2011. https://doi.org/10.1016/j.neuropsychologia.2011.02.012
  26. [26] M. P. M. Kammers, F. de Vignemont, L. Verhagen, and H. C. Dijkerman, “The rubber hand illusion in action,” Neuropsychologia, Vol.47, pp. 204-211, 2009. https://doi.org/10.1016/j.neuropsychologia.2008.07.028
  27. [27] K. Matsumiya, “Awareness of voluntary action, rather than body ownership, improves motor control,” Sci. Rep., Vol.11, 418, 2021. https://doi.org/10.1038/s41598-020-79910-x
  28. [28] I. A. Odermatt, K. A. Buetler, N. Wenk, Ö. Özen, J. Penalver-Andres, T. Nef, F. W. Mast, and L. Marchal-Crespo, “Congruency of Information Rather Than Body Ownership Enhances Motor Performance in Highly Embodied Virtual Reality,” Front. Neurosci., Vol.15, 678909, 2021. https://doi.org/10.3389/fnins.2021.678909
  29. [29] A. T. Reader, V. S. Trifonova, and H. H. Ehrsson, “Little evidence for an effect of the rubber hand illusion on basic movement,” Eur. J. Neurosci., Vol.7, No.54, pp. 6463-6486, 2021. https://doi.org/10.1111/ejn.15444
  30. [30] S. Malpica, A. Serrano, and D. Gutierrez, “Auditory stimuli degrade visual performance in virtual reality,” Sci. Rep., Vol.10, 12363, 2020. https://doi.org/10.1038/s41598-020-69135-3
  31. [31] R. Sigrist, G. Rauter, L. Marchal-Crespo, R. Riener, and P. Wolf, “Sonification and haptic feedback in addition to visual feedback enhances complex motor task learning,” Exp. Brain. Res., Vol.233, No.3, pp. 909-925, 2015. https://doi.org/10.1007/s00221-014-4167-7
  32. [32] J. Ohyama, “Xperigrapher: Social-Lab Experimental Platform to Evaluate Experience in Cyber Physical Society,” Trans. Japan. Soc. Med. Biol. Eng., Vol.59, pp. 811-813, 2021. https://doi.org/10.11239/jsmbe.Annual59.811
  33. [33] T. Maeno, “Structure and Function of Finger Pad and Tactile Receptors,” J. of the Robotics Society of Japan, Vol.18, No.6, pp. 772-775, 2000 (in Japanese). https://doi.org/10.7210/jrsj.18.772
  34. [34] T. C. Peck and M. Gonzalez-Franco, “Avatar Embodiment. A Standardized Questionnaire,” Front. Virtual Reality, Vol.1, 575943, 2021. https://doi.org/10.3389/frvir.2020.575943
  35. [35] H. Shimizu, “An introduction to the statistical free software HAD: Suggestions to improve teaching, learning and practice data analysis,” J. of Media, Information and Communication, Vol.1, pp. 59-73, 2016 (in Japanese).
  36. [36] T. Hashimoto and A. Iriki, “Somatosensation,” Brain Science Dictionary, 2012 (in Japanese). https://bsd.neuroinf.jp/wiki/%E4%BD%93%E6%80%A7%E6%84%9F%E8%A6%9A [Accessed September 6, 2022].
  37. [37] G. Gorisse, O. Christmann, E. A. Amato, and S. Richir, “First- and Third-Person Perspectives in Immersive Virtual Environments: Presence and Performance Analysis of Embodied Users,” Front. Rob. AI, Vol.4, 33, 2017. https://doi.org/10.3389/frobt.2017.00033
  38. [38] F. Argelaguet, L. Hoyet, M. Trico, and A. Lécuyer, “The role of interaction in virtual embodiment: Effects of the virtual hand representation,” 2016 IEEE Virtual Reality (VR), pp. 3-10, 2016. https://doi.org/10.1109/VR.2016.7504682
  39. [39] R. Hanashima and J. Ohyama, “Experimental psychology study using virtual reality: The effects of avatar size on somatosensory transference,” IPSJ Special Interest Group on Entertainment Computing, Vol.54, No.11, pp. 1-2, 2019 (in Japanese).
  40. [40] W. Choi, L. Li, S. Satoh, and K. Hachimura, “Multisensory Integration in the Virtual Hand Illusion with Active Movement,“ Biomed Res. Int., Vol.2016, 8163098, 2016. https://doi.org/10.1155/2016/8163098
  41. [41] C. D. Frith, S. J. Blakemore, and D. W. Wolpert, “Abnormalities in the awareness and control of action,” Philos. Trans. R. Soc. B: Biol. Sci., Vol.355, pp. 1771-1788, 2000. https://doi.org/10.1098/rstb.2000.0734
  42. [42] C. D. Frith, S. Blakemore, and D. W. Wolpert, “Explaining the symptoms of schizophrenia: abnormalities in the awareness of action,” Brain Res. Rev., Vol.31, No.2-3, pp. 357-363, 2000. https://doi.org/10.1016/s0165-0173(99)00052-1
  43. [43] R. Hanashima and J. Ohyama, “How to Elicit Ownership and Agency for an Avatar Presented in the Third-Person Perspective: The Effect of Visuo-Motor and Tactile Feedback,” S. Yamamoto and H. Mori (Eds.), “Human Interface and the Management of Information: Applications in Complex Technological Environments,” HCII 2022. Lecture Notes in Computer Science, Vol.13306, pp. 111-130, 2022. https://doi.org/10.1007/978-3-031-06509-5_9
  44. [44] M. D’Angelo, G. di Pellegrino, S. Seriani, P. Gallina, and F. Frassinetti, “The sense of agency shapes body schema and peripersonal space,” Sci. Rep., Vol.8, 13847, 2018. https://doi.org/10.1038/s41598-018-32238-z
  45. [45] C. J. Winstein and R. A. Schmidt, “Reduced frequency of knowledge of results enhances motor skill learning,” J. Exp. Psychol.: Learn. Mem. Cogn., Vol.16, No.4, pp. 677-91, 1990. https://doi.org/10.1037/0278-7393.16.4.677

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Feb. 19, 2024