The aim of this study is to clarify the characteristics of visually induced motion sickness (VIMS) during tele-operation of construction machinery using subjective and objective indicators. To clearly define and ensure the repeatability of the experiment conditions, we used virtual reality to simulate a real-world construction machinery tele-operation system. 90 people with no prior operating experience participated experiments under the experimental conditions of image presentation device and delay in reflecting operations in generated images. To evaluate visually induced motion sickness, we used body sway and the R-R interval, the ratio of low-frequency to high-frequency components (LF/HF), and the m and S indicators of Lorenz plots derived heart rate variability. In addition, subjective evaluation was performed using the simulator sickness questionnaire. We conducted a multifaceted analysis including interactions between time-series variation in operational performance and physiological indicators and subjective evaluations. In the operational performance results, the participants demonstrated reasonable learning effects while exhibiting negative effects of delays. The variation in body sway indicators among participants was too large to demonstrate the effects of operation delays. No effects of the experimental conditions were found when averaging the heart rate variability under the assumption that there is common time-series variation among the participants. A relation, however, was found between heart rate variability and the subjective symptom of sickness by using an indicator based on broader variation in heart rate variability. This is a finding that previous studies using similar experimental conditions for VIMS in tele-operation of construction machinery have not been able to demonstrate.

1. [1] Y. Ban, “Unmanned Construction System: Present Status and Challenges,” Proc. of the 19th Int. Symp. on Autom. and Robot. in Constr. (ISARC), pp. 241-246, 2002. https://doi.org/10.22260/ISARC2002/0038
2. [2] Q. H. Le, J. W. Lee, and S. Y. Yang, “Remote Control of Excavator Using Head Tracking and Flexible Monitoring Method,” Autom. Constr., Vol.81, pp. 99-111, 2017. https://doi.org/10.1016/j.autcon.2017.06.015
3. [3] A. Nishiyama, M. Moteki, K. Fujino, and T. Hashimoto, “Research on the Comparison of Operator Viewpoints Between Manned and Remote Control Operation in Unmanned Construction Systems,” Proc. of the 30th Int. Symp. on Autom. Robot. in Constr. (ISARC), pp. 677-684, 2013. https://doi.org/10.22260/ISARC2013/0074
4. [4] D. Kondo, “Projection Screen with Wide-FOV and Motion Parallax Display for Teleoperation of Construction Machinery,” J. Robot. Mechatron., Vol.33, No.3, pp. 604-609, 2021. https://doi.org/10.20965/jrm.2021.p0604
5. [5] K. Okuda, Y. Ohbatake, and D. Kondo, “Perception of Vestibular Sensation During Turning Operation of Construction Machine,” J. Robot. Mechatron., Vol.33, No.3, pp. 599-603, 2021. https://doi.org/10.20965/jrm.2021.p0599
6. [6] K. W. Oh, D. Kim, and D. Hong, “Performance Evaluation of Excavator Control Device with EMG-Based Fatigue Analysis,” Int. J. Precis. Eng. Manuf., Vol.15, No.2, pp. 193-199, 2014. https://doi.org/10.1007/s12541-014-0325-x
7. [7] M. N. Selzer, N. F. Gazcon, and M. L. Larrea, “Effects of Virtual Presence and Learning Outcome Using Low-End Virtual Reality Systems,” Displays, Vol.59, pp. 9-15, 2019. https://doi.org/10.1016/j.displa.2019.04.002
8. [8] W. T. Lo and R. H. Y. So, “Cybersickness in the Presence of Scene Rotational Movements Along Different Axes,” Appl. Ergon., Vol.32, Issue 1, pp. 1-14, 2001. https://doi.org/10.1016/S0003-6870(00)00059-4
9. [9] D. Saredakis, A. Szpak, B. Birckhead, H. A. D. Keage, A. Rizzo, and T. Loetscher, “Factors Associated with Virtual Reality Sickness in Head-Mounted Displays: A Systematic Review and Meta-Analysis,” Front. Hum. Neurosci., Vol.14, Article No.96, 2020. https://doi.org/10.3389/fnhum.2020.00096
10. [10] Z. Hong, Q. Zhang, X. Su, and H. Zhang, “Effect of Virtual Annotation on Performance of Construction Equipment Teleoperation Under Adverse Visual Conditions,” Autom. Constr., Vol.118 Article No.103296, 2020. https://doi.org/10.1016/j.autcon.2020.103296
11. [11] M. Moteki, K. Fujino, T. Ohtsuki, and T. Hashimoto, “Research on Visual Point of Operator in Remote Control of Construction Machinery,” Proc. of the 28th Int. Symp.on Autom. and Robot. in Constr. (ISARC), pp. 532-537, 2011. https://doi.org/10.22260/ISARC2011/0096
12. [12] M. Wallmyr, T. A. Sitompul, T. Holstein, and R. Lindell, “Evaluating Mixed Reality Notifications to Support Excavator Operator Awareness,” Hum.-Comp. Interact. (INTERACT 2019), pp. 743-762, 2019. https://doi.org/10.1007/978-3-030-29381-9_44
13. [13] T. Hashimoto, G. Yamauchi, K. Fujino, S. Yuta, and K. Tateyama, “Study of Operator’s Line of Sight in Unmanned Construction Systems,” 2018 IEEE Int. Symp. on Saf., Secur., and Rescue Robot. (SSRR), 2018. https://doi.org/10.1109/SSRR.2018.8468635
14. [14] Y. Sato, H. Matsumoto, and T. Wada, “Effect of Seat Rotation of a Remote-Control System for Construction Machinery on Simulator Sickness and Sense of Agency,” 44th Ann. Meeting of the Jpn. Neurosci. Soc., The 1st CJK Int. Meeting, Article No.4P-139, 2021.
15. [15] A. Matsuyama, M. Minoshima, H. Watanabe, H. Ujike, and H. Endo, “Visually Induced Motion Sickness in Operation of Remote Control Hydraulic Excavator,” The 19th Symp. on Constr. Robot. in Jpn., Article No.O5-3, 2019 (in Japanese).
16. [16] R. M. Stern, S. Hu, R. B. Anderson, H. W. Leibowitz, and K. L. Koch, “The Effects of Fixation and Restricted Visual Field on Vection-Induced Motion Sickness,” Aviat. Space Environ. Med., No.61, Vol.8, pp. 712-715, 1990.
17. [17] J. J.-W. Lin, H. B. L. Duh, D. E. Parker, H. Abi-Rached, and T. A. Furness, “Effects of Field of View on Presence, Enjoyment, Memory, and Simulator Sickness in a Virtual Environment,” Proc. of IEEE Virtual Reality 2002, pp. 164-171, 2002. https://doi.org/10.1109/VR.2002.996519
18. [18] M. Emoto, M. Sugawara, and Y. Nojiri, “Viewing Angle Dependency of Visually-Induced Motion Sickness in Viewing Wide-Field Images by Subjective and Autonomic Nervous Indices,” Displays, Vol.29, Issue 2, pp. 90-99, 2008. https://doi.org/10.1016/j.displa.2007.09.010
19. [19] H. Ujike, “Effects of Rotational Components of Yaw, Roll and Pitch on Visually-Induced Motion Sickness,” Proc. of HCII 2005, pp. 2353-2356, 2005.
20. [20] M. Griffin and M. Newman, “Visual Field Effects on Motion Sickness in Cars,” Aviat. Space Environ. Med., Vol.75, No.9, pp. 739-748, 2004.
21. [21] K. Takeshita, K. Watanabe, K. Sato, K. Minamizawa, and S. Tachi, “Study on Telexistence LXIII—Study of Tolerance of Network Delay for TELESAR3—,” Proc. of the 15th Annual Conf. of the Virtual Reality Society of Japan, pp. 146-149, 2010 (in Japanese).
22. [22] R. S. Allison, L. R. Harris, M. Jenkin, U. Jasiobedzka, and J. E. Zacher, “Tolerance of Temporal Delay in Virtual Environments,” Proc. IEEE Virtual Reality 2001, 2001. https://doi.org/10.1109/VR.2001.913793
23. [23] J. D. Moss, J. Austin, J. Salley, J. Coats, K. Williams, and E. R. Muth, “The Effects of Display Delay on Simulator Sickness,” Displays, Vol.32, Issue 4, pp. 159-168, 2011. https://doi.org/10.1016/j.displa.2011.05.010
24. [24] M. H. Draper, E. S. Viirre, T. A. Furness, and V. J. Gawron, “Effects of Image Scale and System Time Delay on Simulator Sickness within Head-Coupled Virtual Environments,” Hum. Factors, Vol.43, Issue 1, pp. 129-146, 2001. https://doi.org/10.1518/001872001775992552
25. [25] M. E. St. Pierre, S. Banerjee, A. W. Hoover, and E. R. Muth, “The Effects of 0.2 Hz Varying Latency with 20-100 ms Varying Amplitude on Simulator Sickness in a Helmet Mounted Display,” Displays, Vol.36, pp. 1-8, 2015. https://doi.org/10.1016/j.displa.2014.10.005
26. [26] M. Ito, Y. Funahara, S. Saiki, Y. Yamazaki, and Y. Kurita, “Development of a Cross-Platform Cockpit for Simulated and Tele-Operated Excavators,” J. Robot. Mechatron., Vol.31, No.2, pp. 231-239, 2019. https://doi.org/10.20965/jrm.2019.p0231
27. [27] S. Hamasaki and T. Yakoh, “Implementation and Evaluation of Decorators for Delayed Live Streaming Video on Remote Control System,” 2008 6th IEEE Int. Conf. on Ind. Inform., pp. 1220-1225, 2008. https://doi.org/10.1109/INDIN.2008.4618288
28. [28] R. S. Kennedy, N. E. Lane, K. S. Berbaum, and M. G. Lilienthal, “Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness,” The Int. J. Aviat. Psychol., Vol.3, Issue 3, pp. 203-220, 1993. https://doi.org/10.1207/s15327108ijap0303_3
29. [29] J. F. Golding, “Motion Sickness Susceptibility Questionnaire Revised and Its Relationship to Other Forms of Sickness,” Brain Res. Bull., Vol.47, Issue 5, pp. 507-516, 1998. https://doi.org/10.1016/S0361-9230(98)00091-4
30. [30] U. R. Acharya, K. P. Joseph, N. Kannathal, C. M. Lim, and J. S. Suri, “Heart Rate Variability: A Review,” Med. Biol. Eng. Comput., Vol.44, No.12, pp. 1031-1051, 2006. https://doi.org/10.1007/s11517-006-0119-0
31. [31] M. A. Pfeifer, D. Cook, J. Brodsky, D. Tice, A. Reenan, S. Swedine, J. B. Halter, and D. Porte, “Quantitative Evaluation of Cardiac Parasympathetic Activity in Normal and Diabetic Man,” Diabetes, Vol.31, No.4, pp. 339-345, 1982. https://doi.org/10.2337/diab.31.4.339
32. [32] S. Akselrod, D. Gordon, F. A. Ubel, D. C. Shannon, A. C. Berger, and R. J. Cohen, “Power Spectrum Analysis of Heart Rate Fluctuation: A Quantitative Probe of Beat-to-Beat Cardiovascular Control,” Science, Vol.213, Issue 4504, pp. 220-222, 1981. https://doi.org/10.1126/science.6166045
33. [33] J. T. Bigger, J. L. Fleiss, L. M. Rolnitzky, and R. C. Steinman, “The Ability of Several Short-Term Measures of RR Variability to Predict Mortality after Myocardial Infarction,” Circulation, Vol.88, No.3, pp. 927-934, 1993. https://doi.org/10.1161/01.cir.88.3.927
34. [34] M. Toichi, T. Sugiura, T. Murai, and A. Sengoku, “A New Method of Assessing Cardiac Autonomic Function and Its Comparison with Spectral Analysis and Coefficient of Variation of R-R interval,” J. Auton. Nerv. Syst., Vol.62, Issues 1-2, pp. 79-84, 1997. https://doi.org/10.1016/s0165-1838(96)00112-9
35. [35] P. Guzik, J. Piskorski, T. Krauze, R. Schneider, K. H. Wesseling, A. Wykretowicz, and H. Wysocki, “Correlations between the Poincaré Plot and Conventional Heart Rate Variability Parameters Assessed during Paced Breathing,” The J. Physiol. Sci., Vol.57, Issue 1, pp. 63-71, 2007. https://doi.org/10.2170/physiolsci.RP005506
36. [36] W. J. Conover, “Practical Nonparametric Statistics, 3rd Edition,” Wiley, 1999.
37. [37] H. Watanabe, T.-Y. Wang, H. Ando, H. Mizushina, T. Morita, M. Emoto, T. Hatada, T. Bando, and H. Ujike, “Visually Induced Symptoms Questionnaire (VISQ): A Subjective Evaluation Method for Biomedical Effects Induced by Stereoscopic 3D Video,” Appl. Ergon., Vol.117, Article No.104238, 2024. https://doi.org/10.1016/j.apergo.2024.104238
38. [38] A. Graybiel and J. R. Lackner, “Evaluation of the Relationship between Motion Sickness Symptomatology and Blood Pressure, Heart Rate, and Body Temperature,” Aviat. Space Environ. Med., Vol.51, No.3, pp. 211-214, 1980.
39. [39] R. M. Stern, K. L. Koch, H. W. Leibowitz, I. M. Lindblad, C. L. Shupert, and W. R. Stewart, “Tachygastria and Motion Sickness,” Aviat. Space Environ. Med., Vol.56, No.11, pp. 1074-1077, 1985.
40. [40] P. S. Cowings, K. H. Naifeh, and W. B. Toscano, “The Stability of Individual Patterns of Autonomic Responses to Motion Sickness Stimulation,” Aviat. Space Environ. Med., Vol.61, No.5, pp. 399-405, 1990.
41. [41] J. C. Miller, T. J. Sharkey, G. A. Graham, and M. E. McCauley, “Autonomic Physiological Data Associated with Simulator Discomfort,” Aviat. Space Environ. Med., Vol.64, No.9, pp. 813-819, 1993.
42. [42] M. Tada, K. Hyodo, and H. Ujike, “Issues Related to the Physical Load When Wearing a Head-Mounted Displays,” Proc. of the 64th Conf. of Jpn. Hum. Factors and Ergon. Soc., Vol.59, Article No.S1B6-03, 2023 (in Japanese). https://doi.org/10.5100/jje.59.S1B6-03
43. [43] A. Somrak, I. Humar, M. S. Hossain, M. F. Alhamid, M. A. Hossain, and J. Guna, “Estimating VR Sickness and User Experience Using Different HMD Technologies: An Evaluation Study,” Future Gener. Comput. Syst., Vol.94, pp. 302-316, 2019. https://doi.org/10.1016/j.future.2018.11.041
44. [44] A. Borrego, J. Latorre, R. Llorens, M. Alcaniz, and E. Noe, “Feasibility of a Walking Virtual Reality System for Rehabilitation: Objective and Subjective Parameters,” J. Neuroeng. Rehabil., Vol.13, No.1, Article No.68, 2016. https://doi.org/10.1186/s12984-016-0174-1
45. [45] A. Kemeny, P. George, F. Merienne, and F. Colombet, “New VR Navigation Techniques to Reduce Cybersickness,” Electron. Imaging, Vol.29, Vol.3, pp. 48-53, 2017. https://doi.org/10.2352/ISSN.2470-1173.2017.3.ERVR-097
46. [46] K. Kim, M. Z. Rosenthal, D. J. Zielinski, and R. Brady, “Effects of Virtual Environment Platforms on Emotional Responses,” Comput. Methods and Programs in Biomed., Vol.113, Issue 3, pp. 882-893, 2014. https://doi.org/https://doi.org/10.1016/j.cmpb.2013.12.024
47. [47] K. K. K. Kwok, A. K. T. Ng, and H. Y. K. Lau, “Effect of Navigation Speed and VR Devices on Cybersickness,” 2018 IEEE Int. Symp. on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), 2018. https://doi.org/10.1109/ISMAR-Adjunct.2018.00041
48. [48] W. Fujisaki, “Effects of Delayed Visual Feedback on Grooved Pegboard Test Performance,” Front. Psycol., Vol.3, Article No.61, 2012. https://doi.org/10.3389/fpsyg.2012.00061
49. [49] Y. Matsumura, K. Okuda, J. Oyama, H. Endo, and H. Watanabe, “Display system, remote operation device, and display control method,” JP Patent 2024065932A, 2024.