The aim of this work is to develop a technology that allows a remote operator of construction machine to feel the situations in a real working site to prevent fall accidents. In tele-operated maneuvering construction machine, it is difficult to recognize the tilt of the vehicle using only images from a camera mounted on the remote vehicle. Therefore, this study focuses on transmitting the feeling of the tilt using a controller with tactile stimulation. A gamepad-type tactile controller that performs palm pressurization is utilized to provide the tactile stimulus. The vehicle’s tilt is expressed by the palm pressure, which changes in corresponding to the vehicle’s pitch and roll angle. This study involves an experiment in which 10 subjects operate a vehicle remotely to climb on a slope. The subjects reported the tilt of the slope felt during the operation. The reported tilt is compared with those obtained by camera images only. The experiment results show that the accuracy of the recognized tilt was improved by 31.7% by utilizing a tactile stimulus when compared with the case involving operation using vision only. A subjective evaluation is performed using a five-point scale questionnaire. The results confirmed that the feeling of tilt, which is difficult to transmit using only video, was improved by 34%. This is an effective technology that transmits the feelings experienced in the remote field in real time. The proposed technology is thus expected to be useful for further development of teleworking technologies.

1. [1] Construction Dive, “Making up for the construction labor shortage with technology,” https://www.constructiondive.com/spons/making-up-for-the-construction-labor-shortage-with-technology/556147/ [accessed April 22, 2021]
2. [2] Japan Science and Technology Agency (JST), “JST ACCEL Embodied Media project,” https://www.jst.go.jp/kisoken/accel/en/brochure/accel_en_pamph_project_2019_27.pdf [accessed April 22, 2021]
3. [3] S. Miura, K. Hamamoto, and I. Kuronuma, “Next Generation Construction Production System Focusing on Automation Technologies of Construction Machines,” The 7th Civil Engineering Conf. in the Asian Region Proc., 2016.
4. [4] NTT docomo, “Smart Construction Powered by 5G & IoT,” https://www.nttdocomo.co.jp/english/info/media_center/event/mwc19/pdf/about_smart_construction_5g.pdf [accessed April 22, 2021]
5. [5] E. E. Kazan, “Analysis of Fatal and Nonfatal Accidents Involving Earthmoving Equipment Operators And On-Foot Workers,” Ph.D. Thesis, Wayne State University, 2013.
6. [6] Ministry of Health, Labor and Welfare, Workplace Safety Site, “Mechanical Disaster Database,” https://anzeninfo.mhlw.go.jp/anzen/sai/kikaisaigai.html (in Japanese) [accessed April 22, 2021]
7. [7] K. Suzuki and H. Jansson, “An analysis of driver’s steering behaviour during auditory or haptic warnings for the designing of lane departure warning system,” Society of Automotive Engineers of Japan (JSAE) Review, Vol.24, Issue 1, pp. 65-70, 2003.
8. [8] T. Sasaki, T. Nagai, and K. Kawashima, “Remote Control of Backhoe for Rescue Activities Using Pneumatic Robot System,” Proc. of the 2006 IEEE Int. Conf. on Robotics and Automation, pp. 3177-3182, 2006.
9. [9] D. Shin, M.-S. Kang, S. Lee, and C. Han, “Development of Remote Controlled Manipulation Device for a Conventional Excavator without Renovation,” 2012 IEEE/SICE Int. Symp. on System Integration (SII), 2012.
10. [10] S. Okishiba, R. Fukui, M. Takagi, H. Azumi, S. Warisawa, R. Togashi, H. Kitaoka, and T. Ooi, “Tablet interface for direct vision teleoperation of an excavator for urban construction work,” Automation in Construction, Vol.102, pp. 17-26, 2019.
11. [11] R. Sekizuka, M. Ito, S. Saiki, Y. Yamazaki, and Y. Kurita, “System to Evaluate the Skill of Operating Hydraulic Excavators Using a Remote Controlled Excavator and Virtual Reality,” Frontiers in Robot and AI, Vol.6, Article No.142, 2020.
12. [12] J. Uusisalo, O. Karhu, and K. Huhtala, “Assisting Force Feedback Function for Hand-Held Remote Control of Excavator,” Proc. of the 7th JFPS Int. Symp. on Fluid Power, pp. 653-658, 2008.
13. [13] H. Takenouchi, N. Cao, H. Nagano, M. Konyo, and S. Tadokoro, “Extracting Haptic Information from High-Frequency Vibratory signals Measured on a Remote Robot to Transmit Collisions with Environments,” Proc. of the 2017 IEEE/SICE Int. Symp. on System Integration (SIII), pp. 968-973, 2017.
14. [14] H. Nagano, H. Takenouchi, N. Cao, M. Konyo, and S. Tadokoro, “Tactile feedback system of high-frequency vibration signals for supporting delicate teleoperation of construction robots,” Advanced Robotics, Vol.34, No.11, pp. 730-743, 2020.
15. [15] H. Kajimoto, H. Ando, and K.-U. Kyung (Eds.), “Haptic Interaction,” Springer Japan, 2015.
16. [16] T. Asada, T. Nakamura, and A. Yamamoto, “Investigation on Substitution of Force Feedback using Pressure Stimulation to Palm,” Proc. of 2016 IEEE Haptics Symp., pp. 389-390, 2016
17. [17] T. Nakamura, S. Nemoto, T. Ito, and A. Yamamoto, “Substituted force feedback using palm pressurization for a handheld controller,” Proc. of 2016 IEEE Haptics Symp., pp. 197-199, 2016.
18. [18] TAMIYA, Inc., “Remote controlled bulldozer kit,” https://www.tamiya.com/japan/products/70104/index.html (in Japanese) [accessed April 22, 2021]
19. [19] H. Sakaniwa, S. Sutoko, A. Obata, H. Atsumori, N. Fukuda, M. Kiguchi, and A. Kandori, “Effects of Shape Characteristics on Tactile Sensing Recognition and Brain Activation,” J. Adv. Comput. Intell. Intell. Inform., Vol.23, No.6, pp. 1080-1088, doi: 10.20965/jaciii.2019.p1080, 2019.