JRM Vol.36 No.2 pp. 309-319
doi: 10.20965/jrm.2024.p0309


Visual Presentation Interface to Reduce Effect of Machine Switching for Teleoperated Hydraulic Excavators

Masaki Nagai*1 ORCID Icon, Junya Masunaga*2, Masaru Ito*3, Chiaki Raima*4, Seiji Saiki*3, Yoichiro Yamazaki*3, and Yuichi Kurita*1

*1Graduate School of Advanced Science and Engineering, Hiroshima University
1-4-1 Kagamiyama, Higashi-hiroshima, Hiroshima 739-8527, Japan

*2Toshiba Infrastructure Systems & Solutions Corporation
1 Toshiba-cho, Fuchu, Tokyo 183-8511, Japan

*3Kobelco Construction Machinery Co., Ltd.
5-5-15 Kitashinagawa, Shinagawa-ku, Tokyo 141-8626, Japan

*4School of Human and Social Sciences, Tokyo International University
1-13-1 Matoba-Kita, Kawagoe, Saitama 350-1197, Japan

August 4, 2023
December 24, 2023
April 20, 2024
human augmentation, hydraulic excavator, interface, internal model, visual feedback

In the future, a situation in which operators switch between different machine classes in teleoperated hydraulic excavator is envisioned. In such a case, because the classes have different dynamic characteristics, the operator is expected to acquire an internal model for each class and switch models each time the operator switches between machines. However, in the case of teleoperated hydraulic excavator, the operator cannot obtain information such as the size and dynamic characteristics of the machine to be switched to; thus, the operator may not be able to switch internal models properly, which may affect the operation efficiency. Therefore, this study proposes a method in which images and videos are used to present the dynamic characteristics of the next machine to be operated during machine changeover in teleoperated hydraulic excavator. To verify the effectiveness of the proposed method, a simulator that imitates teleoperated hydraulic excavator was built and tested on test subjects. The swing operation time significantly increased when the machine was switched without presentation, compared with the case without switching. Meanwhile, the proposed method did not increase the swing operation time associated with machine switching, suggesting its effectiveness. The video presentation method was more effective than the image presentation method for suppressing the increase in swing operation time, indicating that the operator can immediately switch to an appropriate internal model with the presentation of the dynamic characteristics of the machine in advance using video.

Dynamic characteristics presentation method

Dynamic characteristics presentation method

Cite this article as:
M. Nagai, J. Masunaga, M. Ito, C. Raima, S. Saiki, Y. Yamazaki, and Y. Kurita, “Visual Presentation Interface to Reduce Effect of Machine Switching for Teleoperated Hydraulic Excavators,” J. Robot. Mechatron., Vol.36 No.2, pp. 309-319, 2024.
Data files:
  1. [1] K. Tateyama, “A New Stage of Construction in Japan – i-Construction,” IPA News Letter, Vol.2, Issue 2, pp. 2-11, 2017.
  2. [2] H. Sulaiman, M. N. A. Saadum, and A. A. Yusof, “Modern Manned, Unmanned and Teleoperated Excavator System,” J. of Mechanical Engineering and Technology, Vol.7, No.1, pp. 57-68, 2015.
  3. [3] M. Kawato, “Internal models for motor control and trajectory planning,” Current Opinion in Neurobiology, Vol.9, Issue 6, pp. 718-727, 1999.
  4. [4] B. Mehta and S. Schaal, “Forward models in visuomotor control,” J. of Neurophysiology, Vol.88, Issue 2, pp. 942-953, 2002.
  5. [5] D. M. Wolpert and M. Kawato, “Multiple paired forward and inverse models for motor control,” Neural Networks, Vol.11, Nos.7-8, pp. 1317-1329, 1998.
  6. [6] E. Nakano, J. R. Flanagan, H. Imamizu, R. Osu, T. Yoshioka, and M. Kawato, “Composition and decomposition learning of reaching movements under altered environments: An examination of the multiplicity of internal models,” Systems and Computers in Japan, Vol.33, No.11, pp. 80-94, 2002.
  7. [7] H. Imamizu, N. Sugimoto, R. Osu, K. Tsutsui, K. Sugiyama, Y. Wada, and M. Kawato, “Explicit contextual information selectively contributes to predictive switching of internal models,” Experimental Brain Research, Vol.181, pp. 395-408, 2007.
  8. [8] I. Hayashi, M. Ogino, S. Kita, and J. Leveille, “Recognition for Switching of Feedback and Feedforward Process in Motor Internal Model,” Proc. of the 9th EAI Int. Conf. on Bio-Inspired Information and Communications Technologies, pp. 245-246, 2016.
  9. [9] J. Masunaga, M. Ito, C. Raima, Y. Okawa, R. Sekizuka, S. Saiki, Y. Yamazaki, and Y. Kurita, “Effect of Switching from a Teleoperated Excavator with Different Size on Work Efficiency,” Proc. of the Augmented Humans Int. Conf. 2022, pp. 301-304, 2022.
  10. [10] T. Addou, N. Krouchev, and J. F. Kalaska, “Colored context cues can facilitate the ability to learn and to switch between multiple dynamical force fields,” J. of Neurophysioogy, Vol.106, Issue 1, pp. 163-183, 2011.
  11. [11] E. Matsuda, D. Misawa, S. Yano, and T. Kondo, “Olfactory Cues to Reduce Retrograde Interference During the Simultaneous Learning of Conflicting Motor Tasks,” J. Robot. Mechatron., Vol.34, No.4, pp. 746-755, 2022.
  12. [12] N. Cothros, J. Wong, and P. L. Gribble, “Visual cues signaling object grasp reduce interference in motor learning,” J. of Neurophysiology, Vol.102, Issue 4, pp. 2112-2120, 2009.
  13. [13] I. S. Howard, D. M. Wolpert, and D. W. Franklin, “The effect of contextual cues on the encoding of motor memories,” J. of Neurophysiology, Vol.109, Issue 10, pp. 2632-2644, 2013.
  14. [14] K. Koiwai, R. Miyazaki, T. Yamamoto, K. Ueda, K. Yamashita, and Y. Yamazaki, “Responsiveness evaluation index for an excavator operation based on control engineering approach,” IEEJ Trans. Elec. Info. Sys., Vol.138, No.5, pp. 506-511, 2018 (in Japanese).
  15. [15] R. Sekizuka, M. Ito, C. Raima, S. Saiki, Y. Yamazaki, and Y. Kurita, “Force feedback design of operation levers considering the characteristics of human force perception to improve hydraulic excavator operability,” IEEE Access, Vol.10, pp. 926-938, 2021.
  16. [16] M. Ito, C. Raima, S. Saiki, Y. Yamazaki, and Y. Kurita, “Effects of Machine Instability Feedback on Safety During Digging Operation in Teleoperated Excavators,” IEEE Access, Vol.9, pp. 28987-28998, 2021.
  17. [17] K. Matsumura, M. Ito, S. Saiki, Y. Yamazaki, and Y. Kurita, “Influence of Restrictions on Range of View from Cockpit in Operation of Hydraulic Excavator,” IEEE Access, Vol.8, pp. 90520-90527, 2020.
  18. [18] H. Sakoe and S. Chiba, “Dynamic programming algorithm optimization for spoken word recognition,” IEEE Trans. on Acoustics, Speech and Signal Processing, Vol.26, No.1, pp. 43-49, 1978.
  19. [19] F. Petitjean, A. Ketterlin, and P. Gancarski, “A global averaging method for dynamic time warping, with applications to clustering,” Pattern Recognition, Vol.44, No.3, pp. 678-693, 2011.
  20. [20] S. Holm, “A simple sequentially rejective multiple test procedure,” Scandinavian J. of Statistics, Vol.6, No.2, pp. 65-70, 1979.
  21. [21] K. Koiwai, T. Yamamoto, R. Miyazaki, K. Ueda, K. Yamashita, and Y. Yamazaki, “A consideration on responsiveness evaluation for an excavator based on control engineering approach,” Proc. of the 2017 IEEE Int. Conf. on Systems, Man, and Cybernetics (SMC), pp. 467-470, 2017.

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on May. 19, 2024