The objective of this study is to achieve high efficiency in autonomous hydraulic excavators by imitating the bucket trajectory operated by an expert. For this purpose, bucket trajectories of experts were collected, and a trajectory was planned using machine learning of a model that relates measured soil shapes to the bucket trajectories of the experts. In this study, we proposed a hierarchical model consisting of a model for estimating movement and a trajectory, with a focus on the fact that different trajectories are generated for the same soil shape as a result of the analysis of the skilled persons’ movements. The trajectory output from the model was replanned to have a smooth trajectory using numerical optimization. For the backfilling task, the error from the target shape and the amount of soil transported per movement were compared with those of an expert. The proposed method increased the error from the target shape by approximately 66%, while the amount of soil transported was approximately 58% of that of the experts.

1. [1] Y. Sakaida, D. Chugo, K. Kawabata, H. Kaetsu, and H. Asama, “The analysis of skillful hydraulic excavator operation,” Proc. of the 23rd Annual Conf. of the Robotics Society of Japan, Article No.3I21, 2005 (in Japanese).
2. [2] Y. Sakaida, D. Chugo, K. Kawabata, H. Kaetsu, and H. Asama, “The analysis of skillful hydraulic excavator operation,” Proc. of the 2006 JSME Conf. on Robotics and Mechatronics, Article No.2A1-B01, 2006 (in Japanese). https://doi.org/10.1299/jsmermd.2006._2A1-B01_1
3. [3] I. Kurinov, G. Orzechowski, P. Hämäläinen, and A. Mikkola, “Automated excavator based on reinforcement learning and multibody system dynamics,” IEEE Access, Vol.8, pp. 213998-214006, 2020. https://doi.org/10.1109/ACCESS.2020.3040246
4. [4] D. Jud, P. Leemann, S. Kerscher, and M. Hutter, “Autonomous free-form trenching using a walking excavator,” IEEE Robotics and Automation Letters, Vol.4, No.4, pp. 3208-3215, 2019. https://doi.org/10.1109/LRA.2019.2925758
5. [5] Y. Yang, P. Long, X. Song, J. Pan, and L. Zhang, “Optimization-based framework for excavation trajectory generation,” IEEE Robotics and Automation Letters, Vol.6, No.2, pp. 1479-1486, 2021. https://doi.org/10.1109/LRA.2021.3058071
6. [6] T. Osa et al., “An algorithmic perspective on imitation learning,” Foundations and Trends® in Robotics, Vol.7, Nos.1-2, 2018. https://doi.org/10.1561/2300000053
7. [7] B. Son, C. U. Kim, C. Kim, and D. Lee, “Expert-emulating excavation trajectory planning for autonomous robotic industrial excavator,” 2020 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), pp. 2656-2662, 2020. https://doi.org/10.1109/IROS45743.2020.9341036
8. [8] K. Çelikbilek, “Learning from demonstration with probabilistic movement primitives on Komatsu excavator,” Master’s thesis, Tampere University, 2020.
9. [9] C. Mastalli, J. Cappelletto, R. Acuña, A. Terrones, and G. Fernández-López, “An imitation learning approach for truck loading operations in backhoe machines,” Adaptive Mobile Robotics: Proc. of the 15th Int. Conf. on Climbing and Walking Robots and the Support Technologies for Mobile Machines, pp. 821-830, 2012. https://doi.org/10.1142/9789814415958_0104
10. [10] S. Schaal, “Dynamic movement primitives – A framework for motor control in humans and humanoid robotics,” H. Kimura, K. Tsuchiya, A. Ishiguro, and H. Witte (Eds.), “Adaptive Motion of Animals and Machines,” pp. 261-280, Springer, 2006. https://doi.org/10.1007/4-431-31381-8_23
11. [11] A. Paraschos, C. Daniel, J. Peters, and G. Neumann, “Probabilistic movement primitives,” Proc. of the 26th Int. Conf. on Neural Information Processing Systems (NIPS’13), Vol.2, pp. 2616-2624, 2013.
12. [12] D. A. Pomerleau, “ALVINN: An autonomous land vehicle in a neural network,” Proc. of the 1st Int. Conf. on Neural Information Processing Systems (NIPS’88), pp. 305-313, 1988.
13. [13] F. Codevilla, M. Müller, A. López, V. Koltun, and A. Dosovitskiy, “End-to-end driving via conditional imitation learning,” 2018 IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 4693-4700, 2018. https://doi.org/10.1109/ICRA.2018.8460487
14. [14] E. Ohn-Bar, A. Prakash, A. Behl, K. Chitta, and A. Geiger, “Learning situational driving,” 2020 IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 11293-11302, 2020. https://doi.org/10.1109/CVPR42600.2020.01131
15. [15] A. Sasagawa, K. Fujimoto, S. Sakaino, and T. Tsuji, “Imitation learning based on bilateral control for human–robot cooperation,” IEEE Robotics and Automation Letters, Vol.5, No.4, pp. 6169-6176, 2020. https://doi.org/10.1109/LRA.2020.3011353
16. [16] T. Ogata, “Deep learning and manipulation,” J. of the Robotics Society of Japan, Vol.35, No.1, pp. 28-31, 2017 (in Japanese). https://doi.org/10.7210/jrsj.35.28
17. [17] P. Florence et al., “Implicit behavioral cloning,” Proc. of the 5th Conf. on Robot Learning (CoRL 2021), pp. 158-168, 2022. https://doi.org/10.48550/arXiv.2109.00137
18. [18] C. Badue et al., “Self-driving cars: A survey,” Expert Systems with Applications, Vol.165, Article No.113816, 2021. https://doi.org/10.1016/j.eswa.2020.113816
19. [19] J. Ziegler, P. Bender, T. Dang, and C. Stiller, “Trajectory planning for Bertha – A local, continuous method,” Proc. of the 2014 IEEE Intelligent Vehicles Symp., pp. 450-457, 2014. https://doi.org/10.1109/IVS.2014.6856581
20. [20] K. Noda, H. Arie, Y. Suga, and T. Ogata, “Multimodal integration learning of robot behavior using deep neural networks,” Robotics and Autonomous Systems, Vol.62, No.6, pp. 721-736, 2014. https://doi.org/10.1016/j.robot.2014.03.003
21. [21] J. Nocedal and S. J. Wright, “Numerical Optimization,” Springer, 2006. https://doi.org/10.1007/978-0-387-40065-5