Wheel loaders are construction machines that are mainly used for excavating and loading sedimented ground into dump trucks. The objects to be excavated range from large materials, such as blast rock and crushed stone, to small materials, such as gravel, slag, and coal ash. Therefore, the excavation operation of wheel loaders requires a high skill level to cope with various terrains and soil types. As worker numbers at quarry sites decline, developing highly automated technology to replace operators is crucial. In particular, the geometry of the ground to be excavated by the wheel loader changes with each excavation, so the control parameters must be adapted sequentially during automated excavation. In this study, we proposed an online learning method using Bayesian optimization to search for control parameters from multiple trials and modify them sequentially. In particular, we formulate a multi-objective optimization problem maximizing a weighted linear combination of the payload and workload as an objective function. To validate the proposed method, we constructed an environment in which repeated digging tests can be performed using a robot manipulator with a bucket attached. Through comparative tests between feed-forward control, in which the robot moves along a fixed trajectory independent of the digging reaction force, and off-line control, in which the robot modifies the digging trajectory in response to the digging reaction force, we compared the ability of these methods to cope with terrain volume that is different from that of the optimization trial.

1. [1] M. Shimizu and S. Matsumura, “Productivity Improvement by Visualization of Construction Machinery Operation,” Komatsu Technical Report, Vol.65, No.172, pp. 13-18, 2019.
2. [2] P. J. A. Lever, “An Automated Digging Control for a Wheel Loader,” Robotica, Vol.19, pp. 497-511, 2001. https://doi.org/10.1017/S0263574701003435
3. [3] J. Huang, D. Kong, G. Gao, X. Cheng, and J. Chen, “Data-Driven Reinforcement-Learning-Based Automatic Bucket-Filling for Wheel Loaders,” Applied Sciences, Vol.11, Issue 19, Article No.9191, 2021. https://doi.org/10.3390/app11199191
4. [4] S. Dadhich, U. Bodin, F. Sandin, and U. Andersson, “Machine Learning Approach to Automatic Bucket Loading,” 24th Mediterranean Conf. on Control and Automation, 2016. https://doi.org/10.1109/MED.2016.7535925
5. [5] S. Backman, D. Lindmark, K. Bodin, M. Servin, J. Mörk, and H. Löfgren, “Continuous Control of an Underground Loader Using Deep Reinforcement Learning,” Machines, Vol.9, Issue 10, Article No.216, 2021. https://doi.org/10.3390/machines9100216
6. [6] S. Dadhich, F. Sandin, U. Bodin, U. Andersson, and T. Martinsson, “Adaptation of a Wheel Loader Automatic Bucket Filling Neural Network Using Reinforcement Learning,” 2020 Int. Joint Conf. on Neural Networks, pp. 1-9, 2020. https://doi.org/10.1109/IJCNN48605.2020.9206849
7. [7] O. Azulay and A. Shapiro, “Wheel Loader Scooping Controller Using Deep Reinforcement Learning,” IEEE Access, Vol.9, pp. 24145-24154, 2021. https://doi.org/10.1109/ACCESS.2021.3056625
8. [8] K. Aoshima, M. Servin, and E. Wadbro, “Simulation-Based Optimization of High-Performance Wheel Loading,” Computational Engineering, Finance, and Science, 2021. https://doi.org/10.48550/arXiv.2107.14615
9. [9] H. Osumi, T. Uehara, and S. Sarata, “Estimation of Reaction Force from Rock Piles in Scooping Operation by Wheel Loader,” The 5th Int. Conf. on the Advanced Mechatronics, pp. 693-698, 2010. https://doi.org/10.1299/jsmeicam.2010.5.693
10. [10] D. Jud, G. Hottiger, P. Leemann, and M. Hutter, “Planning and Control for Autonomous Excavation,” IEEE Robotics and Automation Letters, Vol.2, No.4, pp. 2151-2158, 2017. https://doi.org/10.1109/LRA.2017.2721551
11. [11] T. Yuasa, M. Ishikawa, and S. Ogawa, “An Optimal Design Methodology for the Trajectory of Hydraulic Excavators Based on Genetic Algorithm,” J. Robot. Mechatron., Vol.33, No.6, pp. 1248-1254, 2021. https://doi.org/10.20965/jrm.2021.p1248
12. [12] H. Morita, M. Ishikawa, and S. Ogawa, “On-line Optimization of Repetitive Excavation Trajectories and Parameter Importance Analysis for Hydraulic Excavators,” The 9th Multi-Symp. on Control Systems, 3F1-2, 2022 (in Japanese).
13. [13] T. Suzuki, M. Ishikawa, and S. Ogawa, “A Study on Efficiency Improvement of Hydraulic Excavator Using Bayesian Optimization and Distinct Element Method,” The 9th Multi-Symp. on Control Systems, 3E1-5, 2022 (in Japanese).
14. [14] D. Jasrasaria and E. O. Pyzer-Knapp, “Dynamic Control of Explore/Exploit Trade-Off In Bayesian Optimization,” Intelligent Computing: Proc. of Computing Conf., 2018. https://doi.org/10.1007/978-3-030-01174-1_1
15. [15] J. Mockus, “Bayesian Approach to Global Optimization: Theory and Applications,” Springer, 2012. https://doi.org/10.1007/978-94-009-0909-0
16. [16] F. Berkenkamp, A. Krause, and A. P. Schoellig, “Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Robotics,” Machine Learning, Vol.112, pp. 3713-3747, 2021. https://doi.org/10.1007/s10994-021-06019-1
17. [17] R. Calandra, A. Seyfarth, J. Peters, and M. P. Deisenroth, “An Experimental Comparison of Bayesian Optimization for Bipedal Locomotion,” 2014 IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 1951-1958, 2014. https://doi.org/10.1109/ICRA.2014.6907117
18. [18] Y. Sui, A. Gotovos, J. Burdick, and A. Krause, “Safe Exploration for Optimization with Gaussian Processes,” Proc. of the 32nd Int. Conf. on Machine Learning, PMLR, Vol.37, pp. 997-1005, 2015.
19. [19] C. E. Rasmussen and C. K. I. Williams, “Gaussian Processes for Machine Learning,” The MIT Press, 2006. https://doi.org/10.7551/mitpress/3206.001.0001
20. [20] K. Kato, R. Ariizumi, and F. Matsuno, “Multiobjective Optimization Method for Expensive Function with Unknown Failure Regions and Its Application to Mobile Robot,” J. of the Robotics Society of Japan, Vol.35, No.2, pp. 143-152, 2017 (in Japanese). https://doi.org/10.7210/jrsj.35.143
21. [21] R. Ariizumi, M. Tesch, K. Kato, H. Choset, and F. Matsuno, “Multiobjective Optimization Based on Expensive Robotic Experiments Under Heteroscedastic Noise,” IEEE Trans. Robot, Vol.33, Issue 2, pp. 468-483, 2017. https://doi.org/10.1109/TRO.2016.2632739
22. [22] M. Koyama, M. Ishikawa, and H. Muranaka, “Explore-Exploit Strategy on Bayesian Optimization of Wheel-Loader Digging Control Parameters,” The 21th Symp. on Construction Robotics in Japan, O5-5, 2023 (in Japanese).
23. [23] H. Muranaka, M. Koyama, and M. Ishikawa, “Motion Design for Soil Excavation by Wheel Loaders Using Bayesian Optimization,” Trans. of the Institute of Systems, Control and Information Engineers, Vol.37, No.4 (in Japanese) (in press).