We propose a robotic forklift system for stacking multiple mesh pallets. The stacking of mesh pallets is an essential task for the shipping and storage of loads. However, stacking, the placement of pallet feet on pallet edges, is a complex problem owing to the small sizes of the feet and edges, leading to a complexity in the detection and the need for high accuracy in adjusting the pallets. To detect the pallets accurately, we utilize multiple RGB-D (RGB Depth) cameras that produce dense depth data under the limitations of the sensor position. However, the depth data contain noise. Hence, we implement a region growing-based algorithm to extract the pallet feet and edges without removing them. In addition, we design the control law based on path following control for the forklift to adjust the position and orientation of two pallets. To evaluate the performance of the proposed system, we conducted an experiment assuming a real task. The experimental results demonstrated that the proposed system can achieve a stacking operation with a real forklift and mesh pallets.

1. [1] P. Tokekar, J. Vander Hook, D. Mulla, and V. Isler, “Sensor planning for a symbiotic uav and ugv system for precision agriculture,” Proc. of IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 5321-5326, 2013.
2. [2] T. Yoshida, T. Fukao, and T. Hasegawa, “Cutting point detection using a robot with point clouds for tomato harvesting,” J. Robot. Mechatron., Vol.32, No.2, pp. 437-444, 2020.
3. [3] F. Magistri, N. Chebrolu, and C. Stachniss, “Segmentation-based 4d registration of plants point clouds for phenotyping,” Proc. of IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 2433-2439, 2020.
4. [4] T. Fujinaga, S. Yasukawa, and K. Ishii, “Tomato growth state map for the automation of monitoring and harvesting,” J. Robot. Mechatron., Vol.32, No.6, pp. 1279-1291, 2020.
5. [5] Y. Hamaguchi and P. Raksincharoensak, “Automated steering control system for reverse parking maneuver of semi-trailer vehicles considering motion planning by simulation of feedback control system,” J. Robot. Mechatron., Vol.32, No.3, pp. 561-570, 2020.
6. [6] R. Adams and L. Bischof, “Seeded region growing,” IEEE Trans. Pattern Anal. Mach. Intell., Vol.16, No.6, pp. 641-647, 1994.
7. [7] Y. Kanayama, Y. Kimura, F. Miyazaki, and T. Noguchi, “A stable tracking control method for an autonomous mobile robot,” Proc. of IEEE Int. Conf. on Robotics and Automation, Vol.1, pp. 384-389, 1990.
8. [8] R. Varga and S. Nedevschi, “Robust pallet detection for automated logistics operations,” Proc. of the 11th Joint Conf. on Computer Vision, Imaging and Computer Graphics Theory and Applications, pp. 470-477, 2016.
9. [9] R. Cucchiara, M. Piccardi, and A. Prati, “Focus based feature extraction for pallets recognition,” Proc. of British Machine Vision Conf., pp. 70.1-70.10, 2000.
10. [10] L. Baglivoand, N. Biasi, F. Biral, N. Bellomo, E. Bertolazzi, D. Lio, and M. Cecco, “Autonomous pallet localization and picking for industrial forklifts: A robust range and look method,” Measurement Science and Technology, Vol.22, 085502, 2011.
11. [11] T. Tamba, B. Hong, and K. Hong, “A path following control of an unmanned autonomous forklift,” Int. J. of Control, Automation and Systems, Vol.7, pp. 113-122, 2009.
12. [12] A. Mohammadi, I. Mareels, and D. Oetomo, “Model predictive motion control of autonomous forklift vehicles with dynamics balance constraint,” Proc. of 2016 14th Int. Conf. on Control, Automation, Robotics and Vision, pp. 1-6, 2016.
13. [13] B. Molter and J. Fottner, “Semi-automatic pallet pick-up as an advanced driver assistance system for forklifts,” Proc. of IEEE Intelligent Transportation Systems Conf., pp. 4464-4469, 2019.
14. [14] A. Correa, M. Walter, L. Fletcher, J. Glass, S. Teller, and R. Davis, “Multimodal interaction with an autonomous forklift,” Proc. of the ACM/IEEE Int. Conf. on Human-Robot Interaction, pp. 243-250, 2010.
15. [15] D. Haanpaa, G. Beach, and C. J. Cohen, “Machine vision algorithms for robust pallet engagement and stacking,” Proc. of IEEE Applied Imagery Pattern Recognition Workshop, pp. 1-8, 2016.
16. [16] B. Molter and J. Fottner, “Real-time pallet localization with 3d camera technology for forklifts in logistic environments,” Proc. of IEEE Int. Conf. on Service Operations and Logistics, and Informatics, pp. 297-302, 2018.
17. [17] J. Frahm, J. Matas, M. Pollefeys, O. Chum, and R. Raguram, “Usac: A universal framework for random sample consensus,” IEEE Trans. Pattern Anal. Mach. Intell., Vol.35, pp. 2022-2038, 2013.
18. [18] J. Xiao, H. Lu, L. Zhang, and J. Zhang, “Pallet recognition and localization using an rgb-d camera,” Int. J. of Advanced Robotic Systems, Vol.14, No.6, 1729881417737799, 2017.
19. [19] R. Iinuma, Y. Kojima, H. Onoyama, T. Fukao, S. Hattori, and Y. Nonogaki, “Pallet handling system with an autonomous forklift for outdoor fields,” J. Robot. Mechatron., Vol.32, No.5, pp. 1071-1079, 2020.
20. [20] K. S. Arun, T. S. Huang, and S. D. Blostein, “Least-squares fitting of two 3-d point sets,” IEEE Trans. Pattern Anal. Mach. Intell., Vol.PAMI-9, No.5, pp. 698-700, 1987.
21. [21] J. Rehder, J. Nikolic, T. Schneider, T. Hinzmann, and R. Siegwart, “Extending kalibr: Calibrating the extrinsics of multiple imus and of individual axes,” Proc. of IEEE Int. Conf. on Robotics and Automation, pp. 4304-4311, 2016.
22. [22] R. Bostelman, T. Hong, and T. Chang, “Visualization of pallets,” Intelligent Robots and Computer Vision XXIV: Algorithms, Techniques, and Active Vision, Vol.6384, pp. 78-89, 2006.
23. [23] J. Pagès, X. Armangué, J. Salvi, J. Freixenet, and J. Martí, “A computer vision system for autonomous forklift vehicles in industrial environments,” Proc. of the Mediterranean Conf. on Control and Automation, pp. 1-6, 2001.