Cutting Point Detection Using a Robot with Point Clouds for Tomato Harvesting
Takeshi Yoshida*, Takanori Fukao*, and Takaomi Hasegawa**
1-1-1 Noji-higashi, Kusatsu, Shiga 525-8577, Japan
1-1 Syowa-cho, Kariya, Aichi 448-8661, Japan
This paper proposes a method to detect cutting points on tomato peduncles using a harvesting robot. The main objective of this study was to develop automated harvesting robots. The harvesting robot was equipped with an RGB-D (Red, Blue, Green, and Depth) camera to detect peduncles and an end effector to harvest tomatoes. Robots must be able to detect where to cut crops during harvesting. The proposed method was used to detect the cutting points on peduncles using a point cloud captured by the RGB-D camera. Our robot was used to identify the cutting points on target tomato peduncles at an actual farm to demonstrate the effectiveness of our approach experimentally. Using the proposed method, the harvesting robot could detect the cutting points on tomatoes.
-  S. Bachche and K. Oka, “Design, Modeling and Performance Testing of End-Effector for Sweet Pepper Harvesting Robot Hand,” J. Robot. Mechatron., Vol.25, No.4, pp. 705-717, 2013.
-  H. Yaguchi, K. Nagahama, T. Hasegawa, and M. Inaba, “Development of an autonomous tomato harvesting robot with rotational plucking gripper,” Proc. of 2016 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), pp. 652-657, 2016.
-  E. J. Van Henten, J. Hemming, B. A. J. Van Tuijl, J. G. Kornet, J. Meuleman, J. Bontsema, and E. A. van Os, “An Autonomous Robot for Harvesting Cucumbers in Greenhouses,” Autonomous Robots, Vol.13, No.3, pp. 241-258, 2002.
-  E. J. Van Henten, B. A. J. Van Tuijl, J. Hemming, J. G. Kornet, J. Bontsema, and E. A. Van Os, “Field Test of an Autonomous Cucumber Picking Robot,” Biosystems Engineering, Vol.86, No.3, pp. 305-313, 2003.
-  E. J. Van Henten, B. A. J. Van Tuijl, G.-J. Hoogakker, M. J. Van Der Weerd, J. Hemming, J. G. Kornet, and J. Bontsema, “An Autonomous Robot for De-leafing Cucumber Plants grown in a High-wire Cultivation System,” Biosystems Engineering, Vol.94, No.3, pp. 317-323, 2006.
-  X. Chen, K. Chaudhary, Y. Tanaka, K. Nagahama, H. Yaguchi, K. Okada, and M. Inaba, “Reasoning-based vision recognition for agricultural humanoid robot toward tomato harvesting,” Proc. of 2015 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), pp. 6487-6494, 2015.
-  Y. Tokunaga, T. Hakukawa, and T. Inoue, “Algorithm and Design of an Intelligent Digital Integrated Circuit for a Watermelon Harvesting Robot,” J. Robot. Mechatron., Vol.11, No.3, pp. 173-182, 1999.
-  Y. Si, G. Liu, and J. Feng, “Location of apples in trees using stereoscopic vision,” Computers and Electronics in Agriculture, Vol.112, pp. 68-74, 2015.
-  M. Monta, N. Kondo, S. Arima, and K. Namba, “Robotic Vision for Bioproduction Systems,” J. Robot. Mechatron., Vol.15, No.3, pp. 341-348, 2003.
-  T. Fujinaga, S. Yasukawa, B. Li, and K. Ishii, “Image Mosaicing Using Multi-Modal Images for Generation of Tomato Growth State Map,” J. Robot. Mechatron., Vol.30, No.2, pp. 187-197, 2018.
-  R. Fukui, K. Kawae, and S. Warisawa, “Development of a Tomato Volume Estimating Robot that Autonomously Searches an Appropriate Measurement Position – Basic Feasibility Study Using a Tomato Bed Mock-Up –,” J. Robot. Mechatron., Vol.30, No.2, pp. 173-179, 2018.
-  I. Sa, C. Lehnert, A. English, C. McCool, F. Dayoub, B. Upcroft, and T. Perez, “Peduncle Detection of Sweet Pepper for Autonomous Crop Harvesting – Combined Color and 3-D Information,” IEEE Robotics and Automation Letters, Vol.2, No.2, pp. 765-772, 2017.
-  L. Luo, Y. Tang, X. Zou, M. Ye, W. Feng, and G. Li, “Vision-based extraction of spatial information in grape clusters for harvesting robots,” Biosystems Engineering, Vol.151, pp. 90-104, 2016.
-  N. Irie, N. Taguchi, T. Horie, and T. Ishimatsu, “Development of Asparagus Harvester Coordinated with 3-d Vision Sensor,” J. Robot. Mechatron., Vol.21, No.5, pp. 583-589, 2009.
-  N. Irie and N. Taguchi, “Asparagus Harvesting Robot,” J. Robot. Mechatron., Vol.26, No.2, pp. 267-268, 2014.
-  N. Kondo and M. Monta. “Fruit Harvesting Robotics,” J. Robot. Mechatron., Vol.11, No.4, pp. 321-268, 1999.
-  S. Hayashi, K. Shigematsu, S. Yamamoto, K. Kobayashi, Y. Kohno, J. Kamata, and M. Kurita, “Evaluation of a strawberry-harvesting robot in a field test,” Biosystems Engineering, Vol.105, No.2, pp. 160-171, 2010.
-  S. Arima and N. Kondo. “Cucumber Harvesting Robot and Plant Training System,” J. Robot. Mechatron., Vol.11, No.3, pp. 208-212, 1999.
-  L. L. Wang, B. Zhao, J. W. Fan, X. A. Hu, S. Wei, Y. S. Li, Q. B. Zhou, and C. F. Wei, “Development of a tomato harvesting robot used in greenhouse,” Int. J. of Agricultural and Biological Engineering, Vol.10, No.4, pp. 140-149, 2017.
-  T. Yoshida, T. Fukao, and T. Hasegawa, “Fast Detection of Tomato Peduncle Using Point Cloud with a Harvesting Robot,” J. Robot. Mechatron., Vol.30, No.2, pp. 180-186, 2018.
-  R. Stephens, “Essential Algorithms: A Practical Approach to Computer Algorithm,” Wiley, 2013.
-  R. Sedgewick, “Algorithms in C++ Part 5: Graph Algorithms (3rd edition),” Addison-Wesley Professional, 2001.