In Japan, the quantity of domestically produced fruit has been gradually decreasing, while wholesale prices have continued to rise due to declining production volumes and a shift toward high-quality varieties. To address these trends, improving quality and reducing labor through automation have become urgent challenges. In precision viticulture, monitoring the growth of grape clusters plays a key role in yield estimation, disease management, and optimal harvest timing. Although recent advances in deep learning and 3D reconstruction have enabled accurate fruit detection and modeling in vineyards, tracking the same clusters on different days remains challenging because of branch movement, fruit growth, and varying imaging conditions. This study proposes a branch-based 3D alignment framework for the cross-day tracking of grape clusters. Stable vine structures, such as trunks and main branches, are reconstructed using Structure from Motion, and their spatial correspondences are estimated through SIFT-based matching and similarity transformation. Once the coordinate systems of different days are aligned, the grape clusters detected by CenterNet are associated based on spatial proximity in the unified 3D space. Experiments over multiple observation days demonstrated that the proposed method successfully maintained the consistent tracking of grape clusters throughout the growth period. These results indicate that branch-based alignment effectively stabilizes multi-day observations and facilitates the temporal monitoring of fruit growth, supporting automated phenotyping and future field robot applications in viticulture.

1. [1] T. T. Santos, L. P. Koenigkan, E. T. Botelho, and R. E. T. Sarro, “Grape detection, segmentation, and tracking using deep neural networks and three-dimensional association,” Computers and Electronics in Agriculture, Vol.170, Article No.105247, 2020. https://doi.org/10.1016/j.compag.2020.105247
2. [2] S. Nuske, S. Achar, T. Bates, S. Narasimhan, and S. Singh, “Yield estimation in vineyards by visual grape detection,” 2011 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 2352-2358, 2011. https://doi.org/10.1109/IROS.2011.6095069
3. [3] N. Shakoor, S. Lee, and T. Mockler, “High throughput phenotyping to accelerate crop breeding and monitoring of diseases in the field,” Current Opinion in Plant Biology, Vol.38, pp. 184-192, 2017. https://doi.org/10.1016/j.pbi.2017.05.006
4. [4] N. Behroozi-Khazaei and M. R. Maleki, “A robust algorithm based on color features for grape cluster segmentation,” Computers and Electronics in Agriculture, Vol.142, pp. 41-49, 2017. https://doi.org/10.1016/j.compag.2017.08.025
5. [5] J. Torres-Sánchez, F. J. Mesas-Carrascosa, L.-G, Santesteban, F. M. Jiménez-Brenes, O. Oneka, A. Villa-Llop, M. Loidi, and F. López-Granados, “Grape cluster detection using uav photogrammetric point clouds as a low-cost tool for yield forecasting in vineyards,” Sensors, Vol.21, No.9, Article No.3083, 2021. https://doi.org/10.3390/s21093083
6. [6] I. Sa, Z. Ge, F. Dayoub, B. Upcroft, T. Perez, and C. McCool, “Deepfruits: A fruit detection system using deep neural networks,” Sensors, Vol.16, No.8, Article No.1222, 2016. https://doi.org/10.3390/s16081222
7. [7] S. Bargoti and J. Underwood, “Deep fruit detection in orchards,” 2017 IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 3626-3633, 2017. https://doi.org/10.1109/ICRA.2017.7989417
8. [8] T. Fujinaga, S. Yasukawa, B. Li, and K. Ishii, “Image mosaicing using multi-modal images for generation of tomato growth state map,” J. of Robot. and Mechatron., Vol.30, No.2, pp. 187-197, 2018. https://doi.org/10.20965/jrm.2018.p0187
9. [9] T. Fujinaga, S. Yasukawa, and K. Ishii, “Tomato growth state map for the automation of monitoring and harvesting,” J. of Robot. and Mechatron., Vol.32, No.6, pp. 1279-1291, 2020. https://doi.org/10.20965/jrm.2020.p1279
10. [10] Y. Pan, F. Magistri, T. Labe, E. Marks, C. Smitt, C. McCool, J. Behley, and C. Stachniss, “Panoptic mapping with fruit completion and pose estimation for horticultural robots,” 2023 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), pp. 4226-4233, 2023. https://doi.org/10.1109/IROS55552.2023.10342067
11. [11] C. Liang, J. Xiong, Z. Zheng, Z. Zhong, Z. Li, S. Chen, and Z. Yang, “A visual detection method for nighttime litchi fruits and fruiting stems,” Computers and Electronics in Agriculture, Vol.169, Article No.105192, 2020. https://doi.org/10.1016/j.compag.2019.105192
12. [12] L. Lobefaro, M. Sodano, D. Fusaro, F. Magistri, M. V. R. Malladi, T. Guadagnino, A. Pretto, and C. Stachniss, “Spatio-temporal consistent semantic mapping for robotics fruit growth monitoring,” IEEE Robotics and Automation Letters, Vol.10, No.9, pp. 9470-9477, 2025. https://doi.org/10.1109/LRA.2025.3594985
13. [13] N. Noguchi, “Agricultural vehicle robot,” J. of Robot. and Mechatron., Vol.30, No.2, pp. 165-172, 2018. https://doi.org/10.20965/jrm.2018.p0165
14. [14] S. Nishiwaki, H. Kondo, S. Yoshida, and T. Emaru, “Proposal of uav-slam-based 3d point cloud map generation method for orchards measurements,” J. of Robot. and Mechatron., Vol.36, No.5, pp. 1001-1009, 2024. https://doi.org/10.20965/jrm.2024.p1001
15. [15] L. Shen, J. Su, R. He, L. Song, R. Huang, Y. Fang, Y. Song, and B. Su, “Real-time tracking and counting of grape clusters in the field based on channel pruning with yolov5s,” Computers and Electronics in Agriculture, Vol.206, Article No.107662, 2023. https://doi.org/10.1016/j.compag.2023.107662
16. [16] M. Ariza-Sentís, H. Baja, S. Velez, and J. Valente, “Object detection and tracking on uav rgb videos for early extraction of grape phenotypic traits,” Computers and Electronics in Agriculture, Vol.211, Article No.108051, 2023. https://doi.org/10.1016/j.compag.2023.108051
17. [17] K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask R-CNN,” Proc. IEEE Int. Conf. on Computer Vision (ICCV), pp. 2980-2988, 2017. https://doi.org/10.1109/ICCV.2017.322
18. [18] X. Zhou, D. Wang, and P. Krähenbühl, “Objects as points,” arXiv:1904.07850, 2019. https://doi.org/10.48550/arXiv.1904.07850
19. [19] P. Moulon, P. Monasse, R. Perrot, and R. Marlet, “OpenMVG: Open multiple view geometry,” Int. Workshop on Reproducible Research in Pattern Recognition, pp. 60-74, 2016. https://doi.org/10.1007/978-3-319-56414-2_5
20. [20] J. L. Schönberger and J.-M. Frahm, “Structure-from-motion revisited,” Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 4104-4113, 2016. https://doi.org/10.1109/CVPR.2016.445
21. [21] D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. of Computer Vision, Vol.60, No.2, pp. 91-110, 2004. https://doi.org/10.1023/B:VISI.0000029664.99615.94
22. [22] S. Umeyama, “Least-squares estimation of transformation parameters between two point patterns,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.13, No.4, pp. 376-380, 1991. https://doi.org/10.1109/34.88573
23. [23] M. A. Fischler and R. C. Bolles, “Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography,” Communications of the ACM, Vol.24, No.6, pp. 381-395, 1981. https://doi.org/10.1145/358669.358692
24. [24] “The situation surrounding fruit trees.” https://www.maff.go.jp/j/seisan/ryutu/fruits/attach/pdf/index-184.pdf [Accessed October 16, 2025]
25. [25] “Basic policy for promoting fruit tree agriculture.” https://www.maff.go.jp/j/seisan/ryutu/fruits/attach/pdf/index-198.pdf [Accessed October 16, 2025]