Digital image phenotyping has become popular in plant research. Plants are complex in shape, and occlusion can often occur. Three-dimensional (3D) data are expected to measure the morphological traits of plants with higher accuracy. Plants have organs with flat and/or narrow shapes and similar component structures are repeated. Therefore, it is difficult to construct an accurate 3D model by applying methods developed for industrial materials and architecture. Here, we review noncontact and all-around 3D modeling and configuration of camera systems to measure the morphological traits of plants in terms of system composition, accuracy, cost, and usability. Typical noncontact 3D measurement methods can be roughly classified into active and passive methods. We describe their advantages and disadvantages. Structure-from-motion/multi-view stereo (SfM/MVS), a passive method, is the most frequently used measurement method for plants. It is described in terms of “forward intersection” and “backward resection.” We recently developed a novel SfM/MVS approach by mixing the forward and backward methods, and we provide a brief overview of our approach in this paper. While various fields are adopting 3D model construction, nonexpert users struggle to use them and end up selecting inadequate methods, which lead to model failure. We hope that this review will help users who are considering starting to construct and measure 3D models.

1. [1] R. T. Furbank and M. Tester, “Phenomics-technologies to relieve the phenotyping bottleneck,” Trends in Plant Science, Vol.16, pp. 635-644, 2011.
2. [2] R. Pieruschka and T. Lawson, “Phenotyping in plants,” J. of Experimental Botany, Vol.66, pp. 5385-5637, 2015.
3. [3] F. Tardieu, L. Cabrera-Bosquet, T. Pridmore, and M. Bennett, “Plant phenomics, from sensors to knowledge,” Current Biology, Vol.27, pp. R770-R783, 2017.
4. [4] M. Vázquez-Arellano, H. W. Griepentrog, D. Reiser, and D. S. Paraforos, “3D imaging systems for agricultural applications – a review,” Sensors, Vol.16, 618, 2016.
5. [5] S. Paulus, J. Behmann, A. K. Mahlein, L. Plümer, and H. Kuhlmann, “Low-cost 3D systems: suitable tools for plant phenotyping,” Sensors, Vol.14, pp. 3001-3018, 2014.
6. [6] M. Minervini, M. V. Giuffrida, P. Perata, and S. A. Tsaftaris, “Phenotiki: An open software and hardware platform for affordable and easy image – based phenotyping of rosette – shaped plants,” The Plant J., Vol.90, pp. 204-216, 2017.
7. [7] A. Dobrescu, L. C. Scorza, S. A. Tsaftaris, and A. J. McCormick, “A “Do-It-Yourself” phenotyping system: measuring growth and morphology throughout the diel cycle in rosette shaped plants,” Plant Methods, Vol.13, 95, 2017.
8. [8] T. Tanabata and S. Isobe, “DIY Phenology: Open Source and Affordable Technology Development in Plant Phenotyping,” Int. Plant & Animal Genome XXVIII (PAG XXVIII), San Diego, CA, USA, 2020.
9. [9] N. Kochi, T. Tanabata, A. Hayashi, and S. Isobe, “A 3D Shape-Measuring System for Assessing Strawberry Fruits,” Int. J. Automation Technol., Vol.12, No.3, pp. 395-404, 2018.
10. [10] T. Yoshizawa, “Handbook of optical metrology: Principles and Applications,” CRC Press, 2017.
11. [11] Y. Wang, W. Wen, S. Wu, C. Wang, Z. Yu, X. Guo, and C. Zhao, “Maize Plant Phenotyping: Comparing 3D Laser Scanning, Multi-View Stereo Reconstruction, and 3D Digitizing Estimates,” Remote Sensing, Vol.11, No.1, 63, 2019.
12. [12] N. Kochi, “Photogrammetry,” T. Yoshizawa (Ed.), “Handbook of Optical Metrology: Principles and Applications,” Taylor and Francis, 2009.
13. [13] A. Chaudhury, C. Ward, and A. Talasaz, “Machine vision system for 3D plant phenotyping,” IEEE/ACM Trans. on Computational Biology and Bioinformatics, Vol.16, pp. 2009-2022, 2018.
14. [14] QVI OGP. https://www.ogpnet.com/ [Accessed October 30, 2020]
15. [15] Phenospex. https://phenospex.com/products/plant-phenotyping/planteye-f500-multispectral-3d-laser-scanner/ [Accessed October 30, 2020]
16. [16] T. Dornbusch, S. Lorrain, and D. Kuznetsov, “Measuring the diurnal pattern of leaf hyponasty and growth in Arabidopsis – a novel phenotyping approach using laser scanning,” Functional Plant Biology, Vol.39, pp. 860-869, 2012.
17. [17] S. Winkelbach, S. Molkenstruck, and F. M. Wahl, “Low-cost laser range scanner and fast surface registration approach,” Joint Pattern Recognition Symp., pp. 718-728, 2006.
18. [18] CREAFORM, “Portable 3D Measurement Solutions.” https://www.creaform3d.com/en/portable-3d-scanners [Accessed March 31, 2021]
19. [19] Texas Instruments. https://www.ti.com/dlp-chip/advanced-light-control/overview.html [Accessed October 30, 2020]
20. [20] L. Benjamin, “Introduction to ±12 Degree Orthogonal Digital Micromirror Devices (DMDs),” 2018. https://www.tij.co.jp/lit/an/dlpa008b/dlpa008b.pdf [Accessed October 31, 2020]
21. [21] Zeiss ATOS. https://www.handsonmetrology.com/products/atos-q [Accessed March 31, 2021]
22. [22] T. Wakayama and T. Yoshizawa, “Compact camera for three-dimensional profilometry incorporating a single MEMS mirror,” Optical Engineering, Vol.51, No.1, 013601, 2012
23. [23] HP 3D Structured Light Scanner. https://support.hp.com/us-en/product/hp-3d-structured-light-scanner/14169438 [Accessed March 31, 2021]
24. [24] S. Paulus, J. Behmann, A. K. Mahlein, L. Plümer, and H. Kuhlmann, “Low-cost 3D systems: suitable tools for plant phenotyping,” Sensors, Vol.14, No.2, pp. 3001-3018, 2014.
25. [25] L. Keselman, J. Iselin, W. Anders, G. Jepsen, and A. Bhowmik, “Intel(R) RealSense(TM) stereoscopic depth cameras,” Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition Workshops, pp. 1267-1276, 2017.
26. [26] Y. Chéné, D. Rousseau, P. Lucidarme, J. Bertheloot, V. Caffier, P. Morel, and F. Chapeau-Blondeau, “On the use of depth camera for 3D phenotyping of entire plants,” Computers and Electronics in Agriculture, Vol.82, pp. 122-127, 2012.
27. [27] G. Azzari, M. Goulden, and R. Rusu, “Rapid characterization of vegetation structure with a Microsoft Kinect sensor,” Sensors, Vol.13, No.2, pp. 2384-2398, 2013.
28. [28] D. Li, L. Xu, C. Tan, E. Goodman, D. Fu, and L. Xin, “Digitization and visualization of greenhouse tomato plants in indoor environments,” Sensors, Vol.15, No.2, pp. 4019-4051, 2015.
29. [29] https://www.intelrealsense.com/depth-camera-d435/ [Accessed October 31, 2020]
30. [30] https://leica-geosystems.com/en-us/products/laser-scanners/scanners [Accessed October 31, 2020]
31. [31] https://www.faro.com/en/Products/Hardware/Focus-Laser-Scanners [Accessed March 31, 2021]
32. [32] https://www.topcon.co.jp/en/positioning/products/product/3dscanner/GLS-2000_E.html [Accessed March 31, 2021]
33. [33] https://leica-geosystems.com/en-us/products/laser-scanners/scanners/leica-rtc360 [Accessed October 31, 2020]
34. [34] https://leica-geosystems.com/en-us/products/laser-scanners/scanners/blk360 [Accessed October 31, 2020]
35. [35] Y. Lin, “LiDAR: An important tool for next-generation phenotyping technology of high potential for plant phenomics?,” Computers and Electronics in Agriculture, Vol.119, pp. 61-73, 2015.
36. [36] M. Friedli, N. Kirchgessner, C. Grieder, F. Liebisch, M. Mannale, and A. Walter, “Terrestrial 3D laser scanning to track the increase in canopy height of both monocot and dicot crop species under field conditions,” Plant Methods, Vol.12, No.1, 9, 2016.
37. [37] Y. Su, F. Wu, Z. Ao, S. Jin, F. Qin, B. Liu, and Q. Guo, “Evaluating maize phenotype dynamics under drought stress using terrestrial lidar,” Plant methods, Vol.15, No.1, 11, 2019.
38. [38] W. Su, D. Zhu, J. Huang, and H. Guo, “Estimation of the vertical leaf area profile of corn (Zea mays) plants using terrestrial laser scanning (TLS),” Computers and Electronics in Agriculture, Vol.150, pp. 5-13, 2018.
39. [39] T. Oggier et al., “SwissRanger SR3000 and first experiences based on miniaturized 3D-TOF cameras,” Proc. of the First Range Imaging Research Day at ETH Zurich, 2005.
40. [40] Sony. https://www.sony-depthsensing.com/ [Accessed October 31, 2020]
41. [41] https://thinklucid.com/helios-time-of-flight-tof-camera/ [Accessed October 31, 2020]
42. [42] https://www.baslerweb.com/en/products/cameras/3d-cameras/ [Accessed October 31, 2020]
43. [43] https://www.signal.co.jp/english/products/vbc.html#3d [Accessed October 31, 2020]
44. [44] http://autonomousdriving.pioneer/en/3d-lidar/ [Accessed October 31, 2020]
45. [45] https://quanergy.com/products/s3/ [Accessed October 31, 2020]
46. [46] M. Hämmerle and B. Höfle, “Direct derivation of maize plant and crop height from low-cost time-of-flight camera measurements,” Plant Methods, Vol.12, No.1, 50, 2016.
47. [47] X. Wang, D. Singh, S. Marla, G. Morris, and J. Poland, “Field-based high-throughput phenotyping of plant height in sorghum using different sensing technologies,” Plant Methods, Vol.14, No.1, 53, 2018.
48. [48] R. F. McCormick, S. K. Truong, and J. E. Mullet, “3D sorghum reconstructions from depth images identify QTL regulating shoot architecture,” Plant physiology, Vol.172, No.2, pp. 823-834, 2016.
49. [49] https://www.intelrealsense.com/lidar-camera-l515/ [Accessed October 31, 2020]
50. [50] G. Polder and J. W. Hofstee, “Phenotyping large tomato plants in the greenhouse using a 3D light-field camera,” 2014 ASABE and CSBE/SCGAB Annual Int. Meeting, 2014-07-13/2014-07-16, 2014.
51. [51] L. N. Smith, W. Zhang, M. F. Hansen, I. J. Hales, and M. L. Smith, “Innovative 3D and 2D machine vision methods for analysis of plants and crops in the field,” Computers in Industry, Vol.97, pp. 122-131, 2018.
52. [52] G. Bernotas, L. C. Scorza, M. F. Hansen, I. J. Hales, K. Halliday, L. N. J. Smith, and A. J. McCormick, “A photometric stereo-based 3D imaging system using computer vision and deep learning for tracking plant growth,” GigaScience, Vol.8, No.5, giz056, 2019.
53. [53] W. Matusik, C. Buehler, R. Raskar, S. J. Gortler, and L. McMillan, “Image-based visual hulls,” Proc. of the 27th Annual Conf. on Computer Graphics and Interactive Techniques, pp. 369-374, 2000.
54. [54] S. Yamazaki, S. G. Narasimhan, S. Baker, and T. Kanade, “The theory and practice of coplanar shadowgram imaging for acquiring visual hulls of intricate objects,” Int. J. of Computer Vision, Vol.81, pp. 259-280, 2009.
55. [55] C. V. Nguyen, J. Fripp, D. R. Lovell, R. Furbank, P. Kuffner, H. Daily, and X. Sirault, “3D scanning system for automatic high-resolution plant phenotyping,” Proc. of the 2016 Int. Conf. on Digital Image Computing: Techniques and Applications (DICTA), pp. 1-8, 2016.
56. [56] F. Golbach, G. Kootstra, S. Damjanovic, G. Otten, and R. van de Zedde, “Validation of plant part measurements using a 3D reconstruction method suitable for high-throughput seedling phenotyping,” Machine Vision and Applications, Vol.27, No.5, pp. 663-680, 2016.
57. [57] A. Baumberg, A. Lyons, and R. Taylor, “3D SOM – A commercial software solution to 3D scanning,” Graphical Models, Vol.67, No.6, pp. 476-495, 2005.
58. [58] 3DSOM. http://www.3dsom.com/ [Accessed October 31, 2020]
59. [59] A. Paproki, X. Sirault, S. Berry, R. Furbank, and J. Fripp, “A novel mesh processing based technique for 3D plant analysis,” BMC plant biology, Vol.12, No.1, 63, 2012.
60. [60] H. Scharr, C. Briese, P. Embgenbroich, A. Fischbach, F. Fiorani, and M. Müller-Linow, “Fast high resolution volume carving for 3D plant shoot reconstruction,” Frontiers in Plant Science, Vol.8, 1680, 2017.
61. [61] M. P. Pound, A. P. French, E. H. Murchie, and T. P. Pridmore, “Automated recovery of three-dimensional models of plant shoots from multiple color images,” Plant Physiol., Vol.166, pp. 1688-1698. 2014.
62. [62] M. P. Pound, A. P. French, J. A. Fozard, E. H. Murchie, and T. P. Pridmore, “A patch-based approach to 3D plant shoot phenotyping,” Machine Vision and Applications, Vol.27, No.5, pp. 767-779, 2016.
63. [63] L. Lou, Y. Liu, M. Sheng, J. Han, and J. H. Doonan, “A cost-effective automatic 3D reconstruction pipeline for plants using multi-view images,” Proc. 15th Conf. Towards Autonomous Robotic Systems, pp. 221-230, 2014.
64. [64] Z. Ni, T. F. Burks, and W. S. Lee, “3D Reconstruction of Plant/Tree Canopy Using Monocular and Binocular Vision,” J. of Imaging, Vol.2, No.4, 28, 2016.
65. [65] S. Liu, L. Acosta-Gamboa, X. Huang, and A. Lorence, “Novel Low Cost 3D Surface Model Reconstruction System for Plant Phenotyping,” J. of Imaging, Vol.3, No.3, 39, 2017.
66. [66] Y. Zhang, P. Teng, M. Aono, Y. Shimizu, F. Hosoi, and K. Omasa, “3D monitoring for plant growth parameters in field with a single camera by multi-view approach,” J. of Agricultural Meteorology, Vol.74, No.4, pp. 129-139, 2018.
67. [67] S. Wu, W. Wen, Y. Wang, J. Fan, C. Wang, W. Gou, and X. Guo, “MVS-Pheno: A Portable and Low-Cost Phenotyping Platform for Maize Shoots Using Multiview Stereo 3D Reconstruction,” Plant Phenomics, Vol.2020, 1848437, 2020.
68. [68] D. Lowe, “Distinctive Image Features from Scale-invariant keypoints,” Int. J. of Computer Vision, Vol.60, No.2, pp. 91-110, 2004.
69. [69] OpenCV team, OpenCV@CVPR2019. https://opencv.org/ [Accessed October 31, 2020]
70. [70] N. Snavely, “Bundler: Structure from motion (sfm) for unordered image collections,” 2008. https://www.cs.cornell.edu/snavely/bundler/ [Accessed October 31, 2020]
71. [71] C. Wu, S. Agarwal, B. Curless, and S. Seitz, “Multicore bundle adjustment,” Proc. of Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 3057-3064, 2011.
72. [72] C. Wu, “Towards Linear-time Incremental Structure from Motion,” Proc. of the 2013 Int. Conf. on 3D Vision – 3DV 2013, pp. 127-134, 2013.
73. [73] Y. Furukawa and J. Ponce, “Accurate, dense, and robust multiview stereopsis,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.32, No.8, pp. 1362-1376, 2009.
74. [74] S. Liu, L. Acosta-Gamboa, X. Huang, and A. Lorence, “Novel Low Cost 3D Surface Model Reconstruction System for Plant Phenotyping,” J. of Imaging, Vol.3, No.3, 39, 2017.
75. [75] M. P. Deseilligny and I. Cléry, “Apero, an open source bundle adjustment software for automatic calibration and orientation of set of images,” The Int. Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol.XXXVIII-5/W16, pp. 269-276, 2011.
76. [76] P. Moulon, P. Monasse, R. Perrot, and R. Marlet, “Openmvg: Open multiple view geometry,” Int. Workshop on Reproducible Research in Pattern Recognition, pp. 60-74, 2016.
77. [77] C. Sweeney, T. Hollerer, and M. Turk, “Theia: A Fast and Scalable Structure-from-Motion Library,” Proc. of the 23rd ACM Int. Conf. on Multimedia, pp. 693-696, 2015.
78. [78] Mapillary, “OpenSfM: Open source Structure from Motion pipeline,” 2019. https://github.com/mapillary/OpenSfM [Accessed October 31, 2020]
79. [79] J. L. Schonberger and J. M. Frahm, “Structure-from-motion revisited,” Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 4104-4113, 2016.
80. [80] D. Cernea, “OpenMVS: Open Multiple View Stereovision,” 2015. https://github.com/cdcseacave/openMVS/ [Accessed October 31, 2020]
81. [81] E. Rupnik, M. Daakir, and M. P. Deseilligny, “MicMac – a free, open-source solution for photogrammetry,” Open Geospatial Data, Software and Standards, Vol.2, No.1, 14, 2017.
82. [82] Pix4D. https://www.pix4d.com/ [Accessed October 31, 2020]
83. [83] Bentley Systems, “ContextCapture.” https://www.bentley.com/en/products/brands/contextcapture [Accessed October 31, 2020]
84. [84] CapturingReality. https://www.capturingreality.com/Product [Accessed March 31, 2021]
85. [85] Agisoft, “Metashape-Photogrammetric processing of digital images and 3D spatial data generation.” https://www.agisoft.com/ [Accessed October 31, 2020]
86. [86] J. Engel, T. Schöps, and D. Cremers, “LSD-SLAM: Large-Scale Direct Monocular SLAM,” Proc. of European Conf. on Computer Vision (ECCV), pp. 834-849, 2014.
87. [87] R. Mur-Artal, J. M. Montiel, and J. D. Tardos, “ORB-SLAM: a Versatile and Accurate Monocular SLAM System,” IEEE Trans. on Robotics, Vol.31, No.5, pp. 1147-1163, 2015.
88. [88] R. Mur-Artal and J. D. Tardos, “ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras,” IEEE Trans. on Robotics, Vol.33, No.5, pp. 1255-1262, 2017.
89. [89] D. Scaramuzza, F. Fraundorfer, and M. Pollefeys, “Closing the loop in appearance-guided omnidirectional visual odometry by using vocabulary trees,” Robotics and Autonomous Systems, Vol.58, No.6, pp. 820-827, 2010.
90. [90] M. Havlena, A. Torii, and T. Pajdla, “Efficient structure from motion by graph optimization,” Proc. of the European Conf. on Computer Vision, pp. 100-113, 2010.
91. [91] C. Sweeney, T. Sattler, T. Hollerer, M. Turk, and M. Pollefeys, “Optimizing the viewing graph for structure-from-motion,” Proc. of the IEEE Int. Conf. on Computer Vision, pp. 801-809, 2015.
92. [92] K. Kitamura, N. Kochi, H. Watanabe, M. Yamada, and S. Kaneko, “Human Body Measurement by Robust Stereo Matching,” Proc. of the 9th Conf. on Optical 3-D Measurement Techniques, Vol.2, pp. 254-263, 2009.
93. [93] N. Kochi, T. Ito, T. Noma, H. Otani, S. Nishimura, and J. Ito, “PC-Based 3D Image Measuring Station with Digital Camera an Example of its Actual Application on a Historical Ruin,” Int. Archives of Photogrammetry and Remote Sensing, Vol.XXXIV-5/W12, pp. 195-199, 2003.
94. [94] T. Anai, N. Kochi, N. Fukaya, and N. D’Apuzzo, “Application of Orientation Code Matching for Structure from Motion,” The Int. Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol.34, Part XXX Commission V Symp., Newcastle upon Tyne, pp. 33-38, 2010.
95. [95] T. Anai et al., “Examination about influence for precision of 3D image measurement from the ground control point measurement and surface matching,” The Int. Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol.XL-4/W5, pp. 165-170, 2015.
96. [96] A. Hayashi, N. Kochi, K. Kodama, T. Tanabata, and S. Isobe, “Automatic Omnidirectional Image Photographing System and 3D Model Construction System for Plant,” PAG ASIA 2019, P0019, 2019.