In industrial plants, piping and equipment are intricately interconnected, and there are many components with a variety of shapes. To use point clouds of industrial plants for simulation of maintenance work, it is necessary to extract the components. Deep learning is effective for recognizing components in point clouds of industrial plants. However, training classifiers is challenging due to the difficulty in acquiring diverse point cloud datasets and the labor-intensive process of annotating large-scale point clouds. A promising approach to address these issues is to train the classifier on virtual point clouds generated from CAD models. However, classifiers trained on these virtual point clouds often fail to achieve sufficient segmentation accuracy due to discrepancies between virtual point clouds and actual point clouds captured by terrestrial laser scanners. This paper proposes methods to improve segmentation accuracy by reducing these discrepancies. First, we introduce a method to incorporate features such as missing points, noise, and outliers observed in actual point clouds. Furthermore, we propose a data augmentation approach that applies up-sampling using a deep learning model trained on paired virtual and real point clouds to reduce the discrepancy between them. Our evaluation demonstrates that the proposed methods effectively improve the segmentation accuracy of point clouds of industrial plants.

1. [1] P. Alliez, D. Cohen-Steiner, Y. Tong, and M. Desburn, “Voronoi-based variational reconstruction of unoriented point sets,” Eurographics Symp. on Geometry Processing, pp. 39-48, 2007.
2. [2] F. Calakli and G. Taubin, “SSD: smooth signed distance surface reconstruction,” Computer Graphic Forum, Vol.30, No.7, pp. 1993-2002, 2011. https://doi.org/10.1111/j.1467-8659.2011.02058.x
3. [3] A. Bey, R. Chaine, R. Marc, G. Thibault, and S. Akkouche, “Reconstruction of consistent 3D CAD models from point cloud data using a priori CAD models,” Int. Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol.XXXVIII-5/W12, pp. 289-294, 2011. https://doi.org/10.5194/isprsarchives-XXXVIII-5-W12-289-2011
4. [4] J. Chai, H. Chi, X. Wang, C. Wu, K. H. Jung, and J. M. Lee, “Automatic as-built modeling for concurrent progress tracking of plant construction based on laser scanning,” Concurrent Engineering, Vol.24, No.4, pp. 369-380, 2016. https://doi.org/10.1177/1063293X16670449
5. [5] H. Son and C. Kim, “Automatic 3D reconstruction of as-built pipeline based on curvature computations from laser-scanned data,” Construction Research Congress, pp. 925-934, 2014. https://doi.org/10.1061/9780784413517.095
6. [6] K. Kawasaki and H. Masuda, “Shape reconstruction of structural members of steel tower considering symmetrical relationships,” Computer-Aided Design and Applications, Vol.20, No.5, pp. 814-825, 2023. https://doi.org/10.14733/cadaps.2023.814-825
7. [7] N. D. Cornea, D. Silver, and P. Min, “Curve-skeleton properties, applications, and algorithms,” IEEE Trans. on Visualization and Computer Graphics, Vol.13, No.3, pp. 530-548, 2007. https://doi.org/10.1109/TVCG.2007.1002
8. [8] V. Sam, H. Kawata, and T. Kanai, “A robust and centered curve skeleton extraction from 3D point cloud,” Computer-Aided Design and Applications, Vol.9, No.6, pp. 869-879, 2012. https://doi.org/10.3722/cadaps.2012.869-879
9. [9] J. Lee, H. Son, C. Kim, and C. Kim, “Skeleton-based 3D reconstruction of as-built pipelines from laser-scan data,” Automation in Construction, Vol.35, pp. 199-207, 2013. https://doi.org/10.1016/j.autcon.2013.05.009
10. [10] A. Tagliasacchi, H. Zhang, and D. Cohen-Or, “Curve skeleton extraction from incomplete point cloud,” ACM Trans. on Graphics, Vol.28, No.3, Article No.71, 2009. https://doi.org/10.1145/1531326.1531377
11. [11] Y. Midorikawa and H. Masuda, “Extraction of rotational surfaces and generalized cylinders from point-clouds using section curves,” Int. J. Automation Technol., Vol.12, No.6, pp. 901-910, 2018. https://doi.org/10.20965/ijat.2018.p0901
12. [12] H. Masuda, T. Niwa, I. Tanaka, and R. Matsuoka, “Reconstruction of polygonal faces from large-scale point-clouds of engineering plants,” Computer-Aided Design and Applications, Vol.12, No.5, pp. 150-152, 2014. https://doi.org/10.1080/16864360.2015.1014733
13. [13] H. Masuda and I. Tanaka, “As-built 3D modeling of large facilities based on interactive feature editing,” Computer-Aided Design and Applications, Vol.7, No.3, pp. 349-360, 2010. https://doi.org/10.3722/cadaps.2010.349-360
14. [14] H. Masuda, I. Tanaka, and M. Enomoto, “Reliable surface extraction from point-clouds using scanner-dependent parameters,” Computer-Aided Design and Applications, Vol.10, No.2, pp. 265-277, 2013. https://doi.org/10.3722/cadaps.2013.265-277
15. [15] H. Su, S. Maji, E. Kalogerakis, and E. Learned-Miller, “Multiview convolutional neural networks for 3D shape recognition,” Int. Conf. on Computer Vision, pp. 945-953, 2015. https://doi.org/10.1109/ICCV.2015.114
16. [16] T. Aoki, E. Yamamoto, and H. Masuda, “Detection of multiscale deterioration from point-clouds of furnace walls,” Int. J. Automation Technol., Vol.17, No.6, pp. 610-618, 2023. https://doi.org/10.20965/ijat.2023.p0610
17. [17] Y. Guo, H. Wang, Q. Hu, H. Liu, L. Liu, and M. Bennamoun, “Deep learning for 3D point clouds: A survey,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.43, No.12, pp. 4338-4364, 2021. https://doi.org/10.1109/TPAMI.2020.3005434
18. [18] H. Matsumoto, Y. Mori, and H. Masuda, “Extraction of guardrails from MMS data using convolutional neural network,” Int. J. Automation Technol., Vol.15, No.3, pp. 258-267, 2021. https://doi.org/10.20965/ijat.2021.p0258
19. [19] K. Shigeta, D. Hanai, and H. Masuda, “Data augmentation of classifiers for components in industrial plants using CAD models,” Computer-Aided Design & Applications, Vol.19, No.5, pp. 913-923, 2021. https://doi.org/10.14733/cadaps.2022.913-923
20. [20] K. Shigeta, T. Nagumo, and H. Masuda, “Point cloud segmentation for pipelines in industrial facilities using recurrent networks,” Computer-Aided Design and Applications, Vol.20, No.4, pp. 786-796, 2023. https://doi.org/10.14733/cadaps.2023.786-796
21. [21] Q. Zhu, L. Fan, and N. Weng, “Advancements in point cloud data augmentation for deep learning: A survey,” Pattern Recognition, Vol.153, Article No.110532, 2024. https://doi.org/10.1016/j.patcog.2024.110532
22. [22] Y. He, Z. Zhang, Z. Wang, Y. Luo, L. Su, and W. Li, “IPC-Net: Incomplete point cloud classification network based on data augmentation and similarity measurement,” J. of Visual Communication and Image Representation, Vol.91, 2023. https://doi.org/10.1016/j.jvcir.2023.103769
23. [23] W. Ma, J. Chen, Q. Du, and W. Jia, “Pointdrop: Improving object detection from sparse point clouds via adversarial data augmentation,” 25th Int. Conf. on Pattern Recognition, 2021.
24. [24] X. Hu, Z. Duan, X. Huang, Z. Xu, D. Ming, and J. Ma, “Context-aware data augmentation for lidar 3D object detection,” IEEE Int. Conf. on Image Processing, pp. 11-15, 2023. https://doi.org/10.1109/ICIP49359.2023.10222773
25. [25] F. Noichl, F. C. Collins, A. Braun, and A. Borrmann, “Enhancing point cloud semantic segmentation in the data-scarce domain of industrial plants through synthetic data,” Computer-Aided Civil and Infrastructure Engineering, Vol.39, pp. 1530-1549, 2024. https://doi.org/10.1111/mice.13153
26. [26] R. Li, X. Li, C.-W. Fu, D. Cohen-Or, and P.-A. Heng, “PUGAN: a point cloud upsampling adversarial network,” Proc. of the IEEE/CVF Int. Conf. on Computer Vision, 2019.
27. [27] G. Qian, A. Abualshour, G. Li, A. Thabet, and B. Ghanem, “PU-GCN: Point cloud upsampling using graph convolutional networks,” Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 11683-11692, 2021.
28. [28] C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “PointNet++: Deep hierarchical feature learning on point sets in a metric space,” Advances in Neural Information Processing Systems, pp. 5105-5114, 2017.
29. [29] C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “PointNet: Deep learning on point sets for 3D classification and segmentation,” Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 652-660, 2017. https://doi.org/10.1109/CVPR.2017.16