Occlusion handling is a key technical issue in augmented reality research. This paper proposes a new occlusion algorithm based on object contour detection to address issues such as poor real-time occlusion processing, high computational complexity in comparing the depth values of virtual and real objects, and the presence of jagged, blurry, and hollow edges in occluded areas. First, based on the depth and color information, we obtained aligned images of real scenes. Second, we extracted the maximum closed contour of the real object in the scene and overlaid it with the aligned image. Subsequently, we generated a virtual object and obtained a depth map of the virtual object. Finally, by comparing the depth values of the stacked images with the virtual objects, masks are generated in real time and optimized to present the occlusion processing results. Experimental comparisons demonstrated that the algorithm presented in this study not only improves real-time performance but also enhances accuracy at the intersection edges of virtual and real images. Simultaneously, it is no longer limited by the size of real scene images and can achieve real-time virtual and real occlusion effects.

1. [1] M. Billinghurst, A. Clark, and G. Lee, “A survey of augmented reality,” Foundations and Trends® in Human-Computer Interaction, Vol.8, Nos.2-3, pp. 73-272, 2015. https://doi.org/10.1561/1100000049
2. [2] W. Li et al., “Real-time occlusion handling for augmented reality assistance assembly systems with monocular images,” J. of Manufacturing Systems, Vol.62, pp. 561-574, 2022. https://doi.org/10.1016/j.jmsy.2022.01.012
3. [3] R. T. Azuma, “A survey of augmented reality,” Presence: Teleoperators and Virtual Environments, Vol.6, No.4, pp. 355-385, 1997. https://doi.org/10.1162/pres.1997.6.4.355
4. [4] F. De Pace, F. Manuri, and A. Sanna, “Augmented reality in Industry 4.0,” American J. of Computer Science and Information Technology, Vol.6, No.1, Article No.17, 2018. https://doi.org/10.21767/2349-3917.100017
5. [5] K. Lavingia and S. Tanwar, “Augmented reality and Industry 4.0,” A. Nayyar and A. Kumar (Eds.), “A Roadmap to Industry 4.0: Smart Production, Sharp Business and Sustainable Development,” pp. 143-155, Springer, 2020. https://doi.org/10.1007/978-3-030-14544-6_8
6. [6] G. M. Santi et al., “Augmented reality in Industry 4.0 and future innovation programs,” Technologies, Vol.9, No.2, Article No.33, 2021. https://doi.org/10.3390/technologies9020033
7. [7] H. Wang et al., “Research on recognition and 3-D visualization of key equipment in NPP based on AR technology,” 2018 Int. Conf. on Power System Technology (POWERCON), pp. 4661-4667, 2018. https://doi.org/10.1109/POWERCON.2018.8602345
8. [8] F. Amaguaña et al., “Simulation system based on augmented reality for optimization of training tactics on military operations,” Augmented Reality, Virtual Reality, and Computer Graphics (Proc. of the 5th Int. Conf. on AVR 2018), Part 1, pp. 394-403, 2018. https://doi.org/10.1007/978-3-319-95270-3_33
9. [9] W. Wang et al., “Augmented reality in maintenance training for military equipment,” J. of Physics: Conf. Series, Vol.1626, Article No.012184, 2020. https://doi.org/10.1088/1742-6596/1626/1/012184
10. [10] H.-K. Wu et al., “Current status, opportunities and challenges of augmented reality in education,” Computers & Education, Vol.62, pp. 41-49, 2013. https://doi.org/10.1016/j.compedu.2012.10.024
11. [11] N. Elmqaddem, “Augmented reality and virtual reality in education. Myth or reality?,” Int. J. of Emerging Technologies in Learning, Vol.14, No.3, pp. 234-242, 2019. https://doi.org/10.3991/ijet.v14i03.9289
12. [12] J. Gerup, C. B. Soerensen, and P. Dieckmann, “Augmented reality and mixed reality for healthcare education beyond surgery: An integrative review,” Int. J. of Medical Education, Vol.11, pp. 1-18, 2020. https://doi.org/10.5116/ijme.5e01.eb1a
13. [13] V. Ferrari, G. Klinker, and F. Cutolo, “Augmented reality in healthcare,” J. of Healthcare Engineering, Vol.2019, Article No.9321535, 2019. https://doi.org/10.1155/2019/9321535
14. [14] G. Jha, L. S. Sharma, and S. Gupta, “Future of augmented reality in healthcare department,” Proc. of 2nd Int. Conf. on Computing, Communications, and Cyber-Security, pp. 667-678, 2021. https://doi.org/10.1007/978-981-16-0733-2_47
15. [15] R. G. Boboc et al., “Augmented reality in cultural heritage: An overview of the last decade of applications,” Applied Sciences, Vol.12, No.19, Article No.9859, 2022. https://doi.org/10.3390/app12199859
16. [16] M. C. T. Dieck and T. H. Jung, “Value of augmented reality at cultural heritage sites: A stakeholder approach,” J. of Destination Marketing & Management, Vol.6, No.2, pp. 110-117, 2017. https://doi.org/10.1016/j.jdmm.2017.03.002
17. [17] M. C. F. Macedo and A. L. Apolinário, “Occlusion handling in augmented reality: Past, present and future,” IEEE Trans. on Visualization and Computer Graphics, Vol.29, No.2, pp. 1590-1609, 2023. https://doi.org/10.1109/TVCG.2021.3117866
18. [18] T. Luo et al., “A virtual-real occlusion method based on GPU acceleration for MR,” 2019 IEEE Conf. on Virtual Reality and 3D User Interfaces (VR), pp. 1068-1069, 2019. https://doi.org/10.1109/VR.2019.8797811
19. [19] A. K. Hebborn, N. Höhner, and S. Müller, “Occlusion matting: Realistic occlusion handling for augmented reality applications,” 2017 IEEE Int. Symp. on Mixed and Augmented Reality (ISMAR), pp. 62-71, 2017. https://doi.org/10.1109/ISMAR.2017.23
20. [20] Y. Tian et al., “Handling occlusions in augmented reality based on 3D reconstruction method,” Neurocomputing, Vol.156, pp. 96-104, 2015. https://doi.org/10.1016/j.neucom.2014.12.081
21. [21] J. Watson et al., “Virtual occlusions through implicit depth,” 2023 IEEE/CVF Conf. on Computer Vision and Pattern Recognition, pp. 9053-9064, 2023. https://doi.org/10.1109/CVPR52729.2023.00874
22. [22] T. Hamasaki and Y. Itoh, “Varifocal occlusion for optical see-through head-mounted displays using a slide occlusion mask,” IEEE Trans. on Visualization and Computer Graphics, Vol.25, No.5, pp. 1961-1969, 2019. https://doi.org/10.1109/TVCG.2019.2899249
23. [23] J. Liang, H. He, and Y. Wu, “Bare-hand depth perception used in augmented reality assembly supporting,” IEEE Access, Vol.8, pp. 1534-1541, 2020. https://doi.org/10.1109/ACCESS.2019.2962112
24. [24] J. Gimeno et al., “Addressing the occlusion problem in augmented reality environments with Phantom hollow objects,” 2018 IEEE Int. Symp. on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), pp. 21-24, 2018. https://doi.org/10.1109/ISMAR-Adjunct.2018.00024
25. [25] M. Sizintsev et al., “Long-range augmented reality with dynamic occlusion rendering,” IEEE Trans. on Visualization and Computer Graphics, Vol.27, No.11, pp. 4236-4244, 2021. https://doi.org/10.1109/TVCG.2021.3106434
26. [26] W. Li et al., “Integrated registration and occlusion handling based on deep learning for augmented-reality-assisted assembly instruction,” IEEE Trans. on Industrial Informatics, Vol.19, No.5, pp. 6825-6835, 2023. https://doi.org/10.1109/TII.2022.3189428
27. [27] M. H. Davis et al., “A physics-based coordinate transformation for 3-D image matching,” IEEE Trans. on Medical Imaging, Vol.16, No.3, pp. 317-328, 1997.
28. [28] G. Hao, L. Min, and H. Feng, “Improved self-adaptive edge detection method based on Canny,” 2013 5th Int. Conf. on Intelligent Human-Machine Systems and Cybernetics, pp. 527-530, 2013. https://doi.org/10.1109/IHMSC.2013.273
29. [29] T. O. Hodson, T. M. Over, and S. S. Foks, “Mean squared error, deconstructed,” J. of Advances in Modeling Earth Systems, Vol.13, No.12, Article No.e2021MS002681, 2021. https://doi.org/10.1029/2021MS002681
30. [30] Z. Wang, E. P. Simoncelli, and A. C. Bovik, “Multiscale structural similarity for image quality assessment,” The 37th Asilomar Conf. on Signals, Systems & Computers, Vol.2, pp. 1398-1402, 2003. https://doi.org/10.1109/ACSSC.2003.1292216
31. [31] E. K. Yen and R. G. Johnston, “The ineffectiveness of the correlation coefficient for image comparisons,” Vulnerability Assessment Team, Los Alamos National Laboratory, 1996.