JACIII Vol.28 No.3 pp. 586-594
doi: 10.20965/jaciii.2024.p0586

Research Paper:

FFD-SLAM: A Real-Time Visual SLAM Toward Dynamic Scenes with Semantic and Optical Flow Information

Hao Zhang*, Yu Wang**, Tianjie Zhong*, Fangyan Dong*, and Kewei Chen*,†

*Faculty of Mechanical Engineering & Mechanics, Ningbo University
No.818 Fenghua Road, Jiangbei District, Ningbo, Zhejiang 315211, China

Corresponding author

**China Academy of Safety Science & Technology
Building A1, No.32 Beiyuan Road, Chaoyang District, Beijing 100012, China

December 18, 2023
January 18, 2024
May 20, 2024
dynamic object detection, optical flow filtering, image feature points elimination, SLAM algorithm

To solve the problem of poor localization accuracy and robustness of visual simultaneous localization and mapping (SLAM) systems in highly dynamic environments, this paper proposes a dynamic visual SLAM algorithm called FFD-SLAM that fuses the target detection network with the optical flow method. The algorithm considers ORB-SLAM2 as the basic framework, joins the semantic thread in parallel with its tracking thread, initially obtains the set of feature points through the real-time detection of dynamic objects in the environment through YOLOv5 in the semantic thread, then filters the set of feature points obtained in the semantic thread through the optical flow module, and finally utilizes the remaining static feature points for the matching calculation. Experiments showed that the proposed algorithm showed an improvement of approximately 97% in the localization accuracy compared with the ORB-SLAM2 algorithm in a highly dynamic environment, which effectively improves the localization accuracy and robustness of the system. The proposed algorithm also showed a higher real-time performance compared with some excellent dynamic SLAM algorithms.

Cite this article as:
H. Zhang, Y. Wang, T. Zhong, F. Dong, and K. Chen, “FFD-SLAM: A Real-Time Visual SLAM Toward Dynamic Scenes with Semantic and Optical Flow Information,” J. Adv. Comput. Intell. Intell. Inform., Vol.28 No.3, pp. 586-594, 2024.
Data files:
  1. [1] S. Yuan, H. Wang, and L. Xie, “Survey on Localization Systems and Algorithms for Unmanned Systems,” Unmanned Systems, Vol.9, No.2, pp. 129-163, 2021.
  2. [2] H. Wang, C. Wang, and L. Xie, “Intensity-SLAM: Intensity Assisted Localization and Mapping for Large Scale Environment,” IEEE Robotics and Automation Letters, Vol.6, No.2, pp. 1715-1721, 2021.
  3. [3] Y. Fan, Q. Zhang, S. Liu, Y. Tang, X. Jing, J. Yao, and H. Han, “Semantic SLAM with More Accurate Point Cloud Map in Dynamic Environments,” IEEE Access, Vol.8, pp. 112237-112252, 2020.
  4. [4] K. Wang, X. Yao, Y. Huang, M. Liu, and Y. Lu, “Review of Visual SLAM in Dynamic Environment,” Robot, Vol.43, No.6, Article No.715732, 2021.
  5. [5] C. Cadena, L. Carlone, H. Carrillo, Y. Latif, D. Scaramuzza, J. Neira, I. Reid, and J. J. Leonard, “Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age,” IEEE Trans. on Robotics, Vol.32, No.6, pp. 1309-1332, 2016.
  6. [6] R. Mur-Artal and J. D. Tardos, “ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras,” IEEE Trans. on Robotics, Vol.33, No.5, pp. 1255-1262, 2017.
  7. [7] V. Badrinarayanan, A. Kendall, and R. Cipolla, “SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Seg-mentation,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.39, No.12, pp. 24810-2495, 2017.
  8. [8] K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask R-CNN,” 2017 IEEE Int. Conf. on Computer Vision (ICCV), pp. 2961-2969, 2017.
  9. [9] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” Proc. of 2016 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2016.
  10. [10] W. Liu, D. Anguelov, D. Erhan et al., “SSD: Single Shot MultiBox Detector,” European Conf. on Computer Vision, pp. 21-37, 2016.
  11. [11] Y. Liu and J. Miura, “RDS-SLAM: Real-Time Dynamic SLAM Using Semantic Segmentation Methods,” IEEE Access, Vol.9, pp. 23772-23785, 2021.
  12. [12] C. Yu, Z. Liu, X. Liu et al., “DS-SLAM: A Semantic Visual SLAM Towards Dynamic Environments,” Proc. of 2018 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 1168-1174, 2018.
  13. [13] B. Bescos, J. M. Facil, J. Civera, and J. Neira, “DynaSLAM: Tracking, Mapping, and Inpainting in Dynamic Scenes,” IEEE Robotics and Automation Letters, Vol.3, No.4, pp. 4076-4083, 2018.
  14. [14] P. Su, S. Y. Luo, and X. C. Huang, “Real-Time Dynamic SLAM Algorithm Based on Deep Learning,” IEEE Access, Vol.10, pp. 87754-87766, 2022.
  15. [15] F. W. Zhong, S. Wang, Z. Q. Zhang et al., “Detect-SLAM: Making Object Detection and SLAM Mutually Beneficial,” IEEE Winter Conf. on Applications of Computer Vision, pp. 1001-1010, 2018.
  16. [16] B. D. Lucas and T. Kanade, “An Iterative Image Registration Technique with an Application to Stereo Vision,” Proc. of 7th Int. Joint Conf. on Artificial Intelligence, pp. 456-459, 1981.
  17. [17] J. Sturm, N. Engelhard, F. Endres, W. Burgard, and D. Cremers, “A benchmark for the evaluation of RGB-D SLAM systems,” Proc. of 2012 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 573-580, 2012.

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Jul. 12, 2024