single-jc.php

JACIII Vol.30 No.6 pp. 1816-1845
(2026)

Research Paper:

Semantic-Topological Cognitive Maps for Assistive Indoor Navigation of a Robot Partner

Anhar Risnumawan*,** ORCID Icon, Achmad Fahrul Aji*,*** ORCID Icon, Raditya Artha Rochmanto*,*** ORCID Icon, Teuku Zikri Fatahillah* ORCID Icon, and Naoyuki Kubota*,† ORCID Icon

*Department of Mechanical Systems Engineering, Graduate School of Systems Design, Tokyo Metropolitan University
6-6 Asahigaoka, Hino, Tokyo 191-0065, Japan

†Corresponding author

**Mechatronics Engineering Division, Politeknik Elektronika Negeri Surabaya
Jalan Raya ITS, Surabaya 60111, Indonesia

***Department of Electrical Engineering, Politeknik Negeri Semarang
Jl. Prof. H. Soedarto SH, Tembalang, Semarang 50275, Indonesia

Received:
January 1, 2026
Accepted:
June 9, 2026
Published:
November 20, 2026
Keywords:
semantic topological mapping, cognitive maps, assistive navigation, growing neural gas, open-vocabulary detection
Abstract

Assistive indoor mobile robots are increasingly expected to connect natural-language requests about familiar places and objects with the metric maps used for localization and motion planning. Metric simultaneous localization and mapping (SLAM) supports reliable geometry and collision avoidance, whereas many open-vocabulary scene-graph systems remain separated from the navigation stacks used on deployed robots. This paper presents a semantic-topological cognitive map (STCM) that bridges these representations for assistive navigation. The system derives place nodes from a LiDAR–inertial pose stream through a dual growing neural gas backend and consolidates object instances over time. GroundingDINO and MobileSAM provide open-vocabulary semantic grounding in the map frame through LiDAR–image association and temporal score fusion. A tool-mediated large language model interface maps user commands to explicit STCM nodes, while graph search and ROS 2 Navigation2 (Nav2) remain responsible for path computation and execution. Evaluation with recorded robot sessions, an outdoor distribution-shift probe, and a public indoor-scene benchmark indicates that the STCM backend improves object-graph construction and that the language interface improves functional-command grounding when requested objects are recoverable by perception. The pipeline is demonstrated on a Ranger Garden Assistant platform with FAST-LIO2 mapping, incremental graph construction during the mapping pass, and waypoint execution from a serialized map artifact.

Ranger Garden Assistant platform with semantic topological cognitive map

Ranger Garden Assistant platform with semantic topological cognitive map

Cite this article as:
A. Risnumawan, A. Aji, R. Rochmanto, T. Fatahillah, and N. Kubota, “Semantic-Topological Cognitive Maps for Assistive Indoor Navigation of a Robot Partner,” J. Adv. Comput. Intell. Intell. Inform., Vol.30 No.6, pp. 1816-1845, 2026.
Data files:
References
  1. [1] P. A. Hancock, D. R. Billings, K. E. Schaefer, J. Y. C. Chen, E. J. de Visser, and R. Parasuraman, “A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction,” Human Factors, Vol.53, No.5, pp. 517-527, 2011. https://doi.org/10.1177/0018720811417254
  2. [2] A. A. Saputra, K. Wada, S. Masuda, and N. Kubota, “Multi-scopic Neuro-cognitive Adaptation for Legged Locomotion Robots,” Scientific Reports, Vol.12, Article No.16222, 2022. https://doi.org/10.1038/s41598-022-19599-2
  3. [3] E. C. Tolman, “Cognitive Maps in Rats and Men,” Psychological Review, Vol.55, No.4, pp. 189-208, 1948. https://doi.org/10.1037/h0061626
  4. [4] J. O’Keefe and L. Nadel, “The Hippocampus as a Cognitive Map,” Clarendon Press, 1978.
  5. [5] J. J. Gibson, “The Ecological Approach to Visual Perception,” Houghton Mifflin, 1979.
  6. [6] J. Woo and N. Kubota, “Recognition of Indoor Environment by Robot Partner Using Conversation,” J. Adv. Comput. Intell. Intell. Inform., Vol.17, No.5, pp. 753-760, 2013. https://doi.org/10.20965/jaciii.2013.p0753
  7. [7] N. Kubota and A. Yorita, “Topological Environment Reconstruction in Informationally Structured Space for Pocket Robot Partners,” Proc. IEEE Int. Symp. on Computational Intelligence in Robotics and Automation (CIRA), pp. 165-170, 2009. https://doi.org/10.1109/CIRA.2009.5423215
  8. [8] A. Rosinol, M. Abate, Y. Chang, and L. Carlone, “Kimera: An Open-Source Library for Real-Time Metric-Semantic Localization and Mapping,” Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), 2020. https://doi.org/10.1109/ICRA40945.2020.9196885
  9. [9] N. Hughes, Y. Chang, and L. Carlone, “Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimization,” Proc. Robotics: Science and Systems (RSS), 2022. https://doi.org/10.15607/RSS.2022.XVIII.050
  10. [10] H. Bavle, J. L. Sánchez-López, M. Shaheer, J. Civera, and H. Voos, “S-Graphs+: Real-Time Localization and Mapping Leveraging Hierarchical Representations,” IEEE Robotics and Automation Letters, Vol.8, No.8, pp. 4927-4934, 2023. https://doi.org/10.1109/LRA.2023.3290512
  11. [11] C. Huang, O. Mees, A. Zeng, and W. Burgard, “Visual Language Maps for Robot Navigation,” Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 10608-10615, 2023. https://doi.org/10.1109/ICRA48891.2023.10160969
  12. [12] K. M. Jatavallabhula, A. Kuwajerwala, Q. Gu, M. Omama, T. Chen, A. Maalouf, S. Li et al., “ConceptFusion: Open-Set Multimodal 3D Mapping,” Proc. Robotics: Science and Systems (RSS), 2023. https://doi.org/10.15607/RSS.2023.XIX.066
  13. [13] Q. Gu, A. Kuwajerwala, S. Morin, K. M. Jatavallabhula, B. Sen, A. Agarwal et al. “ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planning,” Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 5021-5028, 2024. https://doi.org/10.1109/ICRA57147.2024.10610243
  14. [14] F. L. Busch, T. Homberger, J. Ortega-Peimbert, Q. Yang, and O. Andersson, “One Map to Find Them All: Real-Time Open-Vocabulary Mapping for Zero-Shot Multi-Object Navigation,” arXiv preprint, arXiv:2409.11764, 2024. https://doi.org/10.48550/arXiv.2409.11764
  15. [15] B. Chen, F. Xia, B. Ichter, K. Rao, K. Gopalakrishnan, M. S. Ryoo, A. Stone, and D. Kappler, “Open-Vocabulary Queryable Scene Representations for Real World Planning,” Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 11509-11522, 2023. https://doi.org/10.1109/ICRA48891.2023.10161534
  16. [16] D. Shah, B. Osiński, B. Ichter, and S. Levine, “LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action,” Proc. 6th Conf. on Robot Learning (CoRL), PMLR, Vol.205, pp. 492-504, 2022.
  17. [17] M. Chang, T. Gervet, M. Khanna, S. Yenamandra, D. Shah, S. Y. Min, K. Shah, C. Paxton, S. Gupta, D. Batra, R. Mottaghi, J. Malik, and D. S. Chaplot, “GOAT: GO to Any Thing,” Proc. Robotics: Science and Systems (RSS), 2024. https://doi.org/10.15607/RSS.2024.XX.073
  18. [18] Y. Tang, M. Wang, Y. Deng et al., “OpenIN: Open-Vocabulary Instance-Oriented Navigation in Dynamic Domestic Environments,” IEEE Robotics and Automation Letters, Vol.10, No.9, pp. 9256-9263, 2025. https://doi.org/10.1109/LRA.2025.3592071
  19. [19] S. Raychaudhuri, D. Ta, K. Ashton, A. X. Chang, J. Wang, and B. Bucher, “Zero-Shot Object-Centric Instruction Following: Integrating Foundation Models with Traditional Navigation,” arXiv preprint, arXiv:2411.07848, 2024. https://doi.org/10.48550/arXiv.2411.07848
  20. [20] L. Zhang, X. Hao, Q. Xu et al., “MapNav: A Novel Memory Representation via Annotated Semantic Maps for VLM-based Vision-and-Language Navigation,” Proc. 63rd Annual Meeting of the Association for Computational Linguistics (ACL), pp. 13032-13056, 2025. https://doi.org/10.18653/v1/2025.acl-long.638
  21. [21] D. S. Chaplot et al., “Neural Topological SLAM for Visual Navigation,” Proc. IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), 2020. https://doi.org/10.1109/CVPR42600.2020.01289
  22. [22] D. An, H. Wang, W. Wang, Z. Wang, Y. Huang, K. He, and L. Wang, “ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environments,” IEEE Trans. Pattern Anal. Mach. Intell., Vol.47, No.7, pp. 5130-5145, 2025. https://doi.org/10.1109/TPAMI.2024.3386695
  23. [23] S. Garg, K. Rana, M. Hosseinzadeh, L. Mares, N. Sünderhauf, F. Dayoub, and I. Reid, “RoboHop: Segment-Based Topological Map Representation for Open-World Visual Navigation,” Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 4090-4097, 2024. https://doi.org/10.1109/ICRA57147.2024.10610234
  24. [24] W. Xu, Y. Cai, D. He, J. Lin, and F. Zhang, “FAST-LIO2: Fast Direct LiDAR-Inertial Odometry,” IEEE Trans. Robotics, Vol.38, No.4, pp. 2053-2073, 2022. https://doi.org/10.1109/TRO.2022.3141876
  25. [25] B. Fritzke, “A Growing Neural Gas Network Learns Topologies,” Advances in Neural Information Processing Systems (NeurIPS), Vol.7, pp. 625-632, 1994.
  26. [26] F. Ardilla, A. A. Saputra, and N. Kubota, “Multi-Scale Batch-Learning Growing Neural Gas Efficiently for Dynamic Data Distributions,” Int. J. Automation Technol., Vol.17, No.3, pp. 206-216, 2023. https://doi.org/10.20965/ijat.2023.p0206
  27. [27] S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, Q. Jiang, C. Li, J. Yang, H. Su, J. Zhu, and L. Zhang, “Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection,” Proc. European Conf. on Computer Vision (ECCV), pp. 38-55, 2024. https://doi.org/10.1007/978-3-031-72970-6_3
  28. [28] A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y. Lo, P. Dollár, and R. Girshick, “Segment Anything,” Proc. IEEE/CVF Int. Conf. on Computer Vision (ICCV), pp. 3992-4003, 2023. https://doi.org/10.1109/ICCV51070.2023.00371
  29. [29] C. Zhang, D. Han, Y. Qiao, J. U. Kim, S.-H. Bae, S. Lee, and C. S. Hong, “Faster Segment Anything: Towards Lightweight SAM for Mobile Applications,” arXiv preprint, arXiv:2306.14289, 2023. https://doi.org/10.48550/arXiv.2306.14289
  30. [30] M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Finn et al., “Do As I Can, Not As I Say: Grounding Language in Robotic Affordances,” Proc. 6th Conf. on Robot Learning (CoRL), PMLR, Vol.205, pp. 287-318, 2022.
  31. [31] H. Zhang, H. Deng, J. Ou, and C. Feng, “Mitigating Spatial Hallucination in Large Language Models for Path Planning via Prompt Engineering,” Sci. Rep., Vol.15, Article No.8881, 2025. https://doi.org/10.1038/s41598-025-93601-5
  32. [32] S. Macenski et al., “The Marathon 2: A Navigation System,” Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), 2020. https://doi.org/10.1109/IROS45743.2020.9341207
  33. [33] I. Armeni, Z.-Y. He, A. Zamir, J. Gwak, J. Malik, M. Fischer, and S. Savarese, “3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera,” Proc. IEEE Int. Conf. on Computer Vision (ICCV), pp. 5664-5673, 2019. https://doi.org/10.1109/ICCV.2019.00576
  34. [34] U.-H. Kim, J.-M. Park, T.-J. Song, and J.-H. Kim, “3-D Scene Graph: A Sparse and Semantic Representation of Physical Environments for Intelligent Agents,” IEEE Trans. Cybern., Vol.50, No.12, pp. 4921-4933, 2020. https://doi.org/10.1109/TCYB.2019.2931042
  35. [35] Y. Tian, Y. Chang, F. H. Arias, C. Nieto-Granda, J. P. How, and L. Carlone, “Kimera-Multi: Robust, Distributed, Dense Metric-Semantic SLAM for Multi-Robot Systems,” IEEE Trans. on Robotics, Vol.38, No.4, pp. 2022-2038, 2022. https://doi.org/10.1109/TRO.2021.3137751
  36. [36] S. H. Allu, I. Kadosh, T. Summers, and Y. Xiang, “A Modular Robotic System for Autonomous Exploration and Semantic Updating in Large-Scale Indoor Environments,” arXiv preprint, arXiv:2409.15493, 2024. https://doi.org/10.48550/arXiv.2409.15493
  37. [37] N. Yokoyama, S. Ha, D. Batra, J. Wang, and B. Bucher, “VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigation,” Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 42-48, 2024. https://doi.org/10.1109/ICRA57147.2024.10610712
  38. [38] D. S. Chaplot, D. Gandhi, A. Gupta, and R. Salakhutdinov, “Object Goal Navigation Using Goal-Oriented Semantic Exploration,” Advances in Neural Information Processing Systems (NeurIPS), 2020.
  39. [39] N. Zantout, H. Zhang, P. Kachana, J. Qiu, G. Chen, J. Zhang, and W. Wang, “SORT3D: Spatial Object-Centric Reasoning Toolbox for Zero-Shot 3D Grounding Using Large Language Models,” Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), pp. 2201-2208, 2025. https://doi.org/10.1109/IROS60139.2025.11246145
  40. [40] X. Huang, S. Zhao, Y. Wang et al., “MSGNav: Unleashing the Power of Multi-modal 3D Scene Graph for Zero-Shot Embodied Navigation,” arXiv preprint, arXiv:2511.10376, 2025. https://doi.org/10.48550/arXiv.2511.10376
  41. [41] H. Zhou, X. Wang, H. Li et al., “LagMemo: Language 3D Gaussian Splatting Memory for Multi-modal Open-Vocabulary Multi-Goal Visual Navigation,” arXiv preprint, arXiv:2510.24118, 2025. https://doi.org/10.48550/arXiv.2510.24118
  42. [42] J. Crespo, J. C. Castillo, O. M. Mozos et al., “Semantic Information for Robot Navigation: A Survey,” Applied Sciences, Vol.10, No.2, Article No.497, 2020. https://doi.org/10.3390/app10020497
  43. [43] R. Alqobali, M. Alshmrani, R. Alnasser et al., “A Survey on Robot Semantic Navigation Systems for Indoor Environments,” Applied Sciences, Vol.14, No.1, Article No.89, 2024. https://doi.org/10.3390/app14010089
  44. [44] L. Yang, B. Kang, Z. Huang, X. Xu, J. Feng, and H. Zhao, “Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data,” Proc. IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 10371-10381, 2024. https://doi.org/10.1109/CVPR52733.2024.00987
  45. [45] A. Junaedy, H. Masuta, Y. Fuse, K. Sawai, T. Motoyoshi, and N. Takagi, “Online Topological Mapping on a Quadcopter with Fast Growing Neural Gas,” J. Adv. Comput. Intell. Intell. Inform., Vol.28, No.6, pp. 1354-1366, 2024. https://doi.org/10.20965/jaciii.2024.p1354
  46. [46] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning Transferable Visual Models from Natural Language Supervision,” Proc. Int. Conf. on Machine Learning (ICML), PMLR, Vol.139, pp. 8748-8763, 2021.
  47. [47] J. Long, E. Shelhamer, and T. Darrell, “Fully Convolutional Networks for Semantic Segmentation,” Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 3431-3440, 2015. https://doi.org/10.1109/cvpr.2015.7298965
  48. [48] D. Eigen and R. Fergus, “Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture,” Proc. IEEE Int. Conf. on Computer Vision (ICCV), pp. 2650-2658, 2015. https://doi.org/10.1109/iccv.2015.304
  49. [49] C. Hazirbas, L. Ma, C. Domokos, and D. Cremers, “FuseNet: Incorporating Depth into Semantic Segmentation via Fusion-based CNN Architecture,” Asian Conf. on Computer Vision (ACCV), pp. 213-228, 2016. https://doi.org/10.1007/978-3-319-54181-5_14
  50. [50] S. Lee, S.-J. Park, and K.-S. Hong, “RDFNet: RGB-D Multi-level Residual Feature Fusion for Indoor Semantic Segmentation,” Proc. IEEE Int. Conf. on Computer Vision (ICCV), pp. 4990-4999, 2017. https://doi.org/10.1109/iccv.2017.533
  51. [51] J. McCormac, A. Handa, A. Davison, and S. Leutenegger, “SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networks,” Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 4628-4635, 2017. https://doi.org/10.1109/icra.2017.7989538
  52. [52] X. Hu, K. Yang, L. Fei, and K. Wang, “ACNet: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentation,” Proc. IEEE Int. Conf. on Image Processing (ICIP), pp. 1440-1444, 2019. https://doi.org/10.1109/icip.2019.8803025
  53. [53] X. Chen, K.-Y. Lin, J. Wang, W. Wu, C. Qian, H. Li, and G. Zeng, “Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation,” European Conf. on Computer Vision (ECCV), pp. 561-577, 2020. https://doi.org/10.1007/978-3-030-58621-8_33
  54. [54] D. Seichter, M. Köhler, B. Lewandowski, T. Wengefeld, and H.-M. Gross, “Efficient RGB-D Semantic Segmentation for Indoor Scene Analysis,” Proc. IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 13525-13531, 2021. https://doi.org/10.1109/ICRA48506.2021.9561675
  55. [55] K. Muravyev and K. Yakovlev, “Evaluation of Topological Mapping Methods in Indoor Environments,” IEEE Access, Vol.11, pp. 132683-132698, 2023. https://doi.org/10.1109/ACCESS.2023.3335818
  56. [56] B. Kuipers, “The Spatial Semantic Hierarchy,” Artif. Intell., Vol.119, Nos.1-2, pp. 191-233, 2000. https://doi.org/10.1016/S0004-3702(00)00017-5
  57. [57] S. Tellex, T. Kollar, S. Dickerson, M. Walter, A. Banerjee, S. Teller, and N. Roy, “Understanding Natural Language Commands for Robotic Navigation and Mobile Manipulation,” Proc. AAAI Conf. on Artificial Intelligence, pp. 1507-1514, 2011. https://doi.org/10.1609/aaai.v25i1.7979
  58. [58] S. Stange and S. Kopp, “Effects of a Social Robot’s Self-Explanations on How Humans Understand and Evaluate Its Behavior,” Proc. ACM/IEEE Int. Conf. on Human-Robot Interaction (HRI), pp. 619-627, 2020. https://doi.org/10.1145/3319502.3374802
  59. [59] O. Sotomi, D. Kodi, and A. Arab, “Trust Through Transparency: Explainable Social Navigation for Autonomous Mobile Robots via Vision-Language Models,” arXiv preprint, arXiv:2504.05477, 2025. https://doi.org/10.48550/arXiv.2504.05477

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Nov. 20, 2026