An online-usable digital twin system integrated into a control system is proposed and developed in this study to control an autonomous mobile robot operating on a complex three-dimensional (3D) terrain that includes slopes and uneven surfaces. The system is equipped with validated functions that are necessary for virtual or physical autonomous navigation. In this development, (1) 3D virtual environment, 3D virtual robot model, and 3D virtual sensor models are constructed to accurately reproduce the motion of a mobile robot on a computer. Additionally, (2) by connecting these virtual and physical models through an interface, we integrate them as a digital twin and implement an autonomous navigation control system that is synchronized with a motion simulation by switching between a physical and virtual robot (through a physics engine). This allows the same control system to be applied to both the virtual and physical models. The virtual model is created using the Unity 3D game engine, which integrates environmental terrain data and robot models to enable 3D physical simulations including road slopes and unevenness. Additionally, a path-setting method using Bézier curves and a path-following algorithm are implemented in the simulation system. Autonomous navigation of the mobile robot through the digital twin system is achieved by combining the functions above in a manner that allows online use during operation. Finally, autonomous navigation tests are conducted in a physical environment to confirm the effectiveness of the developed system.

1. [1] A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun, “CARLA: An open urban driving simulator,” Proc. 1st Annu. Conf. Robot Learn., 2017.
2. [2] G. Rong et al., “LGSVL simulator: A high fidelity simulator for autonomous driving,” IEEE 23rd Int. Conf. Intell. Transp. Syst., 2020. https://doi.org/10.1109/ITSC45102.2020.9294422
3. [3] S. Shah, D. Dey, C. Lovett, and A. Kapoor, “AirSim: High-fidelity visual and physical simulation for autonomous vehicles,” Field and Service Robotics: Results of the 11th Int. Conf., pp. 621-635, 2018. https://doi.org/10.1007/978-3-319-67361-5_40
4. [4] W. Zhao, J. P. Queralta, and T. Westerlund, “Sim-to-real transfer in deep reinforcement learning for robotics: A survey,” 2020 IEEE Symp. Ser. Comput. Intell., pp. 737-744, 2020. https://doi.org/10.1109/SSCI47803.2020.9308468
5. [5] J. Tan et al., “Sim-to-real: Learning agile locomotion for quadruped robots,” Proc. Robot.: Sci. Syst. XIV, 2018. https://doi.org/10.15607/RSS.2018.XIV.010
6. [6] S. James et al., “Sim-to-real via sim-to-sim: Data-efficient robotic grasping via randomized-to-canonical adaptation networks,” 2019 IEEE/CVF Conf. Comput. Vis. Pattern Recognit., pp. 12619-12629, 2019. https://doi.org/10.1109/CVPR.2019.01291
7. [7] V. Makoviychuk et al., “Isaac Gym: High performance GPU-based physics simulation for robot learning,” arXiv:2108.10470, 2021. https://doi.org/10.48550/arXiv.2108.10470
8. [8] K. Tomita, “Activities for new industrial production system of YASKAWA,” J. Robot. Soc. Jpn., Vol.35, No.2, pp. 114-117, 2017 (in Japanese). https://doi.org/10.7210/jrsj.35.114
9. [9] V. Bulej, M. Bartoš, V. Tlach, M. Bohušík, and D. Wiecek, “Simulation of manipulation task using iRVision aided robot control in Fanuc RoboGuide software,” IOP Conf. Ser.: Mater. Sci. Eng., Vol.1199, Article No.012091, 2021. https://doi.org/10.1088/1757-899X/1199/1/012091
10. [10] ABB Robotics, “Operating manual: RobotStudio,” 2007.
11. [11] D. Elayaraja, D. Sarathkumar, and R. Srinivasalu, “Modelling and simulation of a pick and place robot using moto sim,” IETI Trans. Eng. Res. Pract., Vol.5, No.2, pp. 31-36, 2021. https://doi.org/10.6723/TERP.202112_5(2).0004
12. [12] K. Shimane, R. Ueda, and S. Tarao, “Prototyping of kinematics simulator for supporting autonomous mobile robot development,” J. Robot. Mechatron., Vol.28, No.4, pp. 470-478, 2016. https://doi.org/10.20965/jrm.2016.p0470
13. [13] A. Vemula, K. Muelling, and J. Oh, “Path planning in dynamic environments with adaptive dimensionality,” Proc. Int. Symp. Comb. Search, Vol.7, No.1, pp. 107-115, 2016. https://doi.org/10.1609/socs.v7i1.18386
14. [14] J. Bae and W. Chung, “Efficient path planning for multiple transportation robots under various loading conditions,” Int. J. Adv. Robot. Syst., Vol.16, No.2, 2019. https://doi.org/10.1177/1729881419835110
15. [15] T. Tomizawa et al., “A robust localization for unknown obstacle based on the gridmap matching,” J. Robot. Soc. Jpn., Vol.30, No.3, pp. 280-286, 2012 (in Japanese). https://doi.org/10.7210/jrsj.30.280
16. [16] S. Yuta, “Open experiment of autonomous navigation of mobile robots in the city: Tsukuba Challenge 2014 and the results,” J. Robot. Mechatron., Vol.27, No.4, pp. 318-326, 2015. https://doi.org/10.20965/jrm.2015.p0318
17. [17] N. Koenig and A. Howard, “Design and use paradigms for Gazebo, an open-source multi-robot simulator,” 2004 IEEE/RSJ Int. Conf. Intell. Robots Syst., Vol.3, pp. 2149-2154, 2004. https://doi.org/10.1109/IROS.2004.1389727
18. [18] S. Nakaoka, “Choreonoid: Extensible virtual robot environment built on an integrated GUI framework,” 2012 IEEE/SICE Int. Symp. Syst. Integr., pp. 79-85, 2012. https://doi.org/10.1109/SII.2012.6427350
19. [19] T. Yoshida et al., “Mobile robot simulator for a single shot navigation task in Tsukuba Challenge 2019,” Proc. 20th Annu. Conf. Syst. Integr., pp. 1948-1952, 2019 (in Japanese).
20. [20] R. Tanaka and R. Awano, “Initiatives of TNK in Tsukuba Challenge 2020,” Tsukuba Challenge 2020 Symp. Report Collection, 2021 (in Japanese).
21. [21] R. Sekine, T. Tomizawa, and S. Tarao, “Trial of utilization of an environmental map generated by a high-precision 3D scanner for a mobile robot,” J. Robot. Mechatron., Vol.35, No.6, pp. 1469-1479, 2023. https://doi.org/10.20965/jrm.2023.p1469
22. [22] Y. Kanuki, N. Ohta, and N. Nakazawa, “Development of autonomous moving robot using appropriate technology for Tsukuba Challenge,” J. Robot. Mechatron., Vol.35, No.2, pp. 279-287, 2023. https://doi.org/10.20965/jrm.2023.p0279
23. [23] P. Jacobs and J. Canny, “Planning smooth paths for mobile robots,” Z. Li and J. F. Canny (Eds.), “Nonholonomic Motion Planning,” pp. 271-342, Springer, 1993. https://doi.org/10.1007/978-1-4615-3176-0_8
24. [24] P. Coelho and U. Nunes, “Path-following control of mobile robots in presence of uncertainties,” IEEE Trans. Robot., Vol.21, No.2, pp. 252-261, 2005. https://doi.org/10.1109/TRO.2004.837240
25. [25] S. Iida and S. Yuta, “Vehicle command system and trajectory control for autonomous mobile robots,” IEEE/RSJ Int. Workshop Intell. Robots Syst. ’91, Vol.1, pp. 212-217, 1991. https://doi.org/10.1109/IROS.1991.174452
26. [26] D. Fox, S. Thrun, W. Burgard, and F. Dellaert, “Particle filters for mobile robot localization,” A. Doucet, N. Freitas, and N. Gordon (Eds.), “Sequential Monte Carlo Methods in Practice,” pp. 401-428, Springer, 2001. https://doi.org/10.1007/978-1-4757-3437-9_19
27. [27] S. Chen et al., “NDT-LOAM: A real-time Lidar odometry and mapping with weighted NDT and LFA,” IEEE Sens. J., Vol.22, No.4, pp. 3660-3671, 2022. https://doi.org/10.1109/JSEN.2021.3135055
28. [28] A. V. Segal, D. Haehnel, and S. Thrun, “Generalized-ICP,” Robot.: Sci. Syst. V, pp. 161-168, 2010. https://doi.org/10.7551/mitpress/8727.003.0022
29. [29] S. Kurebayashi et al., “Tsukuba Challenge 2024 initiatives by National Institute of Technology Tokyo College SRC,” Tsukuba Challenge 2024 Symp. Report Collection, 2025 (in Japanese).