Top > JRM > Target Following for a Snake-Like Robot Using Rolling Locomoti ...

single-rb.php

JRM Vol.38 No.5 pp. 1413-1427
(2026)
doi: 10.20965/jrm.2026.p1413

Paper:

Views over last 60 days: 9

Target Following for a Snake-Like Robot Using Rolling Locomotion via Deep Reinforcement Learning

Tsuyoshi Kimoto ORCID Icon, Akio Yamano ORCID Icon, Satomi Suzuki, and Takashi Iwasa ORCID Icon

Department of Aerospace Engineering, Graduate School of Engineering, Osaka Metropolitan University
1-1 Gakuen-cho, Naka-ku, Sakai, Osaka 599-8531, Japan

Received:
March 25, 2026
Accepted:
July 3, 2026
Published:
October 20, 2026
Creative Commons License
Keywords:
snake-like robot, rolling locomotion, deep reinforcement learning, target following
Abstract

The turning motion of snake-like robots based on center of gravity shift has superior energy efficiency compared to the undulatory locomotion that simultaneously drives many joints, but precise guidance control has been difficult due to its nonlinear dynamic characteristics. In this paper, we propose a control method using deep reinforcement learning for following target points in all directions in the turning motion using center of gravity shift and ground reaction forces. The target direction is defined as a normalized vector in the body coordinate system. This allows us to present target direction information to the agent that is independent of changes in self-orientation, and we show that it can acquire a policy for accurate following in all directions. The results of motion analysis confirmed that the learned policy not only skillfully utilizes the ground reaction forces at the time of contact to make dynamic trajectory corrections, but also achieves target following by flexibly switching the turning strategy according to the target direction.

DRL framework for rolling locomotion

DRL framework for rolling locomotion

Full text
Cite this article as:
T. Kimoto, A. Yamano, S. Suzuki, and T. Iwasa, “Target Following for a Snake-Like Robot Using Rolling Locomotion via Deep Reinforcement Learning,” J. Robot. Mechatron., Vol.38 No.5, pp. 1413-1427, 2026.
Data files:
References
  1. [1] X. Zang, Y. Liu, Z. Lin, C. Zhang, and S. Iqbal, “Two multi-linked rescue robots: Design, construction and field tests,” J. of Advanced Mechanical Design, Systems, and Manufacturing, Vol.10, No.6, Article No.JAMDSM0089, 2016. https://doi.org/10.1299/jamdsm.2016jamdsm0089
  2. [2] T. Kamegawa, T. Akiyama, S. Sakai, K. Fujii, K. Une, Y. Wang, Y. Matsumura, T. Kishutani, E. Nose, Y. Yoshizaki, and A. Gofuku, “Development of a separable search-and-rescue robot composed of a mobile robot and a snake robot,” Advanced Robotics, Vol.34, No.2, pp. 132-139, 2020. https://doi.org/10.1080/01691864.2019.1691941
  3. [3] F. Enner, D. Rollinson, and H. Choset, “Motion estimation of snake robots in straight pipes,” 2013 IEEE Int. Conf. on Robotics and Automation, pp. 5168-5173, 2013. https://doi.org/10.1109/ICRA.2013.6631316
  4. [4] I. N. Ismail, A. Anuar, K. S. M. Sahari, M. Z. Baharuddin, M. Fairuz, A. Jalal, and J. M. Saad, “Development of in-pipe inspection robot: A review,” 2012 IEEE Conf. on Sustainable Utilization and Development in Engineering and Technology (STUDENT), pp. 310-315, 2012. https://doi.org/10.1109/STUDENT.2012.6408425
  5. [5] N. Matsumoto, M. Tanaka, M. Nakajima, M. Fujita, and K. Tadakuma, “Development of a folding arm on an articulated mobile robot for plant disaster prevention,” Advanced Robotics, Vol.34, No.2, pp. 89-103, 2020. https://doi.org/10.1080/01691864.2019.1689167
  6. [6] X. Hou, Y. Shi, L. Li, Y. Tian, Y. Su, T. Ding, and Z. Deng, “Revealing the Mechanical Characteristics via Kinematic Wave Model for Snake-Like Robot Executing Exploration of Lunar Craters,” IEEE Access, Vol.8, pp. 38368-38379, 2020. https://doi.org/10.1109/ACCESS.2020.2971221
  7. [7] T. S. Vaquero, G. Daddi, R. Thakker et al., “EELS: Autonomous snake-like robot with task and motion planning capabilities for ice world exploration,” Science Robotics, Vol.9, No.88, Article No.eadh8332, 2024. https://doi.org/10.1126/scirobotics.adh8332
  8. [8] M. Tesch, K. Lipkin, I. Brown, R. Hatton, A. Peck, J. Rembisz, and H. Choset, “Parameterized and Scripted Gaits for Modular Snake Robots,” Advanced Robotics, Vol.23, No.9, pp. 1131-1158, 2009. https://doi.org/10.1163/156855309X452566
  9. [9] T. Takemori, M. Tanaka, and F. Matsuno, “Hoop-Passing Motion for a Snake Robot to Realize Motion Transition Across Different Environments,” IEEE Trans. on Robotics, Vol.37, No.5, pp. 1696-1711, 2021. https://doi.org/10.1109/TRO.2021.3063438
  10. [10] T. Takemori, M. Tanaka, and F. Matsuno, “Gait design of a snake robot by connecting simple shapes,” 2016 IEEE Int. Symposium on Safety, Security, and Rescue Robotics (SSRR), pp. 189-194, 2016. https://doi.org/10.1109/SSRR.2016.7784297
  11. [11] M. Tesch, J. Schneider, and H. Choset, “Expensive multiobjective optimization for robotics,” 2013 IEEE Int. Conf. on Robotics and Automation, pp. 973-980, 2013. https://doi.org/10.1109/ICRA.2013.6630691
  12. [12] S. Hasanzadeh and A. A. Tootoonchi, “Ground adaptive and optimized locomotion of snake robot moving with a novel gait,” Autonomous Robots, Vol.28, pp. 457-470, 2010. https://doi.org/10.1007/s10514-010-9179-y
  13. [13] A. Crespi and A. J. Ijspeert, “Online Optimization of Swimming and Crawling in an Amphibious Snake Robot,” IEEE Trans. on Robotics, Vol.24, No.1, pp. 75-87, 2008. https://doi.org/10.1109/TRO.2008.915426
  14. [14] S. Jiang, A. Salagame, A. Ramezani, and L. L. S. Wong, “Snake Robot with Tactile Perception Navigates on Large-scale Challenging Terrain,” 2024 IEEE Int. Conf. on Robotics and Automation (ICRA), pp. 5090-5096, 2024. https://doi.org/10.1109/ICRA57147.2024.10611384
  15. [15] S. Jiang, A. Salagame, A. Ramezani, and L. L. Wong, “Hierarchical RL-Guided Large-scale Navigation of a Snake Robot,” 2024 IEEE Int. Conf. on Advanced Intelligent Mechatronics (AIM), pp. 1347-1352, 2024. https://doi.org/10.1109/AIM55361.2024.10637006
  16. [16] J. Naish, J. Rodriguez, J. Zhang, B. Jones, G. Daddi, A. Orekhov, R. Royce, M. Paton, H. Choset, M. Ono, and R. Thakker, “Reinforcement learning-driven parametric curve fitting for snake robot gait design,” Proc. of the 6th Annual Learning for Dynamics & Control Conf., pp. 1715-1727, 2024.
  17. [17] H. Kurokawa, A. Kamimura, E. Yoshida, K. Tomita, S. Murata, and S. Kokaji, “Self-reconfigurable modular robot (M-TRAN) and its motion design,” 7th Int. Conf. on Control, Automation, Robotics and Vision (ICARCV 2002) pp. 51-56, 2002. https://doi.org/10.1109/ICARCV.2002.1234789
  18. [18] J. Zhao, X. Cui, Y. Zhu, and S. Tang, “A new self-reconfigurable modular robotic system UBot: Multi-mode locomotion and self-reconfiguration,” 2011 IEEE Int. Conf. on Robotics and Automation, pp. 1020-1025, 2011. https://doi.org/10.1109/ICRA.2011.5980293
  19. [19] X. Wang, H. Jin, Y. Zhu, B. Chen, D. Bie, Y. Zhang, and J. Zhao, “Serpenoid polygonal rolling for chain-type modular robots: A study of modeling, pattern switching and application,” Robotics and Computer-Integrated Manufacturing, Vol.39, pp. 56-67, 2016. https://doi.org/10.1016/j.rcim.2015.12.003
  20. [20] A. Salagame, N. Bhattachan, A. Caetano et al., “How Strong a Kick Should be to Topple Northeastern’s Tumbling Robot?,” 2024 IEEE Int. Conf. on Advanced Intelligent Mechatronics (AIM), pp. 76-81, 2024. https://doi.org/10.1109/AIM55361.2024.10637166
  21. [21] A. Salagame, H. Noyes, E. Sihite, A. Kalantari, and A. Ramezani, “Crater Observing Bioinspired Rolling Articulator (COBRA),” Advanced Intelligent Systems, Vol.8, No.3, Article No.2500352, 2026. https://doi.org/10.1002/aisy.202500352
  22. [22] K. Miki, C. W. Chin, and M. Tanaka, “Snake Robot with Opposing Helix Gait for Transporting Cylindrical Objects,” IEEE Robotics and Automation Letters, Vol.11, No.2, pp. 2098-2105, 2026. https://doi.org/10.1109/LRA.2025.3645515
  23. [23] A. Yamano, Y. Ikeda, K. Imai, and M. Chiba, “Efficient rolling motion for snake-like robots utilizing center of gravity shift,” Mechatronics, Vol.94, Article No.103024, 2023. https://doi.org/10.1016/j.mechatronics.2023.103024
  24. [24] Y. Sugiyama and S. Hirai, “Crawling and Jumping by a Deformable Robot,” The Int. J. of Robotics Research, Vol.25, Nos.5-6, pp. 603-620, 2006. https://doi.org/10.1177/0278364906065386
  25. [25] Y. Wang, C. Wu, L. Yu, and Y. Mei, “Trajectory planning of a rolling robot of closed five-bow-shaped-bar linkage,” Robotics and Computer-Integrated Manufacturing, Vol.53, pp. 81-92, 2018. https://doi.org/10.1016/j.rcim.2018.03.004
  26. [26] Y. Wang, C. Wu, L. Yu, and Y. Mei, “Dynamics of a rolling robot of closed five-arc-shaped-bar linkage,” Mechanism and Machine Theory, Vol.121, pp. 75-91, 2018. https://doi.org/10.1016/j.mechmachtheory.2017.10.010
  27. [27] A. Yamano, T. Kimoto, and M. Chiba, “Design of Rolling Motion for Snake-like Robots using Center-of-Gravity Shift,” J. of Mechanisms and Robotics, Vol.17, No.6, Article No.061011, 2025. https://doi.org/10.1115/1.4067307
  28. [28] T. Kimoto, S. Suzuki, A. Yamano, and T. Iwasa, “Rolling Motion Control in a Snake Robot Using Center-of-Mass Shift and Reinforcement Learning-Based Motion Exploration,” 2025 JSME Conf. on Robotics and Mechatronics, Vol.2025, Article No.2P1-P11, 2025 (in Japanese). https://doi.org/10.1299/jsmermd.2025.2P1-P11
  29. [29] S. Suzuki, A. Yamano, T. Kimoto, and T. Iwasa, “Study on the Rolling Motion of a Snake-Like Robot That Transforms into a Parallel Two-Wheeled Vehicle Using Deep Reinforcement Learning,” 2025 25th Int. Conf. on Control, Automation and Systems (ICCAS), pp. 1485-1490, 2025. https://doi.org/10.23919/ICCAS66577.2025.11301120
  30. [30] A. Yamano, S. Suzuki, T. Kimoto, and T. Iwasa, “Deep reinforcement learning-based design with observation buffer of rolling motion for snake-like robots,” Robotics and Autonomous Systems, Vol.199, Article No.105370, 2026. https://doi.org/https://doi.org/10.1016/j.robot.2026.105370
  31. [31] H. C. Astley, C. Gong, J. Dai, M. Travers, M. M. Serrano, P. A. Vela, H. Choset, J. Mendelson III, D. Hu, and D. Goldman, “Modulation of orthogonal body waves enables high maneuverability in sidewinding locomotion,” PNAS, Vol.112, No.19, pp. 6200-6205, 2015. https://doi.org/10.1073/pnas.1418965112
  32. [32] R. Ariizumi and F. Matsuno, “Dynamic Analysis of Three Snake Robot Gaits,” IEEE Trans. on Robotics, Vol.33, No.5, pp. 1075-1087, 2017. https://doi.org/10.1109/TRO.2017.2704581
  33. [33] E. Kelasidi, P. Liljebäck, K. Y. Pettersen, and J. T. Gravdahl, “Experimental investigation of efficient locomotion of underwater snake robots for lateral undulation and eel-like motion patterns,” Robotics and Biomimetics, Vol.2, Article No.8, 2015. https://doi.org/10.1186/s40638-015-0029-4
  34. [34] A. Yamano, K. Shimizu, M. Chiba, and H. Ijima, “Fluid force identification acting on snake-like robots swimming in viscous fluids,” J. of Fluids and Structures, Vol.106, Article No.103351, 2021. https://doi.org/10.1016/j.jfluidstructs.2021.103351
  35. [35] K. V. Katsikopoulos and S. E. Engelbrecht, “Markov decision processes with delays and asynchronous cost collection,” IEEE Trans. on Automatic Control, Vol.48, No.4, pp. 568-574, 2003. https://doi.org/10.1109/TAC.2003.809799
  36. [36] J. Ikemoto and T. Ushio, “Application of Deep Reinforcement Learning to Control Problems,” The Brain & Neural Networks, Vol.26, No.4, pp. 135-144, 2019 (in Japanese). https://doi.org/10.3902/jnns.26.135
  37. [37] T. Haarnoja, S. Ha, A. Zhou, J. Tan, G. Tucker, and S. Levine, “Learning to Walk via Deep Reinforcement Learning,” ArXiv preprint, arXiv:1812.11103, 2018. https://doi.org/10.48550/arXiv.1812.11103
  38. [38] T. Haarnoja, A. Zhou, K. Hartikainen, G. Tucker, S. Ha, J. Tan, V. Kumar, H. Zhu, A. Gupta, P. Abbeel, and S. Levine, “Soft Actor-Critic Algorithms and Applications,” ArXiv preprint, arXiv:1812.05905, 2018. https://doi.org/10.48550/arXiv.1812.05905
  39. [39] T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor,” ArXiv preprint, arXiv:1801.01290, 2018. https://doi.org/10.48550/arXiv.1801.01290
  40. [40] A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann, “Stable-Baselines3: Reliable Reinforcement Learning Implementations,” J. of Machine Learning Research, Vol.22, No.268, pp. 1-8, 2021.
  41. [41] G. Authors, “Genesis: A Generative and Universal Physics Engine for Robotics and Beyond,” 2024. https://github.com/Genesis-Embodied-AI/genesis-world [Accessed October 1, 2026]

Creative Commons License  This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Oct. 19, 2026