Conventional model theories are not suitable to control soft-bodied robots as deformable materials present rapidly changing behaviors. Neuromorphic electronics are now entering the field of robotics, demonstrating that a highly integrated device can mimic the fundamental properties of a sensory synaptic system, including learning and proprioception. This research work focuses on the physical implementation of a reservoir computing-based network to actuate a soft-bodied robot. More specifically, modeling the hysteresis of a shape memory alloy (SMA) using echo state networks (ESN) in real-world situations represents a novel approach to enable soft machines with task-learning. In this work, we show that not only does our ESN model enable our SMA-based robot with locomotion, but it also discovers a successful strategy to do so. Compared to standard control modeling, established either by theoretical frameworks or from experimental data, here, we gained knowledge a posteriori, guided by the physical interactions between the trained model and the controlled actuator, interactions from which striking patterns emerged, and informed us about what type of locomotion would work best for our robot.

1. [1] C. Majidi, “Soft Robotics: A Perspective – Current Trends and Prospects for the Future,” Soft Robotics, Vol.1, Issue 1, doi: 10.1089/soro.2013.0001, July 17, 2013.
2. [2] C. Duriez, “Control of elastic soft robots based on real-time finite element method,” 2013 IEEE Int. Conf. on Robotics and Automation, pp. 3982-3987, Karlsruhe, doi: 10.1109/ICRA.2013.6631138, 2013.
3. [3] T. G. Thuruthel, E. Falotico, F. Renda, and C. Laschi, “Model-based reinforcement learning for closed-loop dynamic control of soft robotic manipulators,” IEEE Trans. on Robotics, Vol.35, No.1, pp. 124-134, 2018.
4. [4] M. Ishige, T. Umedachi, Y. Ijiri, T. Taniguchi, and Y. Kawahara, “Blind Bin Picking of Small Screws Through In-finger Manipulation With Compliant Robotic Fingers,” 2020 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS2020), pp. 9337-9344, 2020.
5. [5] A. Tariverdi, V. K. Venkiteswaran, M. Richter, O. J. Elle, J. Tørresen, K. Mathiassen, S. Misra, and Ø. G. Martinsen, “A Recurrent Neural-Network-Based Real-Time Dynamic Model for Soft Continuum Manipulators,” Frontiers in Robotics and AI, Vol.8, 45, 2021.
6. [6] H. Jaeger, “The “echo state” approach to analysing and training recurrent neural networks,” GMD report no.148, GMD – German National Research Institute for Computer Science, 2001.
7. [7] Kudithipudi et al., “Design and Analysis of a Neuromemristive Reservoir Computing Architecture for Biosignal Processing,” Frontiers in Neuroscience, Vol.9, 502, 2016.
8. [8] K. Nakajima, “Physical reservoir computing an introductory perspective,” Jpn. J. Appl. Phys., Vol.59, 060501, 2020.
9. [9] K. Nakajima, H. Hauser, R. Kang, E. Guglielmino, D. G. Caldwell, and R. Pfeifer, “A soft body as a reservoir: case studies in a dynamic model of octopus-inspired soft robotic arm,” Frontiers in Computational Neuroscience, Vol.7, 91, doi: 10.3389/fncom.2013.00091, 2013.
10. [10] M. Lukoševičius and H. Jaeger, “Reservoir computing approaches to recurrent neural network training,” Comput. Sci. Rev., Vol.3, No.3, pp. 127-149, 2009.
11. [11] G. Tanaka, T. Yamane, J. B. Héroux, R. Nakane, N. Kanazawa, S. Takeda, H. Numata, D. Nakano, and A. Hirose, “Recent advances in physical reservoir computing: A review,” Neural Networks, Vol.115, pp. 100-123, doi: 10.1016/j.neunet.2019.03.005, 2019.
12. [12] T. D. Ta, T. Umedachi, and Y. Kawahara, “Inkjet Printable Actuators and Sensors for Soft-Bodied Crawling Robots,” Proc. of 2019 Int. Conf. on Robotics and Automation (ICRA), pp. 3658-3664, May 2019.
13. [13] D. R. Reynolds and P. Kloucek, “Numerical Modeling of Nonlinear Thermodynamics in SMA Wires,” Eighth Copper Mountain Conf. on Iterative Methods Copper Mountain, CO, United States, March 28 to April 2, 2004.
14. [14] F. Auricchio and E. Sacco, “A one-dimensional model for superelastic shape-memory alloys with different elastic properties between austenite and martensite,” Int. J. of Non-Linear Mechanics, Vol.32, Issue 6, pp. 1101-1114, doi: 10.1016/S0020-7462(96)00130-8, 1997.
15. [15] K. Tanaka, S. Kobayashi, and Y. Sato, “Thermomechanics of transformation pseudoelasticity and shape memory effect in alloys,” Int. J. of Plasticity, Vol.2, pp. 59-72, 1986.
16. [16] C. Liang and A. Rogers, “One-dimensional thermomechanical constitutive relations for shape memory materials,” J. of Intelligent Material Systems and Structures, Vol.1, No.2, pp. 207-234, 1990.
17. [17] L. C. Brinson, “One-dimensional constitutive behavior of shape memory alloys: Thermomechanical derivation with non-constant material functions and redefined martensite internal variable,” J. of Intelligent Material Systems and Structures, Vol.4, 1993.
18. [18] T. Okuyama, K. Yakuwa, Y. Luo, M. Higa, and T. Takagi, “Modeling the mechanical behavior of an SMA manipulator,” Int. J. of Applied Electromagnetics and Mechanics, Vol.25, No.1-4, pp. 495-501, 2007.
19. [19] H. Kino, N. Mori, S. Moribe, K. Tsuda, and K. Tahara, “Experiment Verification and Stability Analysis of Iterative Learning Control for Shape Memory Alloy Wire,” J. Robot. Mechatron., Vol.31, No.4, pp. 583-593, 2019.
20. [20] U. D. Schiller and J. J. Steil, “Analyzing the weight dynamics of recurrent learning algorithms,” Neurocomput., Vol.63, pp. 5-23, doi: 10.1016/j.neucom.2004.04.006, 2005.
21. [21] F. Wyffels and B. Schrauwen, “Design of a Central Pattern Generator Using Reservoir Computing for Learning Human Motion,” 2009 Advanced Technologies for Enhanced Quality of Life, pp. 118-122, doi: 10.1109/AT-EQUAL.2009.32, 2009.
22. [22] H. Jaeger, “Echo state network,” Scholarpedia, Vol.2, No.9, 2330, 2007.
23. [23] D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” Proc. of 3rd Int. Conf. on Learning Representations (ICLR 2015), San Diego, CA, USA, May 7-9, 2015.