Autonomous Vehicle Path Tracking Based on Natural Gradient Methods
Ki-Young Kwon*, Keun-Woo Jung**, Dong-Su Yang*,
and Jooyoung Park*
*Department of Control and Instrumentation Engineering, Korea University, Sejong-ro 2511, Sejong 339-700, Korea
**LG Electronics, Gasan-dong, Geumcheon-gu, Seoul 153-802, Korea
Recently, reinforcement learning and evolution strategy have become major tools in the field of machine learning, and have shown excellent performance in various engineering problems. In particular, the Natural Actor-Critic (NAC) approach and the Natural Evolution Strategies (NES) have led to considerable interests in the area of natural-gradient-based machine learning methods with many successful applications. In this paper, we apply the NAC and the NES to pathtracking control problems for autonomous vehicles. Simulation results show that these methods can yield better performance compared to the conventional PID controllers.
-  R. S. Sutton and A. G. Barto, “Reinforcement Learning: An Introduction, MIT Press,” Cambridge, MA, 1998
-  D.Wierstra, T. Schaul, T. Glasmachers, Y. Sun, and J. Schmidhuber, “Natural evolution strategies,” arXiv:1106.4487, 2011.
-  Y. Sun, D. Wierstra, T. Schaul, and J. Schmidhuber, “Stochastic search using the natural gradient,” Proc. of ICML’09, pp. 1161-1168, 2009.
-  J. Peters and S. Schaal, “Natural actor-critic,” Neurocomputing, Vol.71, pp. 1180-1190, 2008.
-  D. Min, K. Jung, K. Kwon, and J. Park, “Mobile robot control based on a recent reinforcement learning method,” Proc. of KIIS Spring Conf. 2011, Vol.21, No.1, pp. 67-70, 2011.
-  J. Park, J. Kim, and D. Kang, “An RLS-based natural actor-critic algorithm for locomotion of a two-linked robot arm,” Lecture Notes in Artificial Intelligence, Vol.3801, pp. 65-72, December 2005.
-  B. Kim, J. Park, S. Park, and S. Kang, “Impedance learning for robotic contact tasks using natural actor-critic algorithm,” IEEE Trans. on Systems, Man, and Cybernetics, Part B: Cybernetics, Vol.40, No.2, pp. 433-443, April 2010.
-  M. Riedmiller, J. Peters, and S. Schaal, “Evaluation of policy gradient methods and variants on the cart-pole benchmark,” Proc. of 2007 IEEE Int. Symposium on Approximate Dynamic Programming and Reinforcement Learning, pp. 254-261, 2007.
-  X. Xu, H. Zhang, B. Dai, and H. He, “Self-learning path-tracking control of autonomous vehicles using kernel-based approximate dynamic programming,” Proc. of Int. Joint Conf. on Neural Networks 2008, pp. 2182-2189, 2008.
-  J. Guldner, H. Tan, and S. Patwardhan, “On fundamental issues of vehicle steering control for highway automation,” Technical Report, California PATHWorking Paper, UCB-ITS-PWP-97-11, University of California, Berkley, 1997.
-  G. Lu, J. Huang, and M. Tomizuka, “Vehicle lateral control under fault in front and/or rear sensor,” California PATH Research Report UCB-ITS-PRR-2003-26, University of California, Berkley, 2003.