Learning of Whole Arm Manipulation with Constraint of Contact Mode Maintaining
Nobuyuki Kawarai and Yuichi Kobayashi
Tokyo University of Agriculture and Technology, 2-14-16 Naka-cho, Koganei, Tokyo 184-8588, Japan
This paper proposes the learning of whole arm manipulation with a two-link manipulator. Our proposal combines a controller obtained by reinforcement learning (actor-critic) and a learning classifier realized by a Support Vector Machine (SVM). The classifier learns the boundary between slip and stick modes in torque space. Using the result of classification, the robot learns to move the object toward desired position while keeping the desired contact modes. Control input (torque) is first specified by the actor. The SVM classifier judges whether torque can maintain the desired slip or stick mode and, if not, it modifies the torque so that the desired mode is maintained. It was verified in the simulation that our proposed learning realized accelerating of the object and decelerating it while keeping the desired mode, i.e., avoiding undesired slipping of the object.
-  M. J. Cherif and K. K. Gupta, “Planning quasi-static fingertip manipulations for reconfiguring objects,” IEEE Trans. on Robotics and Automation, Vol.15, No.5, pp. 837-848, 1999.
-  M. Yashima, Y. Shiina, and H. Yamaguchi, “Randomized Manipulation Planning for A Multi-Fingered Hand by Switching Contact Modes,” Proc. of IEEE Int. Conf. on Robotics and Automation, 2003.
-  E. Yoshida, P. Blazevic, V. Hugel, K. Yokoi, and K. Harada, “Pivoting a Large Object: Whole-body Manipulation by a Humanoid Robot,” Applied Bionics and Biomechanics, Vol.3, No.3, pp. 227-235, 2006.
-  J. Nakanishi, J. Morimoto, G. Endo, G. Cheng, S. Schaal, and M. Kawato, “Learning from demonstration and adaptation of biped locomotion,” Robotics and Autonomous Systems, Vol.47, No.2-3, pp. 79-91, 2004.
-  R. Sutton and A. Barto, “Reinforcement Learning,” MIT Press, 1998.
-  H. Kimura, T. Yamashita, and S. Kobayashi, “Reinforcement Learning of Walking Behavior for a Four-Legged Robot,” Proc. of IEEE Conf. on Decision and Control, pp. 411-416, 2001.
-  J. Morimoto and K. Doya, “Acquisition of stand-up behavior by a real robot using hierarchical reinforcement learning,” Robotics and Autonomous Systems, Vol.36, No.1, pp. 37-51, 2001.
-  H. Miyamoto, J. Morimoto, K. Doya, and M. Kawato, “Reinforcement learning with via-point representation,” Neural Networks, Vol.17, No.3, pp. 299-305, 2004.
-  Y. Kobayashi, M. Shibata, S. Hosoe, and Y. Uno, “Learning of object manipulation with stick/slip mode switching,” Proc. of Int. Conf. on Intelligent Robots and Systems, pp. 373-379, 2008.
-  T. Odashima, et al., “A Soft Human-Interactive Robot RI-MAN,” Video Proc. of IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, 2006.
-  S. Nakaoka, S. Hattori, F. Kanehiro, S. Kajita, and H. Hirukawa, “Constraint-based Dynamics Simulator for Humanoid Robots with Shock Absorbing Mechanisms,” The 2007 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, 2007.
-  A. Schaft and H. Schumacher, “An Introduction to Hybrid Dynamical Systems,” Springer, 2000.
-  V. N. Vapnik, “The Nature of Statistical Learning Theory,” Springer,1995.
-  O. L. Mangasarian and D. R. Musicant, “Lagrangian Support Vector Machines,” J. of Machine Learning Research, Vol.1, pp. 161-177, 2001.
-  S. Bhatnagar, R. Sutton, M. Ghavamzadeh, and M. Lee, “Natural Actor-Critic Algorithms,” Automatica, Vol.45, No.11, pp. 2471-2482, 2009.
-  D. W. Scott and S. R. Sain, “Multi-Dimensional Density Estimation,” Handbook of Statistics – Vol 23: Data Mining and Computational Statistics, 2004.
-  T. Schlegl, M. Buss, and G. Schmidt, “Hybrid Control of Multi-fingered Dexterous Robotic Hands,” S. Engell, G. Frehse, E. Schnieder (Eds.): Modelling, Analysis and Design of Hybrid Systems, LNCIS, Vol.279, pp. 437-465, 2002.
-  Y. Yin, S. Hosoe, and Z. Luo, “A Mixed Logic Dynamical Modeling Formulation and Optimal Control of Intelligent Robots,” Optimization Engineering, Vol.8, pp. 321-340, 2007.
This article is published under a Creative Commons Attribution-NoDerivatives 4.0 International License.
Copyright© 2010 by Fuji Technology Press Ltd. and Japan Society of Mechanical Engineers. All right reserved.