JRM Vol.35 No.5 pp. 1385-1392
doi: 10.20965/jrm.2023.p1385


Behavior Learning System for Robot Soccer Using Neural Network

Moeko Tominaga* ORCID Icon, Yasunori Takemura* ORCID Icon, and Kazuo Ishii**

*Nishinippon Institute of Technology
1-11 Aratsu, Kanda, Miyako-gun, Fukuoka 800-0397, Japan

**Kyutech Institute of Technology
2-4 Hibikino, Wakamatsu-ku, Kitakyushu, Fukuoka 808-0196, Japan

November 28, 2022
August 21, 2023
October 20, 2023
autonomous behavior selection, neural network, human-robot coordination

With technological developments, the prospect of a human-robot symbiotic society has emerged. A soccer game has characteristics similar to those expected in such a society. Soccer is a multiagent game in which the strategy employed depends on each agent’s position and actions. This paper discusses the results of the development of a learning system that uses a self-organizing map to select behaviors depending on the scenario (two-dimensional absolute coordinates of the agent, other agents, and the ball). The system can reproduce the action-selection algorithms of all the players on a certain team, and the robot can instantly select the next cooperative action from information obtained during the game. Thus, common-sense rules can be shared to learn an action-selection algorithm for a set of both human and robot agents.

Validation of behavior learning system

Validation of behavior learning system

Cite this article as:
M. Tominaga, Y. Takemura, and K. Ishii, “Behavior Learning System for Robot Soccer Using Neural Network,” J. Robot. Mechatron., Vol.35 No.5, pp. 1385-1392, 2023.
Data files:
  1. [1] M. Tominaga, Y. Takemura, and K. Ishii, “Analysis of Cooperative Behavior in Futsal for Human and Robot Multi Agent System,” Proc. of JSME Annual Conf. on Robotics and Mechatronics (Robomec), 2P2-L09, 2018.
  2. [2] P. Stone and M. Veloso, “Multiagent Systems: A Survey from a Machine Learning Perspective,” Auton. Robots, Vol.8, No.3, pp. 345-383, 2000.
  3. [3] K. Ueda, “Souhatsu to Multi Agent System [Emergence and Multi Agent System],” Baifukan, 2007.
  4. [4] K. Fujii, “Data-Driven Analysis for Understanding Team Sports Behaviors,” J. Robot. Mechatron., Vol.33, No.3, pp. 505-514, 2021.
  5. [5] T. W. Sandholm and R. H. Crites, “On Multiagent Q-learning in a Semi-Competitive Domain,” Workshop Notes of Adaptation and Learning in Multiagent Systems Workshop, IJCAI-95, 1995.
  6. [6] S. Arai, K. Miyazaki, and S. Kobayashi, “Methodology in Multi-Agent Reinforcement Learning: Approaches by Q-Learning and Profit Sharing,” J. Jpn. Soc. Artif. Intell., Vol.13, No.4, pp. 609-618, 1998.
  7. [7] T. Yasuda and K. Ohkura, “Sharing Experience for Behavior Generation of Real Swarm Robot Systems Using Deep Reinforcement Learning,” J. Robot. Mechatron., Vol.31, No.4, pp. 520-525, 2019.
  8. [8] M. Asada, “RoboCup-97,” J. Robot. Mechatron., Vol.10, No.1, pp. 30-33, 1998.
  9. [9] K. Inoue, T. Arai, and J. Ota, “Behavior Acquisition in Partially Observable Environments by Autonomous Segmentation of the Observation Space,” J. Robot. Mechatron., Vol.27, No.3, pp. 293-304, 2015.
  10. [10] Y. Takahashi and M. Asada, “Behavior Acquisition by Multi-Layered Reinforcement Learning,” J. of the Robotics Society of Japan, Vol.18, No.7, pp. 1040-1046, 2000.
  11. [11] Z. Zhang, “Pass Strategy of Robocup Robot System Based on Deep Learning Network,” Proc. of the 2022 4th Int. Conf. on Robotics, Intelligent Control and Artificial Intelligence (RICAI’22), pp. 80-86, 2022.
  12. [12] Y. Shibuya, A. Hamm, and T. C. Pargman, “Mapping HCI research methods for studying social media interaction: A systematic literature review,” Computers in Human Behavior, Vol.129, Article No.107131, 2022.
  13. [13] D. Shi, L. Wang, Y. Zhang et al., “Review of human-robot coordination control for rehabilitation based on motor function evaluation,” Front. Mech. Eng., Vol.17, Article No.28, 2022.
  14. [14] T. B. Sheridan, “Human-Robot Interaction: Status and Challenges,” Human Factors, Vol.58, No.4, pp. 525-532. 2016.
  15. [15] M. Tominaga, Y. Takemura, and K. Ishii, “Behavior Analysis of Multi-Agent Game for Human-robot Cooperation,” 18th Int. Conf. on Control, Automation and Systems, Korea, pp. 1544-1545, 2018.
  16. [16] S. I. Gallant, “Perceptron-based learning algorithms,” IEEE Trans. Neural Netw., Vol.1, No.2, pp. 179-191, 1990.
  17. [17] J. A. Hartigan and M. A. Wong, “Algorithm AS 136: A K-Means Clustering Algorithm,” J. R. Stat. Soc. Ser. C. (Appl. Stat.), Vol.28, No.1, pp. 100-108, 1979.
  18. [18] L. Breiman, “Random forests,” Machine Learning, Vol.45, No.1, pp. 5-32, 2001.
  19. [19] V. Vapnik and A. Lerner, “Pattern recognition using generalized portrait method,” Autom. Remote Control, Vol.24, pp. 774-780, 1963.
  20. [20] K. Pearson, “LIII. On lines and planes of closest fit to systems of points in space,” Lond. Edinb. Dublin Philos. Mag. J. Sci., Vol.2, No.11, pp. 559-572, 1901.
  21. [21] T. Kohonen, “Self-Organizing Maps,” Springer Japan, 2001.
  22. [22] W. Zhang, K. Itoh, J. Tanida, and Y. Ichioka, “Parallel distributed processing model with local space-invariant interconnections and its optical architecture,” Applied Optics, Vol.29, No.32, pp. 4790-4797, 1990.
  23. [23] H. Masuta, Y. Tamura, and H. Lim, “Self-Organized Map Based Learning System for Estimating the Specific Task by Simple Instructions,” J. Adv. Comput. Intell. Intell. Inform., Vol.17, No.3, pp. 450-458, 2013.
  24. [24] T. Iwasaki and T. Furukawa, “Relational Data Visualization by Tensor SOM,” Proc. of 31st Fuzzy System Symposium, pp. 444-447, 2015.
  25. [25] E. Mäkinen, “A Survey on Binary Tree Codings,” Comput. J., Vol.34, No.5, pp. 438-443, 1991.
  26. [26] S. M. Piryonesi and T. E. El-Diraby, “Data analytics in asset management: Cost-effective prediction of the pavement condition index,” J. Infrastruct. Syst., Vol.26, No.1, Article No.04019036, 2020.
  27. [27] A. Ultsch and H. P. Siemon, “Kohonen’s Self Organizing Feature Maps for Exploratory Data Analysis,” Proc. of the Int. Neural Network Conf. (INNC’90), pp. 305-308, 1990.

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Nov. 24, 2023