With technological developments, the prospect of a human-robot symbiotic society has emerged. A soccer game has characteristics similar to those expected in such a society. Soccer is a multiagent game in which the strategy employed depends on each agent’s position and actions. This paper discusses the results of the development of a learning system that uses a self-organizing map to select behaviors depending on the scenario (two-dimensional absolute coordinates of the agent, other agents, and the ball). The system can reproduce the action-selection algorithms of all the players on a certain team, and the robot can instantly select the next cooperative action from information obtained during the game. Thus, common-sense rules can be shared to learn an action-selection algorithm for a set of both human and robot agents.

1. [1] M. Tominaga, Y. Takemura, and K. Ishii, “Analysis of Cooperative Behavior in Futsal for Human and Robot Multi Agent System,” Proc. of JSME Annual Conf. on Robotics and Mechatronics (Robomec), 2P2-L09, 2018. https://doi.org/10.1299/jsmermd.2018.2P2-L09
2. [2] P. Stone and M. Veloso, “Multiagent Systems: A Survey from a Machine Learning Perspective,” Auton. Robots, Vol.8, No.3, pp. 345-383, 2000. https://doi.org/10.1023/A:1008942012299
3. [3] K. Ueda, “Souhatsu to Multi Agent System [Emergence and Multi Agent System],” Baifukan, 2007.
4. [4] K. Fujii, “Data-Driven Analysis for Understanding Team Sports Behaviors,” J. Robot. Mechatron., Vol.33, No.3, pp. 505-514, 2021. https://doi.org/10.20965/jrm.2021.p0505
5. [5] T. W. Sandholm and R. H. Crites, “On Multiagent Q-learning in a Semi-Competitive Domain,” Workshop Notes of Adaptation and Learning in Multiagent Systems Workshop, IJCAI-95, 1995. https://doi.org/10.1007/3-540-60923-7_28
6. [6] S. Arai, K. Miyazaki, and S. Kobayashi, “Methodology in Multi-Agent Reinforcement Learning: Approaches by Q-Learning and Profit Sharing,” J. Jpn. Soc. Artif. Intell., Vol.13, No.4, pp. 609-618, 1998. https://doi.org/10.11517/jjsai.13.4_609
7. [7] T. Yasuda and K. Ohkura, “Sharing Experience for Behavior Generation of Real Swarm Robot Systems Using Deep Reinforcement Learning,” J. Robot. Mechatron., Vol.31, No.4, pp. 520-525, 2019. https://doi.org/10.20965/jrm.2019.p0520
8. [8] M. Asada, “RoboCup-97,” J. Robot. Mechatron., Vol.10, No.1, pp. 30-33, 1998. https://doi.org/10.20965/jrm.1998.p0030
9. [9] K. Inoue, T. Arai, and J. Ota, “Behavior Acquisition in Partially Observable Environments by Autonomous Segmentation of the Observation Space,” J. Robot. Mechatron., Vol.27, No.3, pp. 293-304, 2015. https://doi.org/10.20965/jrm.2015.p0293
10. [10] Y. Takahashi and M. Asada, “Behavior Acquisition by Multi-Layered Reinforcement Learning,” J. of the Robotics Society of Japan, Vol.18, No.7, pp. 1040-1046, 2000. https://doi.org/10.7210/jrsj.18.1040
11. [11] Z. Zhang, “Pass Strategy of Robocup Robot System Based on Deep Learning Network,” Proc. of the 2022 4th Int. Conf. on Robotics, Intelligent Control and Artificial Intelligence (RICAI’22), pp. 80-86, 2022. https://doi.org/10.1145/3584376.3584392
12. [12] Y. Shibuya, A. Hamm, and T. C. Pargman, “Mapping HCI research methods for studying social media interaction: A systematic literature review,” Computers in Human Behavior, Vol.129, Article No.107131, 2022. https://doi.org/10.1016/j.chb.2021.107131
13. [13] D. Shi, L. Wang, Y. Zhang et al., “Review of human-robot coordination control for rehabilitation based on motor function evaluation,” Front. Mech. Eng., Vol.17, Article No.28, 2022. https://doi.org/10.1007/s11465-022-0684-4
14. [14] T. B. Sheridan, “Human-Robot Interaction: Status and Challenges,” Human Factors, Vol.58, No.4, pp. 525-532. 2016. https://doi.org/10.1177/0018720816644364
15. [15] M. Tominaga, Y. Takemura, and K. Ishii, “Behavior Analysis of Multi-Agent Game for Human-robot Cooperation,” 18th Int. Conf. on Control, Automation and Systems, Korea, pp. 1544-1545, 2018.
16. [16] S. I. Gallant, “Perceptron-based learning algorithms,” IEEE Trans. Neural Netw., Vol.1, No.2, pp. 179-191, 1990. https://doi.org/10.1109/72.80230
17. [17] J. A. Hartigan and M. A. Wong, “Algorithm AS 136: A K-Means Clustering Algorithm,” J. R. Stat. Soc. Ser. C. (Appl. Stat.), Vol.28, No.1, pp. 100-108, 1979. https://doi.org/10.2307/2346830
18. [18] L. Breiman, “Random forests,” Machine Learning, Vol.45, No.1, pp. 5-32, 2001. https://doi.org/10.1023/A:1010933404324
19. [19] V. Vapnik and A. Lerner, “Pattern recognition using generalized portrait method,” Autom. Remote Control, Vol.24, pp. 774-780, 1963. https://doi.org/10.12691/jgg-2-3-9
20. [20] K. Pearson, “LIII. On lines and planes of closest fit to systems of points in space,” Lond. Edinb. Dublin Philos. Mag. J. Sci., Vol.2, No.11, pp. 559-572, 1901. https://doi.org/10.1080/14786440109462720
21. [21] T. Kohonen, “Self-Organizing Maps,” Springer Japan, 2001. https://doi.org/10.1007/978-3-642-56927-2
22. [22] W. Zhang, K. Itoh, J. Tanida, and Y. Ichioka, “Parallel distributed processing model with local space-invariant interconnections and its optical architecture,” Applied Optics, Vol.29, No.32, pp. 4790-4797, 1990. https://doi.org/10.1364/AO.29.004790
23. [23] H. Masuta, Y. Tamura, and H. Lim, “Self-Organized Map Based Learning System for Estimating the Specific Task by Simple Instructions,” J. Adv. Comput. Intell. Intell. Inform., Vol.17, No.3, pp. 450-458, 2013. https://doi.org/10.20965/jaciii.2013.p0450
24. [24] T. Iwasaki and T. Furukawa, “Relational Data Visualization by Tensor SOM,” Proc. of 31st Fuzzy System Symposium, pp. 444-447, 2015. https://doi.org/10.14864/fss.31.0_444
25. [25] E. Mäkinen, “A Survey on Binary Tree Codings,” Comput. J., Vol.34, No.5, pp. 438-443, 1991. https://doi.org/10.1093/comjnl/34.5.438
26. [26] S. M. Piryonesi and T. E. El-Diraby, “Data analytics in asset management: Cost-effective prediction of the pavement condition index,” J. Infrastruct. Syst., Vol.26, No.1, Article No.04019036, 2020. https://doi.org/10.1061/(ASCE)IS.1943-555X.0000512
27. [27] A. Ultsch and H. P. Siemon, “Kohonen’s Self Organizing Feature Maps for Exploratory Data Analysis,” Proc. of the Int. Neural Network Conf. (INNC’90), pp. 305-308, 1990.