Information Theoretic Approach for Measuring Interaction in Multiagent Domain
Sachiyo Arai and Yoshihisa Ishigaki
Graduate School of Engineering, Chiba University, 1-33 Yayoi-cho, Inage-ku, Chiba
Although a large number of reinforcement learning algorithms have been proposed for the generation of cooperative behaviors, the question of how to evaluate mutual benefit or loss among them is still open. As far as we know, an emerged behavior is regarded as a cooperative behavior when embedded agents have finally achieved their global goal, regardless of whether or not mutual interference has had any effect during the course of the learning process of each agent. Thus, we cannot detect any harmful interaction on the way to achieving a fully-converged policy. In this paper, we propose a measure based on information theory for evaluating the degree of interaction during the learning process from the viewpoint of information sharing. In order to discuss the bad effects of concurrent learning, we apply our proposed measure to a situation in which there exist conflicts among the agents, and we show the availability of our measure.
-  A. Namatame, “Emergent Collective Behaviors and Evolution in a Group of Rational Agents,” Proc. of the Australia-Japan Joint Workshop on Intelligent and Evolutionary Systems, pp. 131-140, 1997.
-  L. Gasser, N. Rouquette, R.W. Hil, and J. Lieb, “Representing and Using Organizational Knowledge in Distributed Artificial Systems,” L. Gasser and M.H. Huhns (Eds.), Distributed Artificial Intelligence, Vol.2, pp. 55-78, Morgan Kaufmann, 1989.
-  S.S. Richard and G.B. Andrew, “Reinforcement Learning --An Introduction--, The MIT Press, 1998.
-  S. Arai, K. Miyazaki, and S. Kobayashi, “Methodology in Multiagent Reinforcement Learning -Approaches by Q-learning and Profit Sharing,” J. of Japanese Society for Artificial Intelligence, Vol.13, No.4, pp. 105-114, 1998 (in Japanese).
-  K. Iwata, K. Ikeda, and H. Sakai, “A New Criterion Using Information Gain fot Action Selection Strategy in Reinforcement Learning,” IEEE Trans. Neural Networks, Vol.15, No.4, pp. 792-799, 2004.
-  H. Van Dyke Parunak, Sven Brueckner, “Entropy and Self-Organaization in Multi-Agent Systems,” Proc. the fifth Int. Conf. on Autonomous Agents, pp. 124-130, 2001.
-  N. Ynagisawa, H. Kawamura, M. Yamamoto, and A. Ouchi, “Quantification of Interactive Behavior in Multiagent Systems,” IEICE technical report, Artificial Intelligence and knowledge-based Processing, pp. 71-76, 2004 (in Japanese).
-  C.J.H. Watkins and P. Dayan, “Technical note: Q-learning,” Machine Learning, Vol.8, pp. 55-68, 1992.