Genetic Network Programming-Sarsa with Subroutines for Trading Rules on Stock Markets
Yang Yang, Shingo Mabu, Jianhua Li, and Kotaro Hirasawa
The Graduate School of Information, Production and Systems, Waseda University, 2-7 Hibikino, Wakamatsu-ku, Kitakyushu, Fukuoka 808-0135, Japan
The purpose of this paper is to propose a new approach to generate effective subroutines, which are automatically discovered by the GNP-Sarsa programs combining evolution and reinforcement learning. We name it asGNP-Sarsa with subroutines (GNPsb-Sarsa) and apply it to the trading rules on stock markets. In the proposal method, GNPsb-Sarsa offers an alternative population, where individuals are represented by subroutines. Each main program of GNPsb-Sarsa can refer to an individual in the subroutine population, after adding a new kind of node, namely subroutine node, in the graph network structure of GNP-Sarsa. GNPsb-Sarsa containing a main program and a subroutine evolves by natural selection and genetic operations, where the gene of GNPsb-Sarsa is the pair of the main GNP and its subroutine. That is, the genetic operations on GNPsb-Sarsa are constrained by the gene structure on which they can operate. In the simulations, the stock prices of different brands from 2001 to 2004 are used to test the effectiveness of the GNPsb-Sarsa. The results show that the proposed approach can provide reasonable opportunities for evolving complex solutions.
-  S. Mabu, K. Hirasawa, and T. Furuzuki, “Trading Rules on Stock Markets Using Genetic Network Programming with Reinforcement Learning and Importance Index,” Trans. of the Society of Instrument and Control Engineers, Vol.42, No.5, pp. 559-566, 2006.
-  S.Mabu, Y. Izumi, K. Hirasawa, and T. Furuzuki, “Trading Rules on Stock Markets Using Genetic Network Programming with Candle Chart,” Trans. of the Society of Instrument and Control Engineers, Vol.43, No.4, pp. 317-322, 2007.
-  Y. Chen, S. Mabu, K. Shimada, and K. Hirasawa, “Trading Rules on Stock Markets Using Genetic Network Programming with Sarsa Learning,” J. of Advanced Computational Intelligence and Intelligent Informatics, Vol.12, No.4, pp. 383-392, 2008.
-  T. Eguchi, K. Hirasawa, J. Hu, and N. Ota, “A Study of Evolutionary Multiagent Models Based on Symbiosis,” IEEE Trans. Syst. Man Cybern. B, Cybern., Vol.36, No.1, pp. 179-193, 2006.
-  S. Mabu, K. Hirasawa, and J. Hu, “A Graph-Based Evolutionary Algorithm: Genetic Network Programming (GNP) and Its Extension Using Reinforcement Learning,” Evolutionary Computation, Vol.15, No.3, pp. 369-398, 2007.
-  J. H. Holland, “Adaptation in Natural and Artificial Systems,” University of Michigan Press, 1975.
-  D. E. Goldberg, “Genetic Algorithms in Search, Optimization, and Machine Learning,” Addison-Wesley, 1989.
-  J. R. Koza, “Genetic Programming: On the Programming of Computers by Means of Natural Selection,” The MIT Press, 1992.
-  J. R. Koza, “Genetic Programming II: Automatic Discovery of Reusable Programs,” The MIT Press, 1994.
-  R. S. Sutton and A. G. Barto, “Reinforcement Learning: An Introduction,” The MIT Press, 1998.