Paper:

# Trading Rules on Stock Markets Using Genetic Network Programming with Sarsa Learning

## Yan Chen, Shingo Mabu, Kaoru Shimada, and Kotaro Hirasawa

Graduate school of Information, Production and Systems, Waseda University, 2-7 Hibikino, Wakamatsu-ku, Kitakyushu, Fukuoka

In this paper, the Genetic Network Programming (GNP) for creating trading rules on stocks is described. GNP is an evolutionary computation, which represents its solutions using graph structures and has some useful features inherently. It has been clarified that GNP works well especially in dynamic environments since GNP can create quite compact programs and has an implicit memory function. In this paper, GNP is applied to creating a stock trading model. There are three important points: The first important point is to combine GNP with Sarsa Learning which is one of the reinforcement learning algorithms. Evolution-based methods evolve their programs after task execution because they must calculate fitness values, while reinforcement learning can change programs during task execution, therefore the programs can be created efficiently. The second important point is that GNP uses candlestick chart and selects appropriate technical indices to judge the timing of the buying and selling stocks. The third important point is that sub-nodes are used in each node to determine appropriate actions (buying/selling) and to select appropriate stock price information depending on the situation. In the simulations, the trading model is trained using the stock prices of 16 brands in 2001, 2002 and 2003. Then the generalization ability is tested using the stock prices in 2004. From the simulation results, it is clarified that the trading rules of the proposed method obtain much higher profits than Buy&Hold method and its effectiveness has been confirmed.

*J. Adv. Comput. Intell. Intell. Inform.*, Vol.12, No.4, pp. 383-392, 2008.

- [1] S. Mabu, K. Hirasawa, and J. Hu, “A graph-based evolutionary algorithm: Genetic network programming and its extension using reinforcement learning,” Evolutionary Computation, MIT Press, Vol.15, No.3, pp. 369-398, 2007.
- [2] T. Eguchi, K. Hirasawa, J. Hu, and N. Ota, “Study of evolutionary multiagent models based on symbiosis,” IEEE Trans. Syst., Man and Cybern. B, Vol.36, No.1, pp. 179-193, 2006.
- [3] J. H. Holland, “Adaptation in Natural and Artificial Systems,” Ann Arbor, University of Michigan Press, 1975.
- [4] D. E. Goldberg, “Genetic Algorithm in search, optimization and machine learning,” Addison-Wesley, 1989.
- [5] J. R. Koza, “Genetic Programming, on the programming of computers by means of natural selection,” Cambridge, Mass., MIT Press, 1992.
- [6] J. R. Koza, “Genetic Programming II, Automatic Discovery of Reusable Programs,” Cambridge, Mass., MIT Press, 1994.
- [7] R. S. Sutton and A. G. Barto, “Reinforcement Learning -An Introduction,” Cambridge, Massachusetts, London, England, MIT Press, 1998.
- [8] N. Baba, N. Inoue, and Y. Yanjun, “ Utilization of soft computing techniques for constructing reliable decision support systems for dealing stocks,” in Proc. of Int. Joint Conf. on Neural Networks, 2002.
- [9] J.-Y. Potvin, P. Soriano, and M. Vallee, “Generating trading rules on the stock markets with genetic programming,” Computers & Operations Research, Vol.31, pp. 1033-1047, 2004.
- [10] K. J. Oh, T. Y. Kim, S.-H. Min, and H. Y. Lee, “Portfolio algorithm based on portfolio beta using genetic algorithm,” Expert Systems with Application, Vol.30, pp. 527-534, 2006.
- [11] S. Mabu, H. Hatakeyama, M. T. Thu, K. Hirasawa, and J. Hu, “Genetic Network Programming with Reinforcement Learning and Its Application to Making Mobile Robot Behavior,” IEEJ Trans. EIS, Vol.126, No.8, pp. 1009-1015, 2006.
- [12] K. H. Lee and G. S. Jo, “Expert system for predicting stock market timing using a candlestick chart,” Expert Systems with Applications, Vol.16, pp. 357-364, 1999.
- [13] Y. Izumi, T. Yamaguchi, S. Mabu, K. Hirasawa, and J. Hu, “Trading Rules on the Stock Market using Genetic Network Programming with Candlestick Chart,” 2006 IEEE Congress on Evolutionary Computation, Sheraton Vancouver Wall Centre Hotel, Vancouver, BC, Canada, pp. 8531-8536, July 16-21, 2006.
- [14] S. Mabu, Y. Izumi, K. Hirasawa, and T. Furuzuki, “Trading Rules on Stock Markets Using Genetic Network Progamming with Candle Chart,” T. SICE, Vol.43, No.4, pp. 317-322, 2007 (in Japanese).
- [15] Y. Izumi, K. Hirasawa, and T. Furuzuki, “Trading Rules on the Stock Markets Using Genetic Network Progamming with Importance Index,” T. SICE, Vol.42, No.5, pp. 559-566, 2006 (in Japanese).
- [16] V. Dhar, “A Comparison of GLOWER and Other Machine Learning Methods for Investment Decision Making,” Springer Berlin Press, pp.208-220, 2001.
- [17] S. Duerson, F. S. Khan, V. Kovalev, and A. H. Malik, “Reinforcement Learning in Online Stock Trading Systems,” 2005.

http://www.cc.gatech.edu/grads/h/hisham/projects/ml7641/RLStockTrading.pdf - [18] S. Pafka, M. Potters, and I. Kondor, “Exponential Weighting and Random-Matrix-Theory-Based Filtering of Financial Covariance Matrices for Portfolio Optimization,” arXiv:cond-mat/0402573v1, 2004. Quantitative Finance, (to be appeared).
- [19] N. Basalto, R. Bellotti, F. De Carlo, P. Facchi, and S. Pascazio, “Clustering stock market companies via chaotic map synchronization,” Physica A, 345, p. 196, arXiv:cond-mat/0404497v1, 2005.
- [20] W. Huang, Y. Nakamori, and S. Y. Wang, “Forecasting stock market movement direction with support vector machine Source,” Computers and Operations Research, Vol.32, Issue 10, pp. 2513-2522, 2005.
- [21] M. B. Porecha, P. K. Panigrahi, J. C. Parikh, C. M. Kishtawal, and S. Basu, “Forecasting non-stationary financial time series through genetic algorithm,” arXiv:nlin/0507037v1, 2005.
- [22] M. H. Jensen, A. Johansen, F. Petroni, and I. Simonsen, “Inverse Statistics in the Foreign Exchange Market,” Physica A, 340, p. 678, arXiv:cond-mat/0402591v2, 2004.
- [23] T. Mikosch and C. Starica, “Stock Market Risk-Return Inference. An Unconditional Non-parametric Approach,” SSRN Working Paper Series, 2004.
- [24] H. Iba and T. Sasaki, “Using Genetic Programming to Predict Financial Data,” Proc. of the Congress of Evolutionary Computation, pp. 244-251, 2001.