Effects of Initialization on Rule Extraction in Structural Learning
Hiroshi Shiratsuchi*, Hiromu Gotanda*,
Katsuhiro Inoue**, and Kousuke Kumamaru**
*School of Humanity-Oriented Science and Engineering, Kinki University, 11-6 Kayanomori, Iizuka, Fukuoka 820-8555, Japan
**Department of Systems Innovation and Informatics, Kyushu Institute of Technology, 680-4 Kawazu, Iizuka, Fukuoka 820-8502, Japan
This paper studies how our previously proposed initialization effects the rule extraction of neural networks by structural learning with forgetting. The proposed initialization consists of two steps: (1) initializing weights of hidden units so that their separation hyperplanes should pass through the center of an input pattern set and (2) initializing those of output units to zero. From simulation results on Boolean function discovery problems with 5 and 7 inputs, it has been confirmed that the proposed initialization yields a simpler network structure and higher rule extraction ability than the conventional initialization giving uniform random number to all the initial weights of the network.
Katsuhiro Inoue, and Kousuke Kumamaru, “Effects of Initialization on Rule Extraction in Structural Learning,” J. Adv. Comput. Intell. Intell. Inform., Vol.12, No.1, pp. 57-62, 2008.
-  F. Rosenblatt, “Principles of Neurodynamics,” Spartan, 1961.
-  D. E. Rumelhart, J. L. McClelland, and the PDP Research Group, “Parallel Distributed Processing,” Vol.1&2, MIT Press, Cambridge, 1986.
-  P. D. Wasserman, “Neural Computing: Theory and Practice,” Coriolis Group, 1989.
-  M. Ishikawa, “Structural Learning with Forgetting,” Neural Networks, Vol.9, No.3, pp. 509-521, 1996.
-  M. Ishikawa, “Rule extraction by successive regularization,” Neural Networks, Vol.13, pp. 1171-1183, 2000.
-  H. Shiratsuchi, H. Gotanda, K. Inoue, and K. Kumamaru, “A study on initialization for bp networks,” Int. Conf. on Neural Information Processing, Vol.3, pp. 1595-1599, 1998.
-  H. Shiratsuchi, H. Gotanda, K. Inoue, and K. Kumamaru, “Studies on Effects of Initialization on Structure Formation and Generalization of Structural Learning with Forgetting,” Journal of Advanced Computational Intelligence and Intelligent Informatics, Vol.8, No.6, pp. 621-626, 2004.
-  H. Shiratsuchi, H. Gotanda, K. Inoue, and K. Kumamaru, “On Initializations of BP Networks with a Single Hidden Layer and Normalizations of Training Data,” Institute of Systems, Control and Information Engineers, Vol.15, No.10, pp. 532-538, 2002 (in Japanese).