Paper:
Language: English:

Effects of Initialization on Rule Extraction in Structural Learning

Hiroshi Shiratsuchi^, Hiromu Gotanda^,
Katsuhiro Inoue^, and Kousuke Kumamaru^

^*School of Humanity-Oriented Science and Engineering, Kinki University, 11-6 Kayanomori, Iizuka, Fukuoka 820-8555, Japan
^**Department of Systems Innovation and Informatics, Kyushu Institute of Technology, 680-4 Kawazu, Iizuka, Fukuoka 820-8502, Japan

Received: April 13, 2007

Accepted: October 11, 2007

Keywords: initialization, structural learning with forgetting, rule extraction, optimum skeletal structure

Journal ref: Journal of Advanced Computational Intelligence and Intelligent Informatics, Vol.12, No.1 pp. 57-62, 2008

Abstract

This paper studies how our previously proposed initialization effects the rule extraction of neural networks by structural learning with forgetting. The proposed initialization consists of two steps: (1) initializing weights of hidden units so that their separation hyperplanes should pass through the center of an input pattern set and (2) initializing those of output units to zero. From simulation results on Boolean function discovery problems with 5 and 7 inputs, it has been confirmed that the proposed initialization yields a simpler network structure and higher rule extraction ability than the conventional initialization giving uniform random number to all the initial weights of the network.

Reference

[1] F. Rosenblatt, “Principles of Neurodynamics,” Spartan, 1961.

[2] D. E. Rumelhart, J. L. McClelland, and the PDP Research Group, “Parallel Distributed Processing,” Vol.1&2, MIT Press, Cambridge, 1986.

[3] P. D. Wasserman, “Neural Computing: Theory and Practice,” Coriolis Group, 1989.

[4] M. Ishikawa, “Structural Learning with Forgetting,” Neural Networks, Vol.9, No.3, pp. 509-521, 1996.

[5] M. Ishikawa, “Rule extraction by successive regularization,” Neural Networks, Vol.13, pp. 1171-1183, 2000.

[6] H. Shiratsuchi, H. Gotanda, K. Inoue, and K. Kumamaru, “A study on initialization for bp networks,” Int. Conf. on Neural Information Processing, Vol.3, pp. 1595-1599, 1998.

[7] H. Shiratsuchi, H. Gotanda, K. Inoue, and K. Kumamaru, “Studies on Effects of Initialization on Structure Formation and Generalization of Structural Learning with Forgetting,” Journal of Advanced Computational Intelligence and Intelligent Informatics, Vol.8, No.6, pp. 621-626, 2004.

[8] H. Shiratsuchi, H. Gotanda, K. Inoue, and K. Kumamaru, “On Initializations of BP Networks with a Single Hidden Layer and Normalizations of Training Data,” Institute of Systems, Control and Information Engineers, Vol.15, No.10, pp. 532-538, 2002 (in Japanese).