JACIII Vol.12 No.1 pp. 57-62
doi: 10.20965/jaciii.2008.p0057


Effects of Initialization on Rule Extraction in Structural Learning

Hiroshi Shiratsuchi*, Hiromu Gotanda*,
Katsuhiro Inoue**, and Kousuke Kumamaru**

*School of Humanity-Oriented Science and Engineering, Kinki University, 11-6 Kayanomori, Iizuka, Fukuoka 820-8555, Japan

**Department of Systems Innovation and Informatics, Kyushu Institute of Technology, 680-4 Kawazu, Iizuka, Fukuoka 820-8502, Japan

April 13, 2007
October 11, 2007
January 20, 2008
initialization, structural learning with forgetting, rule extraction, optimum skeletal structure

This paper studies how our previously proposed initialization effects the rule extraction of neural networks by structural learning with forgetting. The proposed initialization consists of two steps: (1) initializing weights of hidden units so that their separation hyperplanes should pass through the center of an input pattern set and (2) initializing those of output units to zero. From simulation results on Boolean function discovery problems with 5 and 7 inputs, it has been confirmed that the proposed initialization yields a simpler network structure and higher rule extraction ability than the conventional initialization giving uniform random number to all the initial weights of the network.

Cite this article as:
Hiroshi Shiratsuchi, Hiromu Gotanda,
Katsuhiro Inoue, and Kousuke Kumamaru, “Effects of Initialization on Rule Extraction in Structural Learning,” J. Adv. Comput. Intell. Intell. Inform., Vol.12, No.1, pp. 57-62, 2008.
Data files:
  1. [1] F. Rosenblatt, “Principles of Neurodynamics,” Spartan, 1961.
  2. [2] D. E. Rumelhart, J. L. McClelland, and the PDP Research Group, “Parallel Distributed Processing,” Vol.1&2, MIT Press, Cambridge, 1986.
  3. [3] P. D. Wasserman, “Neural Computing: Theory and Practice,” Coriolis Group, 1989.
  4. [4] M. Ishikawa, “Structural Learning with Forgetting,” Neural Networks, Vol.9, No.3, pp. 509-521, 1996.
  5. [5] M. Ishikawa, “Rule extraction by successive regularization,” Neural Networks, Vol.13, pp. 1171-1183, 2000.
  6. [6] H. Shiratsuchi, H. Gotanda, K. Inoue, and K. Kumamaru, “A study on initialization for bp networks,” Int. Conf. on Neural Information Processing, Vol.3, pp. 1595-1599, 1998.
  7. [7] H. Shiratsuchi, H. Gotanda, K. Inoue, and K. Kumamaru, “Studies on Effects of Initialization on Structure Formation and Generalization of Structural Learning with Forgetting,” Journal of Advanced Computational Intelligence and Intelligent Informatics, Vol.8, No.6, pp. 621-626, 2004.
  8. [8] H. Shiratsuchi, H. Gotanda, K. Inoue, and K. Kumamaru, “On Initializations of BP Networks with a Single Hidden Layer and Normalizations of Training Data,” Institute of Systems, Control and Information Engineers, Vol.15, No.10, pp. 532-538, 2002 (in Japanese).

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Mar. 05, 2021