Paper:

# Parallel Learning Model and Topological Measurement for Self-Organizing Maps

## Michiharu Maeda^{*}, Hiromi Miyajima^{**}, and Noritaka Shigei^{**}

^{*}Department of Control and Information Systems Engineering, Kurume National College of Technology, 1-1-1 Komorino, Kurume 830-8555, Japan

^{**}Department of Electrical and Electronic Engineering, Faculty of Engineering, Kagoshima University, 1-21-40 Korimoto, Kagoshima 890-0065, Japan

The parallel learning model we propose for self-organizing maps (SOMs) uses reference vectors for simultaneously given multiple input and introduces a topological measurement of ordering for reference vectors in a multidimensional array. We term the parallel SOM algorithm parallel learning and the criterion of ordering the twist index. Parallel learning simultaneously updates reference vectors corresponding to individual input when multiple input is prepared. The twist index is the criterion for evaluating multidimensional ordering of the topological array for reference vectors. When the parallel degree changes for a SOM, the topology preserving map (TPM) for post-learning is evaluated using the twist index. Although adaptation generally yields different results for single and multiple input given at each step in neural networks, parallel learning in SOMs produces results almost the same as sequential learning. Discussing the formation rate of TPMs and average distortion, we examine the effectiveness of our approach through numerical experiments.

*J. Adv. Comput. Intell. Intell. Inform.*, Vol.11, No.3, pp. 327-334, 2007.

- [1] T. Geszti, “Physical models of neural networks,” World Scientific, 1990.
- [2] J. Hertz, A. Krogh, and R. G. Palmer, “Introduction to the theory of neural computation,” Addison-Wesley, 1991.
- [3] T. Kohonen, “Self-organization and associative memory,” Springer-Verlag Berlin, 1989.
- [4] T. Kohonen, “The self-organizing map,” Proc. IEEE, Vol.78, pp. 1464-1480, 1990.
- [5] T. Kohonen, “Self-organizing maps,” Springer-Verlag Berlin, 1995.
- [6] H.-U. Bauer and K. R. Pawelzik, “Quantifying the neighborhood preservation of self-organizing feature maps,” IEEE Trans. Neural Networks, Vol.3, pp. 570-579, 1992.
- [7] T. Martinetz and K. Schulten, “Topology representing networks,” Neural Networks, Vol.7, pp. 507-522, 1994.
- [8] M. Maeda, H. Miyajima, and S. Murashima, “Parallel processing for self-organization,” IEICE Proc. 7-th Circuits & Syst. Karuizawa Workshop, pp. 139-143, 1994.
- [9] M. Maeda, H. Miyajima, and S. Murashima, “Parallel-sequential self-organizing algorithms for feature maps,” IEICE Proc. Symp. Nonlinear Theory and its Applications, pp. 275-278, 1994.
- [10] T. Villmann, M. Herrmann, and T. M. Martinetz, “Topology preservation in self-organizing feature maps: Exact definition and measurement,” IEEE Trans. Neural Networks, Vol.8, pp. 256-266, 1997.
- [11] M. Maeda and H. Miyajima, “Parallel manner for self-organizing maps and its topology preservation,” IEICE Technical Report, Vol.NLP99-129, pp. 41-48, 2000.
- [12] M. Maeda, H. Miyajima, and N. Shigei, “A model of parallel learning in self-organizing maps and topological measurement,” Proc. Int. Conf. Intelligent Technologies, pp. 133-140, 2005.
- [13] T. Kohonen, “Analysis of a simple self-organizing process,” Biol. Cybern., Vol.44, pp. 135-140, 1982.
- [14] R. Durbin and D. Willshaw, “An analogue approach to the traveling salesman problem using an elastic net method,” Nature, Vol.326, pp. 689-691, 1987.
- [15] B. Angéniol, G. de La C. Vaubois, and J.-Y. Le Texier, “Selforganizing feature maps and the traveling salesman problem,” Neural Networks, Vol.1, pp. 289-293, 1988.
- [16] H. Ritter and K. Schulten, “On the stationary state of Kohonen’s self-organizing sensory mapping,” Biol. Cybern., Vol.54, pp. 99-106, 1986.
- [17] H. Ritter and K. Schulten, “Convergence properties of Kohonen’s topology conserving maps, Fluctuations, stability, and dimension selection,” Biol. Cybern., Vol.60, pp. 59-71, 1988.
- [18] T. M. Martinetz, S. G. Berkovich, and K. J. Schulten, “ “Neuralgas” network for vector quantization and its application to timeseries prediction,” IEEE Trans. Neural Networks, Vol.4, pp. 558-569, 1993.
- [19] M. Maeda and H. Miyajima, “Competitive learning methods with refractory and creative approaches,” IEICE Trans. Fundamental, Vol.E82-A, pp. 1825-1833, 1999.
- [20] M. Maeda and H. Miyajima, “Adaptation strength according to neighborhood ranking of self-organizing neural networks,” IEICE Trans. Fundamentals, Vol.E85-A, pp. 2078-2082, 2002.
- [21] N. Shigei, H. Miyajima, and M. Maeda, “Numerical evaluation of incremental vector quantization using stochastic relaxation,” IEICE Trans. Fundamentals, Vol.E87-A, pp. 2364-2371, 2004.
- [22] M. Maeda, N. Shigei, and H. Miyajima, “Adaptive vector quantization with creation and reduction grounded in the equinumber principle,” Journal of Advanced Computational Intelligence and Intelligent Informatics, Vol.9, No.6, pp. 599-606, 2005.
- [23] E. Goles, F. Fogelman, and D. Pellegrin, “Decreasing energy functions as a tool for studying threshold networks,” Discrete Appl. Math., Vol.12, pp. 261-277, 1985.
- [24] H. Miyajima, S. Yatsuki, and M. Maeda, “Some characteristics of higher order neural networks with decreasing energy functions,” IEICE Trans. Fundamentals, Vol.E79-A, pp. 1624-1629, 1996.
- [25] C.-H. Wu, R. E. Hodges, and C.-J. Wang, “Parallelizing the selforganizing feature map on multiprocessor systems,” Parallel Comput., Vol.17, pp. 821-832, 1991.
- [26] K. K. Parhi, F. H. Wu, and K. Genesan, “Sequential and parallel neural network vector quantizers,” IEEE Trans. Comput., Vol.43, pp. 104-109, 1994.
- [27] K. Sano, S. Momose, H. Takizawa, H. Kobayashi, and T. Nakamura, “Efficient parallel processing of competitive learning algorithm,” Parallel Comput., Vol.30, pp. 1361-1383, 2004.
- [28] T. M. Madhyastha and D. A. Reed, “Learning to classify parallel input/output access patterns,” IEEE Trans. Parallel and Distributed Systems, Vol.13, No.8, pp. 802-813, 2002.
- [29] J. Liu, M. A. Brooke, and K. Hirose, “A CMOS feedforward neuralnetwork chip with on-chip parallel learning for oscillation cancellation,” IEEE Trans. Neural Networks, Vol.13, No.5, pp. 1178-1186, 2002.
- [30] T. Kohonen, “Things you haven’t heard about the self-organizing map,” IEEE Proc. Int. Conf. Neural Networks, pp. 1147-1156, 1993.
- [31] Y. Linde, A. Buzo, and R. M. Gray, “An algorithm for vector quantizer design,” IEEE Trans. Commun., Vol.28, pp. 84-95, 1980.
- [32] H. Ritter, T. Martinetz, and K. Schulten, “Neural computation and self-organizing maps, an introduction,” Addison-Wesley, 1992.