Paper:
Language: English:
RBF Networks Ensemble Construction based on Evolutionary Multi-objective Optimization
Nobuhiko Kondo*, Toshiharu Hatanaka*, and Katsuji Uosaki**
*Department of Information and Physical Sciences, Osaka University, Osaka, Japan
**Department of Management and Information Sciences, Fukui University of Technology, Fukui, Japan
Received: May 25, 2007
Accepted: December 19, 2007
Keywords: RBF network, ensemble learning, evolutionary multi-objective optimization, pattern classification
Journal ref: Journal of Advanced Computational Intelligence and Intelligent Informatics, Vol.12, No.3 pp. 297-303, 2008
Abstract
The ensemble learning has attracted much attention over the last decade. While constructing RBF network ensemble the generally encountered problems are how to construct diverse RBF networks and how to combine outputs. The construction of RBF network can be considered as a multi-objective optimization problem regarding model complexity. A set of RBF networks which is multi-objectively optimized can be obtained by solving the above-mentioned problem. In this paper the construction of RBF networks by evolutionary multi-objective optimization method and its ensemble are considered, and it is applied to the pattern classification problem. Also some ensemble member selection methods and output combination methods are considered. Experimental study on the benchmark problem of pattern classification is carried out; then it is illustrated that the RBF network ensemble has a performance, which is comparable to that of other ensemble methods.
Preview (PDF)
Reference
[1] H. Aso, K. Tsuda, and N. Murata, “Statistics of Pattern Recognition and Learning,” Iwanami Shoten, 2003.
[2] A. J. C. Sharkey, “On Combining Artificial Neural Nets,” Connection Science, Vol.8, pp. 299-313, 1996.
[3] O. Nelles, “Nonlinear System Identification,” Springer, 2001.
[4] T. Hatanaka, K. Uosaki, and T. Kuroda, “Structure Selection of RBF Neural Network Using Information Criteria,” Proc. of Fifth Int. Conf. on Knowledge-Based Intelligent Information Engineering Systems and Allied Technologies, pp. 167-171, 2001.
[5] H. A. Abbass, “A Memetic Pareto Evolutionary Approach to Artificial Neural Networks,” Australian Joint Conf. on Artificial Intelligence 2001, pp. 1-12, 2001.
[6] G. G. Yen and H. Lu, “Hierarchical Rank Density Genetic Algorithm for Radial-Basis Function Neural Network Design,” Int. Journal of Computational Intelligence and Applications, Vol.3, No.3, pp. 213-232, 2003.
[7] N. Kondo, T. Hatanaka, and K. Uosaki, “Pattern Classification by Evolutionary RBF Networks Ensemble Based on Multi-objective Optimization,” Proc. of Int. Joint Conf. on Neural Networks ’06, pp. 2919-2925, 2006.
[8] N. Kondo, T. Hatanaka, and K. Uosaki, “Pattern Classification via Multi-objective Evolutionary RBF Networks Ensemble,” Proc. of SICE-ICASE Int. Joint Conf., pp. 137-142, 2006.
[9] T. G. Dietterich, “Ensemble Methods in Machine Learning,” Proc. of the First Int. Workshop on Multiple Classifier Systems 2000, pp. 1-15, 2000.
[10] I. Kumazawa, “Learning and Neural Networks,” Morikita Publishing, 1998.
[11] X. Yao, “Evolving Artificial Neural Networks,” Proc. of the IEEE, Vol.87, No.9, pp. 1423-1447, 1999.
[12] Y. Bai and L. Zhang, “Genetic Algorithm Based Self-Growing Training for RBF Neural Networks,” Proc. of Int. Joint Conf. on Neural Networks 2002, pp. 840-845, 2002.
[13] W. Zhao, D. S. Huang, and G. Yunjian, “The Structure Optimization of Radial Basis Probabilistic Neural Networks Based on Genetic Algorithms,” Proc. of Int. Joint Conf. on Neural Networks 2002, pp. 1086-1091, 2002.
[14] Y. Jin, T. Okabe, and B. Sendhoff, “Neural Network Regularization and Ensembling Using Multi-objective Evolutionary Algorithms,” Proc. of Congress on Evolutionary Computation, pp. 1-8, 2004.
[15] N. Kondo, T. Hatanaka, and K. Uosaki, “Pareto RBF Networks Based on Multiobjective Evolutionary Computation,” SICE Annual Conf. in Sapporo, pp. 2177-2182, 2004.
[16] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, “A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II,” IEEE Transactions on Evolutionary Computation, Vol.6, No.2, pp. 182-197, 2002.
[17] J. Moody and C. J. Darken, “Fast learning in networks of locallytuned processing units,” Neural Computation, Vol.1, pp. 281-294, 1989.
[18] L. I. Kuncheva and C. J. Whitaker, “Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy,” Machine Learning, Vol.51, pp. 181-207, 2003.
[19] http://www.ics.uci.edu/˜mlearn/
[20] H. A. Abbass, “Pareto Neuro-Evolution: Constructing Ensemble of Neural Networks Using Multi-objective Optimization,” The IEEE Congress on Evolutionary Computation 2003, Vol.3, pp. 2074-2080, 2003.
[21] Y. Liu, X. Yao, and T. Higuchi, “Evolutionary Ensembles with Negative Correlation Learning,” IEEE Transactions on Evolutionary Computation, Vol.4, No.4, pp. 380-387, 2000.
[Notice]
* "Preview" is the first 2 pages of the article. You don't need the registration.
* To read the PDF file you will then need to download and install the Adobe Reader.
Adobe Reader is free and available for download here:
