Fujipress Home | Search | About FINDER

Paper:
Language: English:

Fuzzy c-Means Algorithms Using Kullback-Leibler Divergence and Helliger Distance Based on Multinomial Manifold


Ryo Inokuchi* and Sadaaki Miyamoto**


*Doctoral Program in Risk Engineering, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki 305-8573, Japan
**Department of Risk Engineering, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki 305-8573, Japan


Received: October 10, 2007

Accepted: February 15, 2008


Keywords: fuzzy clustering, information geometry, Kullback-Leibler divergence

Journal ref: Journal of Advanced Computational Intelligence and Intelligent Informatics, Vol.12, No.5 pp. 443-447, 2008

Abstract



In this paper, we discuss fuzzy clustering algorithms for discrete data. Data space is represented as a statistical manifold of the multinomial distribution, and then the Euclidean distance are not adequate in this setting. The geodesic distance on the multinomial manifold can be derived analytically, but it is difficult to use it as a metric directly. We propose fuzzy c-means algorithms using other metrics: the Kullback-Leibler divergence and the Hellinger distance, instead of the Euclidean distance. These two metrics are regarded as approximations of the geodesic distance.
preview Preview (PDF)  full text Full Text (PDF 130KB)

Reference


[1] S. Amari and H. Nagaoka, “Methods of Information Geometry,” 10th European Conference on Machine Learning, Cambridge University Press, Vol.70, pp. 1-25, 2000.
[2] J. C. Bezdek, “Pattern Recognition with Fuzzy Objective Function Algorithms,” Kluwer Academic Publishers Norwell, MA, USA, 1981.
[3] M. Collins, S. Dasgupta, and R. Schapire, “A generalization of principal component analysis to the exponential family,” Advances in Neural Information Processing Systems, MIT Press, Vol.14, 2002.
[4] T. Jebara, R. Kondor, and A. Howard, “Probability product kernels,” The Journal of Machine Learning Research, MIT Press Cambridge, MA, USA, Vol.5, pp. 819-844, 2004.
[5] R. E. Kass and P. W. Vos, “Geometrical Foundations of Asymptotic Inference,” Wiley Series in Probability and Statistics, 1997.
[6] J. Lafferty and G. Lebanon, “Diffusion kernels on statistical manifolds,” The Journal of Machine Learning Research, MIT Press Cambridge, MA, USA, Vol.6, pp. 129-163, 2005.
[7] J. MacQueen, “Some methods for classification and analysis of multivariate observations,” Proc. of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Vol.1, pp. 281-297, 1967.
[8] C. D. Manning and H. Sch{ü}tze, “Foundations of statistical natural language processing,” MIT Press, 1999.
[9] S. Miyamoto and M. Mukaidono, “Fuzzy c-means as a regularization and maximum entropy approach,” Proc. of the 7th International Fuzzy Systems Association World Congress (IFSA'97), Vol.2, pp. 86-92, 1997.

[Notice]
* "Preview" is the first 2 pages of the article. You don't need the registration.
* To read the PDF file you will then need to download and install the Adobe Reader.
Adobe Reader is free and available for download here:

adobe reader

Terms and Conditions | Privacy Policy | Recruit | Advertising Information | Contact Us