Fuzzy <i>c</i>-Means Algorithms Using Kullback-Leibler Divergence and  Helliger Distance Based on Multinomial Manifold

Ryo Inokuchi; Sadaaki Miyamoto

doi:10.20965/jaciii.2008.p0443

single-jc.php

« previous

JACIII Vol.12 No.5 pp. 443-447

doi: 10.20965/jaciii.2008.p0443

(2008)

Paper:

Views over last 60 days: 351

Fuzzy c-Means Algorithms Using Kullback-Leibler Divergence and Helliger Distance Based on Multinomial Manifold

Ryo Inokuchi^* and Sadaaki Miyamoto^**

^*Doctoral Program in Risk Engineering, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki 305-8573, Japan

^**Department of Risk Engineering, University of Tsukuba, 1-1-1 Tennodai, Tsukuba, Ibaraki 305-8573, Japan

Received:

October 10, 2007

Accepted:

February 15, 2008

Published:

September 20, 2008

Keywords:

fuzzy clustering, information geometry, Kullback-Leibler divergence

Abstract

In this paper, we discuss fuzzy clustering algorithms for discrete data. Data space is represented as a statistical manifold of the multinomial distribution, and then the Euclidean distance are not adequate in this setting. The geodesic distance on the multinomial manifold can be derived analytically, but it is difficult to use it as a metric directly. We propose fuzzy c-means algorithms using other metrics: the Kullback-Leibler divergence and the Hellinger distance, instead of the Euclidean distance. These two metrics are regarded as approximations of the geodesic distance.

Cite this article as:

R. Inokuchi and S. Miyamoto, “Fuzzy c-Means Algorithms Using Kullback-Leibler Divergence and Helliger Distance Based on Multinomial Manifold,” J. Adv. Comput. Intell. Intell. Inform., Vol.12 No.5, pp. 443-447, 2008.

Data files:

References

[1]
S. Amari and H. Nagaoka, “Methods of Information Geometry,” 10th European Conference on Machine Learning, Cambridge University Press, Vol.70, pp. 1-25, 2000.
J. C. Bezdek, “Pattern Recognition with Fuzzy Objective Function Algorithms,” Kluwer Academic Publishers Norwell, MA, USA, 1981.
M. Collins, S. Dasgupta, and R. Schapire, “A generalization of principal component analysis to the exponential family,” Advances in Neural Information Processing Systems, MIT Press, Vol.14, 2002.
T. Jebara, R. Kondor, and A. Howard, “Probability product kernels,” The Journal of Machine Learning Research, MIT Press Cambridge, MA, USA, Vol.5, pp. 819-844, 2004.
R. E. Kass and P. W. Vos, “Geometrical Foundations of Asymptotic Inference,” Wiley Series in Probability and Statistics, 1997.
J. Lafferty and G. Lebanon, “Diffusion kernels on statistical manifolds,” The Journal of Machine Learning Research, MIT Press Cambridge, MA, USA, Vol.6, pp. 129-163, 2005.
J. MacQueen, “Some methods for classification and analysis of multivariate observations,” Proc. of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Vol.1, pp. 281-297, 1967.
C. D. Manning and H. Schütze, “Foundations of statistical natural language processing,” MIT Press, 1999.
S. Miyamoto and M. Mukaidono, “Fuzzy c-means as a regularization and maximum entropy approach,” Proc. of the 7th International Fuzzy Systems Association World Congress (IFSA'97), Vol.2, pp. 86-92, 1997.

This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

[1] [1]
S. Amari and H. Nagaoka, “Methods of Information Geometry,” 10th European Conference on Machine Learning, Cambridge University Press, Vol.70, pp. 1-25, 2000.
J. C. Bezdek, “Pattern Recognition with Fuzzy Objective Function Algorithms,” Kluwer Academic Publishers Norwell, MA, USA, 1981.
M. Collins, S. Dasgupta, and R. Schapire, “A generalization of principal component analysis to the exponential family,” Advances in Neural Information Processing Systems, MIT Press, Vol.14, 2002.
T. Jebara, R. Kondor, and A. Howard, “Probability product kernels,” The Journal of Machine Learning Research, MIT Press Cambridge, MA, USA, Vol.5, pp. 819-844, 2004.
R. E. Kass and P. W. Vos, “Geometrical Foundations of Asymptotic Inference,” Wiley Series in Probability and Statistics, 1997.
J. Lafferty and G. Lebanon, “Diffusion kernels on statistical manifolds,” The Journal of Machine Learning Research, MIT Press Cambridge, MA, USA, Vol.6, pp. 129-163, 2005.
J. MacQueen, “Some methods for classification and analysis of multivariate observations,” Proc. of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Vol.1, pp. 281-297, 1967.
C. D. Manning and H. Schütze, “Foundations of statistical natural language processing,” MIT Press, 1999.
S. Miyamoto and M. Mukaidono, “Fuzzy c-means as a regularization and maximum entropy approach,” Proc. of the 7th International Fuzzy Systems Association World Congress (IFSA'97), Vol.2, pp. 86-92, 1997.

Fuzzy c-Means Algorithms Using Kullback-Leibler Divergence and Helliger Distance Based on Multinomial Manifold

Ryo Inokuchi* and Sadaaki Miyamoto**

Ryo Inokuchi^* and Sadaaki Miyamoto^**