Paper:
Language: English:

A Regularization Approach to Fuzzy Clustering with Nonlinear Membership Weights

Katsuhiro Honda and Hidetomo Ichihashi

Graduate School of Engineering, Osaka Prefecture University, 1-1 Gakuen-cho, Nakaku, Sakai, Osaka 599-8531, Japan

Received: October 31, 2005

Accepted: March 31, 2006

Keywords: fuzzy clustering, fuzzification of membership, regularization

Journal ref: Journal of Advanced Computational Intelligence and Intelligent Informatics, Vol.11, No.1 pp. 28-34, 2007

Abstract

Fuzzy c-means (FCM) is the fuzzy version of c-means clustering, in which memberships are fuzzified by introducing an additional parameter into the linear objective function of the weighted sum of distances between datapoints and cluster centers. Regularization of hard c-means clustering is another approach to fuzzification, in which regularization terms such as entropy and quadratic terms have been adopted. We generalized the fuzzification concept and propose a new approach to fuzzy clustering in which linear weights of hard c-means clustering are replaced by nonlinear ones through regularization. Numerical experiments demonstrated that the proposed algorithm has the characteristic features of the standard FCM algorithm and of regularization approaches. One of the proposed nonlinear weights makes it possible to both to attract data to clusters and to repulse different clusters. This feature derives different types of fuzzy classification functions in both probabilistic and possibilistic models.

Reference

[1] J. B. MacQueen, “Some methods of classification and analysis of multivariate observations,” Proc. 5th Berkeley Symposium on Math. Stat. and Prob., pp. 281-297, 1967.

[2] J. C. Bezdek, “Pattern Recognition with Fuzzy Objective Function Algorithms,” Plenum Press, 1981.

[3] S. Miyamoto and M. Mukaidono, “Fuzzy c-means as a regularization and maximum entropy approach,” Proc. 7th Int. Fuzzy Syst. Assoc. World Cong., Vol.2, pp. 86-92, 1997.

[4] S. Miyamoto and K. Umayahara, “Fuzzy clustering by quadratic regularization,” Proc. 1998 IEEE Int. Conf. Fuzzy Syst., pp. 1394-1399, 1998.

[5] F. Höppner, F. Klawonn, R. Kruse, and T. Runkler, “Fuzzy Cluster Analysis,” John Wiley & Sons, 1999.

[6] Z.-Q. Liu and S. Miyamoto (eds.), “Soft Computing and Human-Centered Machines,” Springer-Verlag, 2000.

[7] R. Krishnapuram and J. M. Keller, “A possibilistic approach to clustering,” IEEE Trans. Fuzzy Syst., Vol.1, pp. 98-110, 1993.

[8] H. Ichihashi, K. Miyagishi, and K. Honda, “Fuzzy c-means clustering with regularization by K-L information,” Proc. 2001 IEEE Int. Conf. Fuzzy Syst., pp. 924-927, 2001.

[9] M. Yasuda, T. Furuhashi, M. Matsuzaki, and S. Okuma, “Fuzzy clustering using deterministic annealing method and its statistical mechanical characteristics,” Proc. 2001 IEEE Int. Conf. Fuzzy Syst., 2001.

[10] M. Menard, V. Courboulay, and P.-A. Dardignac, “Possibilistic and probabilistic fuzzy clustering: Unification within the framework of the non-extensive thermostatistics,” Pattern Recognition, Vol.36, No.6, pp. 1325-1342, 2003.

[11] K. Honda and H. Ichihashi, “Regularized linear fuzzy clustering and probabilistic PCA mixture models,” IEEE Trans. Fuzzy Syst., Vol.13, No.4, pp. 508-516, 2005.