Multi-<i>q</i> Extension of Tsallis Entropy Based Fuzzy <i>c</i>-Means Clustering

Makoto Yasuda; Yasuyuki Orito

doi:10.20965/jaciii.2014.p0289

single-jc.php

« previous

JACIII Vol.18 No.3 pp. 289-296

doi: 10.20965/jaciii.2014.p0289

(2014)

Paper:

Views over last 60 days: 529

Multi-q Extension of Tsallis Entropy Based Fuzzy c-Means Clustering

Makoto Yasuda and Yasuyuki Orito

Department of Electrical and Computer Engineering, Gifu National College of Technology, 2236-2 Kamimakuwa, Motosu-shi, Gifu 501-0495, Japan

Received:

April 8, 2013

Accepted:

March 12, 2014

Published:

May 20, 2014

Keywords:

fuzzy c-means clustering, tsallis entropy, entropy maximization (extremization), multi-q extension

Abstract

Tsallis entropy is a q-parameter extension of Shannon entropy. Based on the Tsallis entropy, we have introduced an entropy maximization method to fuzzy c-means clustering (FCM), and developed a new clustering algorithm using a single-q value. In this article, we propose a multi-q extension of the conventional single-q method. In this method, the qs are assigned individually to each cluster. Each q value is determined so that the membership function fits the corresponding cluster distribution. This is done to improve the accuracy of clustering over that of the conventional single-q method. Experiments are performed on randomly generated numerical data and Fisher’s iris dataset, and it is confirmed that the proposed method improves the accuracy of clustering and is superior to the conventional single-q method. If the parameters introduced in the proposed method can be optimized, it is expected that the clusters in data distributions that are composed of clusters of various sizes can be determined more accurately.

Cite this article as:

M. Yasuda and Y. Orito, “Multi-q Extension of Tsallis Entropy Based Fuzzy c-Means Clustering,” J. Adv. Comput. Intell. Intell. Inform., Vol.18 No.3, pp. 289-296, 2014.

Data files:

References

[1] J. C. Bezdek, “Pattern Recognition with Fuzzy Objective Function Algorithms,” Prenum Press, New York, 1981.
[2] R. -P. Li and M. Mukaidono, “A Maximum entropy approach to fuzzy clustering,” Proc. of the 4th IEEE Int. Conf. on Fuzzy Systems (FUZZ-IEEE/IFES’95), pp. 2227-2232, 1995.
[3] S. Miyamoto and M. Mukaidono, “Fuzzy c-means as a regularization and maximum entropy approach,” Proc. of the 7th Int. Fuzzy Systems Association World Congress, Vol.2, pp. 86-92, 1997.
[4] K. Rose, E. Gurewitz, and B. C. Fox, “A deterministic annealing approach to clustering,” Pattern Recognition Letters, Vol.11, No.9, pp. 589-594, 1990. 1997.
[5] A. De Luca and S. Termini, “A definition of a nonprobabilistic entropy in the setting of fuzzy sets theory,” Information and Control, Vol.20, pp. 301-312, 1972.
[6] M. Yasuda, T. Furuhashi, M. Matsuzaki, and S. Okuma, “Fuzzy clustering using deterministic annealing method and its statistical mechanical characteristics,” Proc. of the 10th IEEE Int. Conf. Fuzzy Systems, p. 294, 2001.
[7] C. Tsallis, “Possible generalization of Boltzmann-Gibbs statistics,” J. of Statistical Phys., Vol.52, pp. 479-487, 1988.
[8] S. Abe and Y. Okamoto (ed.), “Nonextensive Statistical Mechanics and Its Applications,” Springer, 2001.
[9] M. Gell-Mann and C. Tsallis (ed.) “Nonextensive Entropy – Interdisciplinary Applications,” Oxford University Press, New York, 2004.
[10] C. Tsallis (ed.) “Introduction to Nonextensive Statistical Mechanics,” Springer, 2009.
[11] M. Menard, V. Courboulay, and P. Dardignac, “Possibilistic and probabilistic fuzzy clustering: unification within the framework of the non-extensive thermostatistics,” Pattern Recognition, Vol.36, pp. 1325-1342, 2003.
[12] M. Menard, P. Dardignac, and C. C. Chibelushi, “Non-extensive thermostatistics and extreme physical information for fuzzy clustering,” Int. J. of Computational Cognition, Vol.2, No.4, pp. 1-63, 2004.
[13] M. Yasuda, “Entropy maximization and deterministic annealing approach to fuzzy c-means clustering,” Proc. of the Joint 4th Int. Conf. on Soft Computing and 9th Int. Symp. on Intelligent Systems, THA2-3, pp. 33-37, 2008.
[14] M. Yasuda, “Entropy maximization and very fast deterministic annealing approach to fuzzy c-means clustering,” Proc. of the Joint 5th Int. Conf. on Soft Computing and 11th Int. Symp. on Intelligent Systems, SU-B1-3, pp. 1515-1520, 2010.
[15] F. Rossi and N. Villa-Vialaneix, “Optimizing an organized modularity measure for topographic graph measure clustering: a deterministic annealing approach,” Neurocomputing, Vol.73, No.7-9, pp. 1142-1163, 2010.
[16] R.A. Fisher, “The use of multiple measurements in taxonomic problems,” Annals of Eugenics, 7, Part 2, pp. 179-188, 1936.
[17] L. E. Reichl, “A Modern Course in Statistical Physics,” New York: John Wiley & Sons, 1998.
[18] M. Yasuda, “Deterministic annealing approach to fuzzy c-means clustering based on entropy maximization,” Advances in Fuzzy Systems, 960635, 2011.
[19] M. Yasuda, “Tsallis entropy based fuzzy c-means clustering with parameter adjustment,” Proc. of the Joint 6th Int. Conf. on Soft Computing and 13th Int. Symp. on Intelligent Systems, F2-54-5, pp. 1534-1539, 2012.
[20] B. E. Rosen, “Function optimization based on advanced simulated annealing,” IEEE Workshop on Physics and Computation, PhysComp 92, pp. 289-293, 1992.

This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

[1] [1] J. C. Bezdek, “Pattern Recognition with Fuzzy Objective Function Algorithms,” Prenum Press, New York, 1981.

[2] [2] R. -P. Li and M. Mukaidono, “A Maximum entropy approach to fuzzy clustering,” Proc. of the 4th IEEE Int. Conf. on Fuzzy Systems (FUZZ-IEEE/IFES’95), pp. 2227-2232, 1995.

[3] [3] S. Miyamoto and M. Mukaidono, “Fuzzy c-means as a regularization and maximum entropy approach,” Proc. of the 7th Int. Fuzzy Systems Association World Congress, Vol.2, pp. 86-92, 1997.

[4] [4] K. Rose, E. Gurewitz, and B. C. Fox, “A deterministic annealing approach to clustering,” Pattern Recognition Letters, Vol.11, No.9, pp. 589-594, 1990. 1997.

[5] [5] A. De Luca and S. Termini, “A definition of a nonprobabilistic entropy in the setting of fuzzy sets theory,” Information and Control, Vol.20, pp. 301-312, 1972.

[6] [6] M. Yasuda, T. Furuhashi, M. Matsuzaki, and S. Okuma, “Fuzzy clustering using deterministic annealing method and its statistical mechanical characteristics,” Proc. of the 10th IEEE Int. Conf. Fuzzy Systems, p. 294, 2001.

[7] [7] C. Tsallis, “Possible generalization of Boltzmann-Gibbs statistics,” J. of Statistical Phys., Vol.52, pp. 479-487, 1988.

[8] [8] S. Abe and Y. Okamoto (ed.), “Nonextensive Statistical Mechanics and Its Applications,” Springer, 2001.

[9] [9] M. Gell-Mann and C. Tsallis (ed.) “Nonextensive Entropy – Interdisciplinary Applications,” Oxford University Press, New York, 2004.

[10] [10] C. Tsallis (ed.) “Introduction to Nonextensive Statistical Mechanics,” Springer, 2009.

[11] [11] M. Menard, V. Courboulay, and P. Dardignac, “Possibilistic and probabilistic fuzzy clustering: unification within the framework of the non-extensive thermostatistics,” Pattern Recognition, Vol.36, pp. 1325-1342, 2003.

[12] [12] M. Menard, P. Dardignac, and C. C. Chibelushi, “Non-extensive thermostatistics and extreme physical information for fuzzy clustering,” Int. J. of Computational Cognition, Vol.2, No.4, pp. 1-63, 2004.

[13] [13] M. Yasuda, “Entropy maximization and deterministic annealing approach to fuzzy c-means clustering,” Proc. of the Joint 4th Int. Conf. on Soft Computing and 9th Int. Symp. on Intelligent Systems, THA2-3, pp. 33-37, 2008.

[14] [14] M. Yasuda, “Entropy maximization and very fast deterministic annealing approach to fuzzy c-means clustering,” Proc. of the Joint 5th Int. Conf. on Soft Computing and 11th Int. Symp. on Intelligent Systems, SU-B1-3, pp. 1515-1520, 2010.

[15] [15] F. Rossi and N. Villa-Vialaneix, “Optimizing an organized modularity measure for topographic graph measure clustering: a deterministic annealing approach,” Neurocomputing, Vol.73, No.7-9, pp. 1142-1163, 2010.

[16] [16] R.A. Fisher, “The use of multiple measurements in taxonomic problems,” Annals of Eugenics, 7, Part 2, pp. 179-188, 1936.

[17] [17] L. E. Reichl, “A Modern Course in Statistical Physics,” New York: John Wiley & Sons, 1998.

[18] [18] M. Yasuda, “Deterministic annealing approach to fuzzy c-means clustering based on entropy maximization,” Advances in Fuzzy Systems, 960635, 2011.

[19] [19] M. Yasuda, “Tsallis entropy based fuzzy c-means clustering with parameter adjustment,” Proc. of the Joint 6th Int. Conf. on Soft Computing and 13th Int. Symp. on Intelligent Systems, F2-54-5, pp. 1534-1539, 2012.

[20] [20] B. E. Rosen, “Function optimization based on advanced simulated annealing,” IEEE Workshop on Physics and Computation, PhysComp 92, pp. 289-293, 1992.