Approximate Determination of <i>q</i>-Parameter for FCM with Tsallis Entropy Maximization

Makoto Yasuda

doi:10.20965/jaciii.2017.p1152

single-jc.php

« previous

JACIII Vol.21 No.7 pp. 1152-1160

doi: 10.20965/jaciii.2017.p1152

(2017)

Paper:

Views over last 60 days: 1,152

Approximate Determination of q-Parameter for FCM with Tsallis Entropy Maximization

Makoto Yasuda

National Institute of Technology, Gifu College
Kamimakuwa 2236-2, Motosu, Gifu 501-0495, Japan

Received:

February 20, 2017

Accepted:

July 21, 2017

Published:

November 20, 2017

Keywords:

fuzzy c-means clustering, entropy maximization, entropy regularization, Tsallis entropy, deterministic annealing

Abstract

This paper considers a fuzzy c-means (FCM) clustering algorithm in combination with deterministic annealing and the Tsallis entropy maximization. The Tsallis entropy is a q-parameter extension of the Shannon entropy. By maximizing the Tsallis entropy within the framework of FCM, statistical mechanical membership functions can be derived. One of the major considerations when using this method is how to determine appropriate values for q and the highest annealing temperature, T_high, for a given data set. Accordingly, in this paper, a method for determining these values simultaneously without introducing any additional parameters is presented, where the membership function is approximated using a series expansion method. The results of experiments indicate that the proposed method is effective, and both q and T_high can be determined automatically and algebraically from a given data set.

Cite this article as:

M. Yasuda, “Approximate Determination of q-Parameter for FCM with Tsallis Entropy Maximization,” J. Adv. Comput. Intell. Intell. Inform., Vol.21 No.7, pp. 1152-1160, 2017.

Data files:

References

[1] K. Rose, “Deterministic annealing for clustering, compression, classification, regression, and related optimization problems,” Proc. of the IEEE, Vol.86, No.11, pp. 2210-2239, 1998.
[2] K. Rose, E. Gurewitz, and B. C. Fox, “A deterministic annealing approach to clustering,” Pattern Recognition Letters, Vol.11, No.9, pp. 589-594, 1990.
[3] S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, “Optimization by simulated annealing,” Science, Vol.220, pp. 671-680, 1983.
[4] J. C. Bezdek, “Pattern Recognition with Fuzzy Objective Function Algorithms,” Prenum Press, New York, 1981.
[5] K. Honda and H. Ichihashi, “A regularization approach to fuzzy clustering with nonlinear membership weights,” J. Adv. Comput. Intell. Intell. Inform., Vol.11, No.1, pp. 28-34, Fuji Technology Press Ltd., 2007.
[6] Y. Kanzawa, “Entropy-based fuzzy clustering for non-euclidean relational data and indefinite kernel data,” J. Adv. Comput. Intell. Intell. Inform., Vol.16, No.7, pp.784-792, Fuji Technology Press Ltd., 2012.
[7] C. Tsallis, “Possible generalization of Boltzmann-Gibbs statistics,” J. of Statistical Phys., Vol.52, pp. 479-487, 1988.
[8] S. Abe and Y. Okamoto (ed.), “Nonextensive Statistical Mechanics and Its Applications,” Springer, Berlin, 2001.
[9] M. Gell-Mann and C. Tsallis (ed.), “Nonextensive Entropy – Interdisciplinary Applications,” Oxford University Press, New York, 2004.
[10] M. Menard, V. Courboulay, and P. Dardignac, “Possibilistic and probabilistic fuzzy clustering: unification within the framework of the non-extensive thermostatistics,” Pattern Recognition, Vol.36, pp. 1325-1342, 2003.
[11] M. Menard, P. Dardignac, and C. C. Chibelushi, “Non-extensive thermostatistics and extreme physical information for fuzzy clustering,” Int. J. of Computational Cognition, Vol.2, No.4, pp. 1-63, 2004.
[12] M. Yasuda, “Entropy maximization and very fast deterministic annealing approach to fuzzy c-means clustering,” Proc. of the Joint 5th Int. Conf. on Soft Computing and 11th Int. Symp. on Advanced Intelligent Systems, SU-B1-3, pp. 1515-1520, 2010.
[13] M. Yasuda and Y. Orito, “Multi-q extension of Tsallis entropy based fuzzy c-means clustering,” J. Adv. Comput. Intell. Intell. Inform., Vol.18, No.3, pp .289-296, Fuji Technology Press Ltd., 2014.
[14] M. Yasuda, “Approximate determination of q-parameter for FCM with Tsallis entropy maximization,” Proc. of the Joint 8th Int. Conf. on Soft Computing and 17th Int. Symp. on Advanced Intelligent Systems, pp. 700-705, 2016.
[15] UCI Machine Learning Repository: Iris Data Set, http://archive.ics.uci.edu/ml/datasets/Iris/ [donated Jul. 1, 1998]
[16] L. E. Reichl, “A Modern Corse in Statistical Physics,” John Wiley and Sons, New York, 1998.
[17] A. McCallum, K. Nigam, and L. H. Ungar, “Efficient clustering of high dimensional data sets with application to reference matching,” Proc. of the 6th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, pp. 169-178, 2000.

This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

[1] [1] K. Rose, “Deterministic annealing for clustering, compression, classification, regression, and related optimization problems,” Proc. of the IEEE, Vol.86, No.11, pp. 2210-2239, 1998.

[2] [2] K. Rose, E. Gurewitz, and B. C. Fox, “A deterministic annealing approach to clustering,” Pattern Recognition Letters, Vol.11, No.9, pp. 589-594, 1990.

[3] [3] S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, “Optimization by simulated annealing,” Science, Vol.220, pp. 671-680, 1983.

[4] [4] J. C. Bezdek, “Pattern Recognition with Fuzzy Objective Function Algorithms,” Prenum Press, New York, 1981.

[5] [5] K. Honda and H. Ichihashi, “A regularization approach to fuzzy clustering with nonlinear membership weights,” J. Adv. Comput. Intell. Intell. Inform., Vol.11, No.1, pp. 28-34, Fuji Technology Press Ltd., 2007.

[6] [6] Y. Kanzawa, “Entropy-based fuzzy clustering for non-euclidean relational data and indefinite kernel data,” J. Adv. Comput. Intell. Intell. Inform., Vol.16, No.7, pp.784-792, Fuji Technology Press Ltd., 2012.

[7] [7] C. Tsallis, “Possible generalization of Boltzmann-Gibbs statistics,” J. of Statistical Phys., Vol.52, pp. 479-487, 1988.

[8] [8] S. Abe and Y. Okamoto (ed.), “Nonextensive Statistical Mechanics and Its Applications,” Springer, Berlin, 2001.

[9] [9] M. Gell-Mann and C. Tsallis (ed.), “Nonextensive Entropy – Interdisciplinary Applications,” Oxford University Press, New York, 2004.

[10] [10] M. Menard, V. Courboulay, and P. Dardignac, “Possibilistic and probabilistic fuzzy clustering: unification within the framework of the non-extensive thermostatistics,” Pattern Recognition, Vol.36, pp. 1325-1342, 2003.

[11] [11] M. Menard, P. Dardignac, and C. C. Chibelushi, “Non-extensive thermostatistics and extreme physical information for fuzzy clustering,” Int. J. of Computational Cognition, Vol.2, No.4, pp. 1-63, 2004.

[12] [12] M. Yasuda, “Entropy maximization and very fast deterministic annealing approach to fuzzy c-means clustering,” Proc. of the Joint 5th Int. Conf. on Soft Computing and 11th Int. Symp. on Advanced Intelligent Systems, SU-B1-3, pp. 1515-1520, 2010.

[13] [13] M. Yasuda and Y. Orito, “Multi-q extension of Tsallis entropy based fuzzy c-means clustering,” J. Adv. Comput. Intell. Intell. Inform., Vol.18, No.3, pp .289-296, Fuji Technology Press Ltd., 2014.

[14] [14] M. Yasuda, “Approximate determination of q-parameter for FCM with Tsallis entropy maximization,” Proc. of the Joint 8th Int. Conf. on Soft Computing and 17th Int. Symp. on Advanced Intelligent Systems, pp. 700-705, 2016.

[15] [15] UCI Machine Learning Repository: Iris Data Set, http://archive.ics.uci.edu/ml/datasets/Iris/ [donated Jul. 1, 1998]

[16] [16] L. E. Reichl, “A Modern Corse in Statistical Physics,” John Wiley and Sons, New York, 1998.

[17] [17] A. McCallum, K. Nigam, and L. H. Ungar, “Efficient clustering of high dimensional data sets with application to reference matching,” Proc. of the 6th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, pp. 169-178, 2000.