<i>q</i>-Divergence-Based Relational Fuzzy <i>c</i>-Means Clustering

Yuchi Kanzawa

doi:10.20965/jaciii.2018.p0034

single-jc.php

« previous

JACIII Vol.22 No.1 pp. 34-43

doi: 10.20965/jaciii.2018.p0034

(2018)

Paper:

Views over last 60 days: 1,066

q-Divergence-Based Relational Fuzzy c-Means Clustering

Yuchi Kanzawa

Shibaura Institute of Technology
3-7-5 Toyosu, Koto-ku, Tokyo 135-8548, Japan

Received:

November 20, 2016

Accepted:

September 23, 2017

Published:

January 20, 2018

Keywords:

fuzzy clustering, relational data, q-divergence

Abstract

In this paper, a clustering algorithm for relational data based on q-divergence between memberships and variables that control cluster sizes is proposed. A conventional method for vectorial data is first presented for interpretation as the regularization of another conventional method with q-divergence. With this interpretation, a clustering algorithm for relational data, based on q-divergence, is then derived from an optimization problem built by regularizing the conventional method with q-divergence. A theoretical discussion reveals the property of the proposed method. Numerical results are presented that substantiate this property and show that the proposed method outperforms two conventional methods in terms of accuracy.

Cite this article as:

Y. Kanzawa, “q-Divergence-Based Relational Fuzzy c-Means Clustering,” J. Adv. Comput. Intell. Intell. Inform., Vol.22 No.1, pp. 34-43, 2018.

Data files:

References

[1] J. B. MacQueen, “Some Methods of Classification and Analysis of Multivariate Observations,” Proc. 5th Berkeley Symposium on Math. Stat. and Prob., pp. 281-297, 1967.
[2] J. Bezdek, “Pattern Recognition with Fuzzy Objective Function Algorithms,” Plenum Press, New York, 1981.
[3] S. Miyamoto and M. Mukaidono, “Fuzzy c-Means as a Regularization and Maximum Entropy Approach,” Proc. 7th Int. Fuzzy Systems Association World Congress (IFSA’97), Vol.2, pp. 86-92, 1997.
[4] S. Miyamoto and N. Kurosawa, “Controlling Cluster Volume Sizes in Fuzzy c-means Clustering,” Proc. SCIS&ISIS2004, pp. 1-4, 2004.
[5] H. Ichihashi, K. Honda, and N. Tani, “Gaussian Mixture PDF Approximation and Fuzzy c-means Clustering with Entropy Regularization,” Proc. 4th Asian Fuzzy System Symposium, pp. 217-221, 2000.
[6] S. Miyamoto, H. Ichihashi, and K. Honda, “Algorithms for Fuzzy Clustering,” Springer, 2008.
[7] R. J. Hathaway, J. W. Davenport, and J. C. Bezdek, “Relational Duals of the c-means Clustering Algorithms,” Pattern Recog., Vol.22, No.2, pp. 205-212, 1989.
[8] R. J. Hathaway and J. C. Bezdek, “NERF C-means: Non-Euclidean Relational Fuzzy Clustering,” Pattern Recog., Vol.27, pp. 429-437, 1994.
[9] M. Filippone, “Dealing with Non-metric Dissimilarities in Fuzzy Central Clustering Algorithms,” Int. J. Approx. Reason., Vol.40, No.2, pp. 363-384, 2009.
[10] C. Tsallis, “Generalized Entropy-based Criterion for Consistent Testing,” Phys. Rev. E, Vol.58, pp. 1442-1445, 1998.
[11] C. Tsallis, “Possible Generalization of Boltzmann-Gibbs Statistics,” J. Statist. Phys., Vol.52, pp. 479-487, 1988.
[12] M. Menard, V. Courboulay, and P. Dardignac, “Possibilistic and Probabilistic Fuzzy Clustering:Unification within the Framework of the Non-extensive Thermostatistics,” Pattern Recogn., Vol.36, pp.1325-1342, 2003.
[13] J. Dunn, “A Fuzzy Relative of the Isodata Process and Its Use in Detecting Compact Well-Separated Clusters,” Cybernet. Syst., Vol.3, No.3, pp. 32-57, 1973.
[14] Y. Kanzawa, “Entropy-regularized Fuzzy Clustering for Non-Euclidean Relational Data and for Indefinite Kernel Data,” J. Adv. Comput. Intell. Intell. Inform., Vol.16, No.7, pp. 784-792, 2012.
[15] B. Haasdonk and C. Bahlmann, “Learning with Distance Substitution Kernels,” Pattern Recognition, pp. 220-227, 2004.
[16] M. Khalilia, J. C. Bezdek, M. Popescu, and J. M. Keller, “Improvements to the Relational Fuzzy c-Means Clustering Algorithm,” Pattern Recog., Vol.47, No.12, pp. 3920-3930, 2014.
[17] L. Hubert and P. Arabie, “Comparing Partitions,” J. of Classification, Vol.2, pp. 193-218, 1985.
[18] I. S. Dhillon and D. S. Modha, “Concept Decompositions for Large Sparse Text Data Using Clustering,” Machine Learning, Vol.42, pp. 143-175, 2001.
[19] S. Miyamoto and K. Mizutani, “Fuzzy Multiset Model and Methods of Nonlinear Document Clustering for Information Retrieval,” Modeling Decisions for Artificial Intelligence, pp. 273-283, 2004.
[20] K. Mizutani, R. Inokuchi, and S. Miyamoto, “Algorithms of Nonlinear Document Clustering based on Fuzzy Set Model,” Int. J. of Intel. Sys., Vol.23, No.2, pp. 176-198, 2008.
[21] Y. Kanzawa, “On Kernelization for a Maximizing Model of Bezdek-like Spherical Fuzzy c-means Clustering,” Modeling Decisions for Artificial Intelligence, pp. 108-121, 2014.
[22] Y. Kanzawa, “A Maximizing Model of Bezdek-like Spherical Fuzzy c-Means clustering,” J. Adv. Comput. Intell. Intell. Inform., Vol.19, No.5, pp. 662-669, 2015.
[23] Y. Kanzawa, “A Maximizing Model of Spherical Bezdek-type Fuzzy Multi-medoids Clustering,” J. Adv. Comput. Intell. Intell. Inform., Vol.19, No.6, pp. 738-746, 2015.
[24] C. Oh, K. Honda, and H. Ichihashi, “Fuzzy Clustering for Categorical Multivariate Data,” Proc. IFSA World Congress 20th NAFIPS Int. Conf., pp. 2154-2159, 2001.
[25] K. Honda, S. Oshio, and A. Notsu, “FCM-type Fuzzy Co-clustering by K-L Information Regularization,” Proc. 2014 IEEE Int. Conf. Fuzzy Sys., pp. 2505-2510, 2014.
[26] K. Honda, S. Oshio, and A. Notsu, “Item Membership Fuzzification in Fuzzy Co-clustering Based on Multinomial Mixture Concept,” Proc. 2014 IEEE Int. Conf. on Granular Comput., pp. 94-99, 2014.
[27] Y. Kanzawa, “Fuzzy Co-Clustering Algorithms Based on Fuzzy Relational Clustering and TIBA Imputation,” J. Adv. Comput. Intell. Intell. Inform., Vol.18, No.2, pp. 182-189, 2014.
[28] Y. Kanzawa, “On Possibilistic Clustering Methods Based on Shannon/Tsallis-Entropy for Spherical Data and Categorical Multivariate Data,” Modeling Decisions for Artificial Intelligence, pp. 125-138, 2015.
[29] Y. Kanzawa, “Bezdek-type Fuzzified Co-Clustering Algorithm,” J. Adv. Comput. Intell. Intell. Inform., Vol.19, No.6, pp. 852-860, 2015.
[30] A. Cichocki and S. Amari, “Families of Alpha- Beta- and Gamma- Divergences: Flexible and Robust Measures of Similarities,” Entropy, Vol.12, pp. 1532-1568, 2010.
[31] A. Cichocki, S. Cruces, and S. Amari, “Generalized Alpha-Beta Divergences and Their Application to Robust Nonnegative Matrix Factorization,” Entropy, Vol.13, pp. 134-170, 2011.

This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

[1] [1] J. B. MacQueen, “Some Methods of Classification and Analysis of Multivariate Observations,” Proc. 5th Berkeley Symposium on Math. Stat. and Prob., pp. 281-297, 1967.

[2] [2] J. Bezdek, “Pattern Recognition with Fuzzy Objective Function Algorithms,” Plenum Press, New York, 1981.

[3] [3] S. Miyamoto and M. Mukaidono, “Fuzzy c-Means as a Regularization and Maximum Entropy Approach,” Proc. 7th Int. Fuzzy Systems Association World Congress (IFSA’97), Vol.2, pp. 86-92, 1997.

[4] [4] S. Miyamoto and N. Kurosawa, “Controlling Cluster Volume Sizes in Fuzzy c-means Clustering,” Proc. SCIS&ISIS2004, pp. 1-4, 2004.

[5] [5] H. Ichihashi, K. Honda, and N. Tani, “Gaussian Mixture PDF Approximation and Fuzzy c-means Clustering with Entropy Regularization,” Proc. 4th Asian Fuzzy System Symposium, pp. 217-221, 2000.

[6] [6] S. Miyamoto, H. Ichihashi, and K. Honda, “Algorithms for Fuzzy Clustering,” Springer, 2008.

[7] [7] R. J. Hathaway, J. W. Davenport, and J. C. Bezdek, “Relational Duals of the c-means Clustering Algorithms,” Pattern Recog., Vol.22, No.2, pp. 205-212, 1989.

[8] [8] R. J. Hathaway and J. C. Bezdek, “NERF C-means: Non-Euclidean Relational Fuzzy Clustering,” Pattern Recog., Vol.27, pp. 429-437, 1994.

[9] [9] M. Filippone, “Dealing with Non-metric Dissimilarities in Fuzzy Central Clustering Algorithms,” Int. J. Approx. Reason., Vol.40, No.2, pp. 363-384, 2009.

[10] [10] C. Tsallis, “Generalized Entropy-based Criterion for Consistent Testing,” Phys. Rev. E, Vol.58, pp. 1442-1445, 1998.

[11] [11] C. Tsallis, “Possible Generalization of Boltzmann-Gibbs Statistics,” J. Statist. Phys., Vol.52, pp. 479-487, 1988.

[12] [12] M. Menard, V. Courboulay, and P. Dardignac, “Possibilistic and Probabilistic Fuzzy Clustering:Unification within the Framework of the Non-extensive Thermostatistics,” Pattern Recogn., Vol.36, pp.1325-1342, 2003.

[13] [13] J. Dunn, “A Fuzzy Relative of the Isodata Process and Its Use in Detecting Compact Well-Separated Clusters,” Cybernet. Syst., Vol.3, No.3, pp. 32-57, 1973.

[14] [14] Y. Kanzawa, “Entropy-regularized Fuzzy Clustering for Non-Euclidean Relational Data and for Indefinite Kernel Data,” J. Adv. Comput. Intell. Intell. Inform., Vol.16, No.7, pp. 784-792, 2012.

[15] [15] B. Haasdonk and C. Bahlmann, “Learning with Distance Substitution Kernels,” Pattern Recognition, pp. 220-227, 2004.

[16] [16] M. Khalilia, J. C. Bezdek, M. Popescu, and J. M. Keller, “Improvements to the Relational Fuzzy c-Means Clustering Algorithm,” Pattern Recog., Vol.47, No.12, pp. 3920-3930, 2014.

[17] [17] L. Hubert and P. Arabie, “Comparing Partitions,” J. of Classification, Vol.2, pp. 193-218, 1985.

[18] [18] I. S. Dhillon and D. S. Modha, “Concept Decompositions for Large Sparse Text Data Using Clustering,” Machine Learning, Vol.42, pp. 143-175, 2001.

[19] [19] S. Miyamoto and K. Mizutani, “Fuzzy Multiset Model and Methods of Nonlinear Document Clustering for Information Retrieval,” Modeling Decisions for Artificial Intelligence, pp. 273-283, 2004.

[20] [20] K. Mizutani, R. Inokuchi, and S. Miyamoto, “Algorithms of Nonlinear Document Clustering based on Fuzzy Set Model,” Int. J. of Intel. Sys., Vol.23, No.2, pp. 176-198, 2008.

[21] [21] Y. Kanzawa, “On Kernelization for a Maximizing Model of Bezdek-like Spherical Fuzzy c-means Clustering,” Modeling Decisions for Artificial Intelligence, pp. 108-121, 2014.

[22] [22] Y. Kanzawa, “A Maximizing Model of Bezdek-like Spherical Fuzzy c-Means clustering,” J. Adv. Comput. Intell. Intell. Inform., Vol.19, No.5, pp. 662-669, 2015.

[23] [23] Y. Kanzawa, “A Maximizing Model of Spherical Bezdek-type Fuzzy Multi-medoids Clustering,” J. Adv. Comput. Intell. Intell. Inform., Vol.19, No.6, pp. 738-746, 2015.

[24] [24] C. Oh, K. Honda, and H. Ichihashi, “Fuzzy Clustering for Categorical Multivariate Data,” Proc. IFSA World Congress 20th NAFIPS Int. Conf., pp. 2154-2159, 2001.

[25] [25] K. Honda, S. Oshio, and A. Notsu, “FCM-type Fuzzy Co-clustering by K-L Information Regularization,” Proc. 2014 IEEE Int. Conf. Fuzzy Sys., pp. 2505-2510, 2014.

[26] [26] K. Honda, S. Oshio, and A. Notsu, “Item Membership Fuzzification in Fuzzy Co-clustering Based on Multinomial Mixture Concept,” Proc. 2014 IEEE Int. Conf. on Granular Comput., pp. 94-99, 2014.

[27] [27] Y. Kanzawa, “Fuzzy Co-Clustering Algorithms Based on Fuzzy Relational Clustering and TIBA Imputation,” J. Adv. Comput. Intell. Intell. Inform., Vol.18, No.2, pp. 182-189, 2014.

[28] [28] Y. Kanzawa, “On Possibilistic Clustering Methods Based on Shannon/Tsallis-Entropy for Spherical Data and Categorical Multivariate Data,” Modeling Decisions for Artificial Intelligence, pp. 125-138, 2015.

[29] [29] Y. Kanzawa, “Bezdek-type Fuzzified Co-Clustering Algorithm,” J. Adv. Comput. Intell. Intell. Inform., Vol.19, No.6, pp. 852-860, 2015.

[30] [30] A. Cichocki and S. Amari, “Families of Alpha- Beta- and Gamma- Divergences: Flexible and Robust Measures of Similarities,” Entropy, Vol.12, pp. 1532-1568, 2010.

[31] [31] A. Cichocki, S. Cruces, and S. Amari, “Generalized Alpha-Beta Divergences and Their Application to Robust Nonnegative Matrix Factorization,” Entropy, Vol.13, pp. 134-170, 2011.