Paper:

# Fuzzy Co-Clustering Induced by Multinomial Mixture Models

## Katsuhiro Honda, Shunnya Oshio, and Akira Notsu

Graduate School of Engineering, Osaka Prefecture University

1-1 Gakuen-cho, Nakaku, Sakai, Osaka 599-8531, Japan

A close connection between fuzzy *c*-means (FCM) and Gaussian mixture models (GMMs) have been discussed and several extended FCM algorithms were induced by the GMMs concept, where fuzzy partitions are proved to be more useful for revealing intrinsic cluster structures than probabilistic ones. Co-clustering is a promising technique for summarizing cooccurrence information such as document-keyword frequencies. In this paper, a fuzzy co-clustering model is induced based on the multinomial mixture models (MMMs) concept, in which the degree of fuzziness of both object and item fuzzy memberships can be properly tuned. The advantages of the dual fuzzy partition are demonstrated through several experimental results including document clustering applications.

*J. Adv. Comput. Intell. Intell. Inform.*, Vol.19, No.6, pp. 717-726, 2015.

- [1] J. C. Bezdek, “Pattern Recognition with Fuzzy Objective Function Algorithms,” Plenum Press, 1981.
- [2] J. B. MacQueen, “Some methods of classification and analysis of multivariate observations,” Proc. of 5
^{th}Berkeley Symp. on Math. Stat. and Prob.,” pp. 281-297, 1967. - [3] E. H. Ruspini, “A new approach to clustering,” Information and Control, Vol.15, No.1, pp. 22-32, July 1969.
- [4] N. R. Pal and J. C. Bezdek, “On cluster validity for the fuzzy
*c*-mean model,” IEEE Trans. Fuzzy Systems, Vol.3, pp. 370-379, Aug. 1995. - [5] J. Yu, Q. Cheng, and H. Huang, “Analysis of the weighting exponent in the FCM,” IEEE Trans. on Systems, Man, and Cybernetics, Part B: Cybernetics, Vol.34, No.1, pp. 634-639, 2004.
- [6] R. O. Duda and P. E. Hart, “Pattern Classification and Scene Analysis,” John Wiley & Sons, 1973.
- [7] C. M. Bishop, “Neural Networks for Pattern Recognition,” Clarendon Press, 1995.
- [8] R. J. Hathaway, “Another interpretation of the EM algorithm for mixture distributions,” Statistics & Probability Letters, Vol.4, pp. 53-56, 1986.
- [9] A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” J. of the Royal Statistical Society, Series B, Vol.39, pp. 1-38, 1977.
- [10] K. Rose, E. Gurewitz, and G. Fox, “A deterministic annealing approach to clustering,” Pattern Recognition Letters, Vol.11, pp. 589-594, 1990.
- [11] S. Miyamoto and M. Mukaidono, “Fuzzy
*c*-means as a regularization and maximum entropy approach,” Proc. of the 7^{th}Int. Fuzzy Systems Association World Congress, Vol.2, pp 86-92, 1997. - [12] S. Miyamoto, H. Ichihashi, and K. Honda, “Algorithms for Fuzzy Clustering,” Springer, 2008.
- [13] H. Ichihashi, K. Miyagishi, and K. Honda, “Fuzzy
*c*-means clustering with regularization by K-L information,” Proc. of 10^{th}IEEE Int. Conf. on Fuzzy Systems, Vol.2, pp. 924-927, 2001. - [14] M. E. Tipping and C. M. Bishop, “Mixtures of probabilistic principal component analysers,” Neural Computation, Vol.11, No.2, pp. 443-482, 1999.
- [15] K. Honda and H. Ichihashi, “Linear fuzzy clustering techniques with missing values and their application to local principal component analysis,” IEEE Trans. Fuzzy Systems, Vol.12, No.2, pp. 183-193, 2004.
- [16] K. Honda and H. Ichihashi, “Regularized linear fuzzy clustering and probabilistic PCA mixture models,” IEEE Trans. Fuzzy Systems, Vol.13, No.4, pp. 508-516, 2005.
- [17] C.-H. Oh, K. Honda, and H. Ichihashi, “Fuzzy clustering for categorical multivariate data,” Proc. of Joint 9
^{th}IFSA World Congress and 20^{th}NAFIPS Int. Conf., pp. 2154-2159, 2001. - [18] K. Kummamuru, A. Dhawale, and R. Krishnapuram, “Fuzzy co-clustering of documents and keywords,” Proc. 2003 IEEE Int. Conf. Fuzzy Systems, Vol.2, pp. 772-777, 2003.
- [19] S. Miyamoto and K. Umayahara, “Fuzzy clustering by quadratic regularization,” Proc. 1998 IEEE Int. Conf. Fuzzy Systems and IEEE World Congr. Computational Intelligence, Vol.2, pp. 1394-1399, 1998.
- [20] L. Rigouste, O. Cappée, and F. Yvon, “Inference and evaluation of the multinomial mixture model for text clustering,” Information Processing and Management, Vol.43, No.5, pp. 1260-1280, 2007.
- [21] F. Klawonn, “Understanding and controlling the membership degrees in fuzzy clustering,” From Data and Information Analysis to Knowledge Engineering, M, Spiliopoulou, R. Kruse, C. Borgelt, A. Nrnberger and W. Gaul (Ed.), Springer, Berlin Heidelberg, pp. 446-453, 2006.
- [22] I. Holmes, K. Harris, and C. Quince, “Dirichlet multinomial mixtures: generative models for microbial metagenomics,” PLoS ONE, Vol.7, Iss.2, e30126, 2012.
- [23] K. Honda, A. Notsu, and H. Ichihashi, “Fuzzy PCA-guided robust
*k*-means clustering,” IEEE Trans. on Fuzzy Systems, Vol.18, No.1, pp. 67-79, 2010. - [24] G. Salton and C. Buckley, “Term-weighting approaches in automatic text retrieval,” Information Processing and Management, Vol.24, Iss.5, pp. 513-523, 1988.
- [25] W. Wang and Y. Zhang, “On fuzzy cluster validity indices,” Fuzzy Sets and Systems, Vol.158, pp. 2095-2117, 2007.
- [26] K. Honda, M. Muranishi, A. Notsu, and H. Ichihashi, “FCM-type cluster validation in fuzzy co-clustering and collaborative filtering applicability,” Int. J. of Computer Science and Network Security, Vol.13, No.1, pp. 24-29, 2013.
- [27] F. Masulli and S. Rovetta, “Soft transition from probabilistic to possibilistic fuzzy clustering,” IEEE Trans. on Fuzzy Systems, Vol.14, No.4, pp. 516-527, Aug. 2006.
- [28] K. Honda, H. Ichihashi, F. Masulli, and S. Rovetta, “Linear fuzzy clustering with selection of variables using graded possibilistic approach,” IEEE Trans. Fuzzy Systems, Vol.15, No.5, pp. 878-889, 2007.
- [29] K. Sjölander, K. Karplus, M. Brown, R. Hughey, A. Krogh, I.Saira Mian, and D. Haussler, “Dirichlet mixtures: a method for improved detection of weak but significant protein sequence homology,” Computer Applications in the Biosciences, Vol.12, No.4, pp. 327-345, 1996.
- [30] X. Ye, Y.-K. Yu, and S. F. Altschul, “Compositional adjustment of Dirichlet mixture priors,” J. of Computational Biology, Vol.17, No.12, pp. 1607-1620, 2010.