On Bayesian Clustering with a Structured Gaussian Mixture
Department of Computational Intelligence and Systems Science, Tokyo Institute of Technology, G5-19, 4259 Nagatsuta, Midori-ku, Yokohama 226-8503, Japan
Cluster analysis is commonly used in the fields of computational intelligence and pattern recognition. The task is to detect the unobservable labels that show to which clusters the observable data belong. A Gaussian mixture is a representative hierarchical model that is often used when taking a probabilistic approach to this task. Although it is widely used, the statistical properties of cluster analysis have not yet been clarified. The present paper analyzes the theory of Bayesian clustering for the case when the number of clusters is unknown and the variance-covariance matrix of the Gaussian distribution has a constraint. We refer to this constraint as the structure of the components. The result of this analysis shows that, even if the estimation method does not take account of the structure, the Bayes method provides an effective, tractable, and efficient algorithm. Based on an experiment with simulated data, we confirmed the advantages of the Bayes method over the expectationmaximization (EM) method.
-  K. Yamazaki, “A Theoretical Analysis of KL-type Generalization Error on Hidden Variable Distribution,” Technical Report NC2010-165, IEICE, 2011.
-  C. Andrieu, N. d. Freitas, A. Doucet, and M. I. Jordan, “An Introduction to MCMC for Machine Learning,” Machine Learning, Vol.50, No.1-2, pp. 5-43, 2003.
-  M. E. Tipping and C. M. Bishop, “Probabilistic Principal Component Analysis,” J. of the Royal Statistical Society, Series B, Vol.61, pp. 611-622, 1999.
-  M. E. Tipping and C. M. Bishop, “Mixtures of probabilistic principal component analyzers,” Neural Computation, Vol.11, No.2, pp. 443-482, February 1999.
-  C. M. Bishop, “Bayesian PCA,” In NIPS, pp. 382-388, 1998.
-  C. M. Bishop, “Variational principal components,” Proc. 9th Int. Conf. on Artificial Neural Networks (ICANN’99), pp. 509-514, 1999.
-  S. Watanabe, “Algebraic Geometry and Statistical Learning Theory,” Cambridge University Press, New York, NY, USA, 2009.