JACIII Vol.17 No.4 pp. 473-479
doi: 10.20965/jaciii.2013.p0473


Constructing Generative Topographic Mapping by Variational Bayes with ARD Hierarchical Prior

Nobuhiko Yamaguchi

Graduate School of Science and Engineering, Saga University, 1 Honjo, Saga 840-8502, Japan

January 11, 2013
April 2, 2013
July 20, 2013
generative topographic mapping, data visualization, variational Bayes, automatic relevance determination

Generative Topographic Mapping (GTM) is a nonlinear latent variable model introduced as a data visualization technique by Bishop et al. In this paper, we focus on variational Bayesian inference in GTM. Variational Bayesian GTM, first proposed by Olier et al., uses a single regularization term and regularization parameter to avoid overfitting and therefore cannot be used to control the degree of regularization locally. To overcome this problem, we propose variational Bayesian inference with Automatic Relevance Determination (ARD) hierarchical prior for use with GTM. The proposed model uses multiple regularization parameters and therefore can be used to control the degree of regularization in local areas of data space individually. Several experiments show that GTM that we propose provides better visualization than conventional GTM approaches.

Cite this article as:
Nobuhiko Yamaguchi, “Constructing Generative Topographic Mapping by Variational Bayes with ARD Hierarchical Prior,” J. Adv. Comput. Intell. Intell. Inform., Vol.17, No.4, pp. 473-479, 2013.
Data files:
  1. [1] C. M. Bishop, M. Svensén, and C. K. I. Williams, “GTM: the generative topographic mapping,” Neural Computation, Vol.10, No.1, pp. 215-234, 1998.
  2. [2] A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum Likelihood from Incomplete Data via the EM Algorithm,” J. of the Royal Statistical Society. Series B (Methodological), Vol.39, No.1, pp. 1-38, 1977.
  3. [3] T. Kohonen, M. R. Schroeder, and T. S. Huang (Eds.), “Self-Organizing Maps,” Springer-Verlag, 2001.
  4. [4] C. M. Bishop and C. K. I. Williams, “Developments of the generative topographic mapping,” Neurocomputing, Vol.21, pp. 203-224, 1998.
  5. [5] A. Vellido, W. El-Deredy, and P. J. G. Lisboa, “Selective smoothing of the generative topographic mapping,” IEEE Trans. on Neural Networks, Vol.14, No.4, pp. 847-852, 2003.
  6. [6] D. J. C. MacKay, “Probable networks and plausible predictions – a review of practical Bayesian methods for supervised neural networks,” Network: Computation in Neural Systems, Vol.6, No.3, pp. 469-505, 1995.
  7. [7] H. Attias, “Inferring parameters and structure of latent variable models by variational bayes,” Proc. of the 15th Conf. on Uncertainty in artificial intelligence, pp. 21-30, 1999.
  8. [8] M. Beal, “Variational Algorithms for Approximate Bayesian Inference,” Ph.D. thesis, Gatsby Computational Neuroscience Unit, University College London, 2003.
  9. [9] I. Olier, J. Amengual, and A. Vellido, “Variational Bayesian Generative Topographic Mapping,” J. of Mathematical Modelling and Algorithms, Vol.7, No.4, pp. 371-387, 2008.
  10. [10] R. M. Neal, “Bayesian Learning for Neural Networks,” Springer-Verlag, New York, 1996.
  11. [11] C. Blake and C. Merz, “UCI Repository of machine learning databases,” 1998.
  12. [12] M. Sato, “Online Model Selection Based on the Variational Bayes,” Neural Computation, Vol.13, No.7, pp. 1649-1681, 2001.

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Mar. 05, 2021