Tsallis Entropy-Based Fuzzy Latent Semantics Analysis

Yuchi Kanzawa

doi:10.20965/jaciii.2020.p0058

single-jc.php

« previous

JACIII Vol.24 No.1 pp. 58-64

doi: 10.20965/jaciii.2020.p0058

(2020)

Paper:

Views over last 60 days: 832

Tsallis Entropy-Based Fuzzy Latent Semantics Analysis

Yuchi Kanzawa

Shibaura Institute of Technology
3-7-5 Toyosu, Koto, Tokyo 135-8548, Japan

Received:

March 20, 2019

Accepted:

September 12, 2019

Published:

January 20, 2020

Keywords:

latent semantic analysis, Tsallis entropy, fuzzy clustering

Abstract

In this study, we present a fuzzy counterpart to the probabilistic latent semantic analysis (PLSA) approach. It is derived by solving the optimization problem of Tsallis entropy-penalizing free energy of a pseudo PLSA model by using a modified i.i.d. assumption. This derivation is similar to that of the conventional fuzzy counterpart of the PLSA, which involves solving the optimization problem of Shannon entropy-penalizing free energy. Furthermore, the proposed method is validated using numerical examples.

Cite this article as:

Y. Kanzawa, “Tsallis Entropy-Based Fuzzy Latent Semantics Analysis,” J. Adv. Comput. Intell. Intell. Inform., Vol.24 No.1, pp. 58-64, 2020.

Data files:

References

[1] S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman, “Indexing by latent semantic analysis,” J. of the American Society for Information Science, Vol.41, No.6, pp. 391-407, 1990.
[2] T. Hofmann, “Probabilistic latent semantic analysis,” Proc. 15th Conf. on Uncertainty in Artificial Intelligence, pp. 289-296, 1999.
[3] D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent Dirichlet allocation,” J. of Machine Learning Research, Vol.3, pp. 993-1022, 2003.
[4] K. Honda, T. Goshima, S. Ubukata, and A. Notsu, “A Fuzzy Co-clustering Interpretation of Probabilistic Latent Semantic Analysis,” Proc. 25th IEEE Int. Conf. on Fuzzy Systems, pp. 718-723, 2016.
[5] H. Ichihashi, K. Miyagishi, and K. Honda, “Fuzzy c-means clustering with regularization by K-L information,” Proc. 10th IEEE Int. Conf. on Fuzzy Systems, Vol.3, pp. 924-927, 2001.
[6] K. Honda, S. Oshio, and A. Notsu, “Fuzzy co-clustering induced by multinomial mixture models,” J. Adv. Comput. Intell. Intell. Inform., Vol.19, No.6, pp. 717-726, 2015.
[7] S. Miyamoto and M. Mukaidono, “Fuzzy c-Means as a Regularization and Maximum Entropy Approach,” Proc. 7th Int. Fuzzy Systems Association World Congress (IFSA’97), Vol.2, pp. 86-92, 1997.
[8] S. Miyamoto, H. Ichihashi, and K. Honda, “Algorithms for Fuzzy Clustering,” Springer, 2008.
[9] C. Tsallis, “Possible Generalization of Boltzmann-Gibbs Statistics,” J. Statist. Phys., Vol.52, No.1-2, pp. 479-487, 1988.
[10] M. Ménard, V. Courboulay, and P.-A. Dardignac, “Possibilistic and Probabilistic Fuzzy Clustering: Unification within the Framework of the Nonextensive Thermostatistics,” Pattern Recognition, Vol.36, No.6, pp. 1325-1342, 2003.
[11] Y. Kanzawa, “Fuzzy Clustering based on α-Divergence for Spherical Data and for Categorical Multivariate Data,” Proc. 24th IEEE Int. Conf. on Fuzzy Systems, pp. 1-8, 2015.
[12] T. Hazan, R. Hardoon, and A. Shashua, “pLSA for Sparse Arrays With Tsallis Pseudo-Additive Divergence: Noise Robustness and Algorithm,” Proc. 11th IEEE Int. Conf. on Computer Vision, pp. 1-8, 2007.
[13] L. Nivanen, A. Le Méhauté, and Q. A. Wang, “Generalized algebra within a nonextensive statistics,” Rep. Math. Phys., Vol.52, No.3, pp. 437-434, 2003.
[14] Text REtrieval conference (TREC), http://trec.nist.gov [accessed February 10, 2017]
[15] C. L. Blake and C. J. Merz, “UCI repository of machine learning databases, a huge collection of artificial and real-world data sets,” 1998, http://archive.ics.uci.edu/ml/ [accessed February 10, 2017]
[16] D. D. Lewis, “Reuters-21578, text categorization test collection distribution 1.0,” http://www.daviddlewis.com/resources/testcollections/reuters21578/ [accessed February 15, 2017]
[17] L. Azzopardi, M. Girolami, and K. van Risjbergen, “Investigating the Relationship between Language Model Perplexity and IR Precision-Recall Measures,” Proc. Special Interest Group on Information Retrieval (SIGIR)’03, pp. 369-370, 2003.
[18] M. Steyvers, P. Smyth, and T. Griffiths, “Probabilistic Author-topic Models for information Discovery,” Proc. Knowledge Discovery and Data Mining (KDD)’04, pp. 306-315, 2004.

This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

[1] [1] S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman, “Indexing by latent semantic analysis,” J. of the American Society for Information Science, Vol.41, No.6, pp. 391-407, 1990.

[2] [2] T. Hofmann, “Probabilistic latent semantic analysis,” Proc. 15th Conf. on Uncertainty in Artificial Intelligence, pp. 289-296, 1999.

[3] [3] D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent Dirichlet allocation,” J. of Machine Learning Research, Vol.3, pp. 993-1022, 2003.

[4] [4] K. Honda, T. Goshima, S. Ubukata, and A. Notsu, “A Fuzzy Co-clustering Interpretation of Probabilistic Latent Semantic Analysis,” Proc. 25th IEEE Int. Conf. on Fuzzy Systems, pp. 718-723, 2016.

[5] [5] H. Ichihashi, K. Miyagishi, and K. Honda, “Fuzzy c-means clustering with regularization by K-L information,” Proc. 10th IEEE Int. Conf. on Fuzzy Systems, Vol.3, pp. 924-927, 2001.

[6] [6] K. Honda, S. Oshio, and A. Notsu, “Fuzzy co-clustering induced by multinomial mixture models,” J. Adv. Comput. Intell. Intell. Inform., Vol.19, No.6, pp. 717-726, 2015.

[7] [7] S. Miyamoto and M. Mukaidono, “Fuzzy c-Means as a Regularization and Maximum Entropy Approach,” Proc. 7th Int. Fuzzy Systems Association World Congress (IFSA’97), Vol.2, pp. 86-92, 1997.

[8] [8] S. Miyamoto, H. Ichihashi, and K. Honda, “Algorithms for Fuzzy Clustering,” Springer, 2008.

[9] [9] C. Tsallis, “Possible Generalization of Boltzmann-Gibbs Statistics,” J. Statist. Phys., Vol.52, No.1-2, pp. 479-487, 1988.

[10] [10] M. Ménard, V. Courboulay, and P.-A. Dardignac, “Possibilistic and Probabilistic Fuzzy Clustering: Unification within the Framework of the Nonextensive Thermostatistics,” Pattern Recognition, Vol.36, No.6, pp. 1325-1342, 2003.

[11] [11] Y. Kanzawa, “Fuzzy Clustering based on α-Divergence for Spherical Data and for Categorical Multivariate Data,” Proc. 24th IEEE Int. Conf. on Fuzzy Systems, pp. 1-8, 2015.

[12] [12] T. Hazan, R. Hardoon, and A. Shashua, “pLSA for Sparse Arrays With Tsallis Pseudo-Additive Divergence: Noise Robustness and Algorithm,” Proc. 11th IEEE Int. Conf. on Computer Vision, pp. 1-8, 2007.

[13] [13] L. Nivanen, A. Le Méhauté, and Q. A. Wang, “Generalized algebra within a nonextensive statistics,” Rep. Math. Phys., Vol.52, No.3, pp. 437-434, 2003.

[14] [14] Text REtrieval conference (TREC), http://trec.nist.gov [accessed February 10, 2017]

[15] [15] C. L. Blake and C. J. Merz, “UCI repository of machine learning databases, a huge collection of artificial and real-world data sets,” 1998, http://archive.ics.uci.edu/ml/ [accessed February 10, 2017]

[16] [16] D. D. Lewis, “Reuters-21578, text categorization test collection distribution 1.0,” http://www.daviddlewis.com/resources/testcollections/reuters21578/ [accessed February 15, 2017]

[17] [17] L. Azzopardi, M. Girolami, and K. van Risjbergen, “Investigating the Relationship between Language Model Perplexity and IR Precision-Recall Measures,” Proc. Special Interest Group on Information Retrieval (SIGIR)’03, pp. 369-370, 2003.

[18] [18] M. Steyvers, P. Smyth, and T. Griffiths, “Probabilistic Author-topic Models for information Discovery,” Proc. Knowledge Discovery and Data Mining (KDD)’04, pp. 306-315, 2004.