Paper:

# Tsallis Entropy-Based Fuzzy Latent Semantics Analysis

## Yuchi Kanzawa

Shibaura Institute of Technology

3-7-5 Toyosu, Koto, Tokyo 135-8548, Japan

In this study, we present a fuzzy counterpart to the probabilistic latent semantic analysis (PLSA) approach. It is derived by solving the optimization problem of Tsallis entropy-penalizing free energy of a pseudo PLSA model by using a modified i.i.d. assumption. This derivation is similar to that of the conventional fuzzy counterpart of the PLSA, which involves solving the optimization problem of Shannon entropy-penalizing free energy. Furthermore, the proposed method is validated using numerical examples.

*J. Adv. Comput. Intell. Intell. Inform.*, Vol.24, No.1, pp. 58-64, 2020.

- [1] S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, and R. Harshman, “Indexing by latent semantic analysis,” J. of the American Society for Information Science, Vol.41, No.6, pp. 391-407, 1990.
- [2] T. Hofmann, “Probabilistic latent semantic analysis,” Proc. 15th Conf. on Uncertainty in Artificial Intelligence, pp. 289-296, 1999.
- [3] D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent Dirichlet allocation,” J. of Machine Learning Research, Vol.3, pp. 993-1022, 2003.
- [4] K. Honda, T. Goshima, S. Ubukata, and A. Notsu, “A Fuzzy Co-clustering Interpretation of Probabilistic Latent Semantic Analysis,” Proc. 25th IEEE Int. Conf. on Fuzzy Systems, pp. 718-723, 2016.
- [5] H. Ichihashi, K. Miyagishi, and K. Honda, “Fuzzy c-means clustering with regularization by K-L information,” Proc. 10th IEEE Int. Conf. on Fuzzy Systems, Vol.3, pp. 924-927, 2001.
- [6] K. Honda, S. Oshio, and A. Notsu, “Fuzzy co-clustering induced by multinomial mixture models,” J. Adv. Comput. Intell. Intell. Inform., Vol.19, No.6, pp. 717-726, 2015.
- [7] S. Miyamoto and M. Mukaidono, “Fuzzy
*c*-Means as a Regularization and Maximum Entropy Approach,” Proc. 7th Int. Fuzzy Systems Association World Congress (IFSA’97), Vol.2, pp. 86-92, 1997. - [8] S. Miyamoto, H. Ichihashi, and K. Honda, “Algorithms for Fuzzy Clustering,” Springer, 2008.
- [9] C. Tsallis, “Possible Generalization of Boltzmann-Gibbs Statistics,” J. Statist. Phys., Vol.52, No.1-2, pp. 479-487, 1988.
- [10] M. Ménard, V. Courboulay, and P.-A. Dardignac, “Possibilistic and Probabilistic Fuzzy Clustering: Unification within the Framework of the Nonextensive Thermostatistics,” Pattern Recognition, Vol.36, No.6, pp. 1325-1342, 2003.
- [11] Y. Kanzawa, “Fuzzy Clustering based on α-Divergence for Spherical Data and for Categorical Multivariate Data,” Proc. 24th IEEE Int. Conf. on Fuzzy Systems, pp. 1-8, 2015.
- [12] T. Hazan, R. Hardoon, and A. Shashua, “pLSA for Sparse Arrays With Tsallis Pseudo-Additive Divergence: Noise Robustness and Algorithm,” Proc. 11th IEEE Int. Conf. on Computer Vision, pp. 1-8, 2007.
- [13] L. Nivanen, A. Le Méhauté, and Q. A. Wang, “Generalized algebra within a nonextensive statistics,” Rep. Math. Phys., Vol.52, No.3, pp. 437-434, 2003.
- [14] Text REtrieval conference (TREC), http://trec.nist.gov [accessed February 10, 2017]
- [15] C. L. Blake and C. J. Merz, “UCI repository of machine learning databases, a huge collection of artificial and real-world data sets,” 1998, http://archive.ics.uci.edu/ml/ [accessed February 10, 2017]
- [16] D. D. Lewis, “Reuters-21578, text categorization test collection distribution 1.0,” http://www.daviddlewis.com/resources/testcollections/reuters21578/ [accessed February 15, 2017]
- [17] L. Azzopardi, M. Girolami, and K. van Risjbergen, “Investigating the Relationship between Language Model Perplexity and IR Precision-Recall Measures,” Proc. Special Interest Group on Information Retrieval (SIGIR)’03, pp. 369-370, 2003.
- [18] M. Steyvers, P. Smyth, and T. Griffiths, “Probabilistic Author-topic Models for information Discovery,” Proc. Knowledge Discovery and Data Mining (KDD)’04, pp. 306-315, 2004.