single-jc.php

JACIII Vol.30 No.6 pp. 1866-1875
(2026)

Research Paper:

Multilingual Emotion Profiling of Large Language Models Through Literary Texts and Persona Conditioning

Naruki Shirahama*1,† ORCID Icon, Yuma Yoshimoto*2, Naofumi Nakaya*3 ORCID Icon, and Satoshi Watanabe*4 ORCID Icon

*1Shimonoseki City University
2-1-1 Daigaku-cho, Shimonoseki, Yamaguchi 751-8510, Japan

†Corresponding author

*2National Institute of Technology (KOSEN), Kitakyushu College
5-20-1 Shii, Kokuraminami, Kitakyushu, Fukuoka 802-0985, Japan

*3Juntendo University
3-6-1 Hinode, Urayasu, Chiba 279-0013, Japan

*4Shizuoka Institute of Science and Technology
2200-2 Toyosawa, Fukuroi, Shizuoka 437-0032, Japan

Received:
March 10, 2026
Accepted:
June 25, 2026
Published:
November 20, 2026
Keywords:
large language models, emotion analysis, multilingual evaluation, literary text analysis, reproducible benchmark
Abstract

This paper examines whether multilingual prompt-and-text conditions materially alter large language model (LLM) emotion evaluation rather than merely rephrasing the same task. Using OpenRouter as a unified inference layer, we evaluated six contemporary LLMs under three full-language conditions (Japanese, English, and Traditional Chinese), three literary texts, four personas, and three repeated API calls per condition. Each completion rated four emotional dimensions — Interesting, Surprise, Sadness, and Anger — and provided a brief rationale. The design comprised 648 planned evaluations (6 models × 3 languages × 3 texts × 4 personas × 3 trials). The final combined dataset contained 648 valid records, comprising 631 original records and 17 replacement records obtained under a repair amendment registered after the main run and before the patch run. Descriptive summaries and preregistered ordinary least squares analyses with HC3 robust standard errors showed that text content was a major driver of scores, especially for Sadness and Anger. Simultaneously, model-language interactions were significant for all four emotions, indicating that the effect of the language condition varied substantially across models. English yielded the highest mean Interesting, Surprise, and Sadness scores, whereas Japanese yielded the highest mean Anger score. However, these aggregate patterns mask the strong model-specific contrasts. The findings show that multilingual literary evaluation is shaped by both the text itself and the model processing of its translation. This study contributes a multilingual literary emotion benchmark, a preregistered and repair-aware evaluation pipeline, and evidence that multilingual conditions are substantive experimental factors for LLM emotion profiling.

Multilingual LLM emotion profiles

Multilingual LLM emotion profiles

Cite this article as:
N. Shirahama, Y. Yoshimoto, N. Nakaya, and S. Watanabe, “Multilingual Emotion Profiling of Large Language Models Through Literary Texts and Persona Conditioning,” J. Adv. Comput. Intell. Intell. Inform., Vol.30 No.6, pp. 1866-1875, 2026.
Data files:
References
  1. [1] N. Shirahama, N. Nakaya, and S. Watanabe, “Characterizing ‘emotional individuality’ in 36 LLMs: Persona and Japanese text conditioning,” S.-C. Chu, T.-C. Lu, J.-S. Pan, G. A. Tsihrintzis, and T.-W. Sung (Eds.), “Advances in Intelligent Information Hiding and Multimedia Signal Processing (IIHMSP 2025),” pp. 287-298, Springer, 2026. https://doi.org/10.1007/978-3-032-22588-7_27
  2. [2] S. Havaldar, S. Rai, B. Singhal, L. Liu, S. C. Guntuku, and L. Ungar, “Multilingual language models are not multicultural: A case study in emotion,” Proc. of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, and Social Media Analysis, pp. 202-214, 2023. https://doi.org/10.18653/v1/2023.wassa-1.19
  3. [3] T. D. Belay, A. H. Ahmed, A. Grissom II, I. Ameer, G. Sidorov, O. Kolesnikova, and S. M. Yimam, “CULEMO: Cultural lenses on emotion – Benchmarking LLMs for cross-cultural emotion understanding,” Proc. of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 18894-18909, 2025. https://doi.org/10.18653/v1/2025.acl-long.925
  4. [4] W. Lee, Y. Sim, H. Kim, and H. Kim, “Multilingual, not multicultural: Uncovering the cultural empathy gap in LLMs through a comparative empathetic dialogue benchmark,” Proc. of the 14th Int. Joint Conf. on Natural Language Processing and the 4th Conf. of the Asia-Pacific Chapter of the Association for Computational Linguistics, pp. 791-809, 2025. https://doi.org/10.18653/v1/2025.ijcnlp-long.44
  5. [5] Y. Chen and Y. Xiao, “Recent advancement of emotion cognition in large language models,” arXiv:2409.13354, 2024. https://doi.org/10.48550/arXiv.2409.13354
  6. [6] Y. Chen, S. Yan, S. Liu, Y. Li, and Y. Xiao, “EmotionQueen: A benchmark for evaluating empathy of large language models,” Findings of the Association for Computational Linguistics: ACL 2024, pp. 2149-2176, 2024. https://doi.org/10.18653/v1/2024.findings-acl.128
  7. [7] J.-T. Huang, M. Lam, E. Li, S. Ren, W. Wang, W. Jiao, Z. Tu, and M. Lyu, “Apathetic or empathetic? evaluating LLMs’ emotional alignments with humans,” Advances in Neural Information Processing Systems 37, pp. 97053-97087, 2024. https://doi.org/10.52202/079017-3077
  8. [8] S. J. Paech, “EQ-Bench: An emotional intelligence benchmark for large language models,” arXiv:2312.06281, 2024. https://doi.org/10.48550/arXiv.2312.06281
  9. [9] E. Jeon, M. Lee, S. Kim, J. Kim, W. Cho, T.-E. Kam, and S. Lee, ““Going to a trap house” conveys more fear than “Going to a mall”: Benchmarking emotion context sensitivity for LLMs,” Findings of the Association for Computational Linguistics: EMNLP 2025, pp. 14848-14869, 2025. https://doi.org/10.18653/v1/2025.findings-emnlp.802
  10. [10] S. Wu, Y. Zhu, W. Hsu, M.-L. Lee, and Y. Deng, “From personas to talks: Revisiting the impact of personas on LLM-synthesized emotional support conversations,” Proc. of the 2025 Conf. on Empirical Methods in Natural Language Processing, pp. 5439-5453, 2025. https://doi.org/10.18653/v1/2025.emnlp-main.277
  11. [11] Y. Jun and H. Lee, “Exploring persona sentiment sensitivity in personalized dialogue generation,” Proc. of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 18384-18402, 2025. https://doi.org/10.18653/v1/2025.acl-long.900
  12. [12] E. Kim and R. Klinger, “A survey on sentiment and emotion analysis for computational literary studies,” Zeitschrift für digitale Geisteswissenschaften, 2019. https://doi.org/10.17175/2019_008
  13. [13] E. Öhman and R. Rossi, “Affect as a proxy for literary mood,” J. of Data Mining and Digital Humanities, Vol.NLP4DH, 2023. https://doi.org/10.46298/jdmdh.11164
  14. [14] F. M. Plaza-del-Arco, A. A. Cercas Curry, A. Cercas Curry, and D. Hovy, “Emotion analysis in NLP: Trends, gaps and roadmap for future directions,” Joint Int. Conf. on Computational Linguistics, Language Resources and Evaluation, pp. 5696-5710, 2024. https://doi.org/10.63317/3kradf7tw6gm
  15. [15] L. P. Hung and S. Alias, “Beyond sentiment analysis: A review of recent trends in text based sentiment analysis and emotion detection,” J. Adv. Comput. Intell. Intell. Inform., Vol.27, No.1, pp. 84-95, 2023. https://doi.org/10.20965/jaciii.2023.p0084
  16. [16] N. Lin, “A CNN-BiGRU-GNN model for sentiment analysis of complex English texts,” J. Adv. Comput. Intell. Intell. Inform., Vol.30, No.1, pp. 144-155, 2026. https://doi.org/10.20965/jaciii.2026.p0144
  17. [17] G. C. Yeo and K. Jaidka, “Beyond context to cognitive appraisal: Emotion reasoning as a theory of mind benchmark for large language models,” Findings of the Association for Computational Linguistics: ACL 2025, pp. 26517-26525, 2025. https://doi.org/10.18653/v1/2025.findings-acl.1359
  18. [18] J. Huang, W. Jiao, M. H. Lam, E. J. Li, W. Wang, and M. Lyu, “On the reliability of psychological scales on large language models,” Proc. of the 2024 Conf. on Empirical Methods in Natural Language Processing, pp. 6152-6173, 2024. https://doi.org/10.18653/v1/2024.emnlp-main.354
  19. [19] Y. Tokutake and K. Okamoto, “Can large language models assess serendipity in recommender systems?,” J. Adv. Comput. Intell. Intell. Inform., Vol.28, No.6, pp. 1263-1272, 2024. https://doi.org/10.20965/jaciii.2024.p1263
  20. [20] S. H. Muhammad, N. Ousidhoum, I. Abdulmumin, S. M. Yimam, J. P. Wahle, T. L. Ruas, M. Beloucif, C. De Kock, T. D. Belay, I. S. Ahmad, N. Surange, D. Teodorescu, D. I. Adelani, A. F. Aji, F. D. M. Ali, V. Araujo, A. A. Ayele, O. Ignat, A. Panchenko, Y. Zhou, and S. Mohammad, “SemEval-2025 task 11: Bridging the gap in text-based emotion detection,” Proc. of the 19th Int. Workshop on Semantic Evaluation (SemEval-2025), pp. 2558-2569, 2025.
  21. [21] K. Takahashi, T. Oka, and M. Komachi, “Effectiveness of pre-trained language models for the Japanese Winograd schema challenge,” J. Adv. Comput. Intell. Intell. Inform., Vol.27, No.3, pp. 511-521, 2023. https://doi.org/10.20965/jaciii.2023.p0511
  22. [22] N. Shirahama, N. Nakaya, S. Watanabe, K. Moriya, K. Matsumoto, and K. Koshi, “Assessing emotional intelligence in AI: Visual analogue scale vs. Likert scale in short story comprehension,” J. of the Institute of Industrial Applications Engineers, Vol.13, No.1, pp. 21-31, 2025. https://doi.org/10.12792/jiiae.13.21
  23. [23] N. Shirahama, N. Nakaya, and S. Watanabe, “Development of an experimental platform for evaluating LLM emotion understanding,” ICIC Express Letters, Vol.20, No.2, pp. 191-198, 2026. https://doi.org/10.24507/icicel.20.02.191
  24. [24] N. Shirahama, N. Nakaya, and S. Watanabe, “Temperature-controlled response reliability in large language models: A quantitative analysis of 36 models for emotion understanding tasks,” The 12th Int. Conf. on Intelligent Systems and Image Processing 2025, pp. 78-84, 2025. https://doi.org/10.12792/icisip2025.016
  25. [25] Z. Lian, H. Chen, L. Chen, H. Sun, L. Sun, Y. Ren, Z. Cheng, B. Liu, R. Liu, X. Peng, J. Yi, and J. Tao, “AffectGPT: A new dataset, model, and benchmark for emotion understanding with multimodal large language models,” Proc. of the 42nd Int. Conf. on Machine Learning, pp. 36993-37014, 2025.
  26. [26] H. Zhao, L. Li, S. Chen, S. Kong, J. Wang, K. Huang, T. Gu, Y. Wang, J. Wang, D. Liang, Z. Li, Y. Teng, Y. Xiao, and Y. Wang, “ESC-Eval: Evaluating emotion support conversations in large language models,” Proc. of the 2024 Conf. on Empirical Methods in Natural Language Processing, pp. 15785-15810, 2024. https://doi.org/10.18653/v1/2024.emnlp-main.883
  27. [27] Y. Zhou, Z. Zhang, J. Cao, J. Jia, Y. Jiang, F. Wen, X. Liu, X. Min, and G. Zhai, “MEMO-Bench: A multiple benchmark for text-to-image and multimodal large language models on human emotion analysis,” arXiv:2411.11235, 2024. https://doi.org/10.48550/arXiv.2411.11235

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Nov. 20, 2026