Advances in neural recording and deep learning have enabled a more precise analysis of how the brain processes language. However, neural representations of diverse emotions during natural narrative language comprehension remains unclear. In this study, we investigated how emotional content in naturalistic spoken language is reflected in brain activity using two independent narrative functional magnetic resonance imaging datasets: Alice and Le Petit Prince. Participants listened to narrative readings and their blood-oxygenation-level-dependent responses were recorded. An emotion recognition model was developed by fine-tuning BERT using the GoEmotions dataset together with sentence-level annotations for 80 emotions. Emotional and linguistic features were extracted and used to train voxel-wise encoding models to predict brain activity. To examine emotion-related neural representations, we subtracted predictions based on linguistic features from those based on combined emotional and linguistic features. The resulting emotion-related activity patterns were distributed across cortical regions associated with affective and semantic processing and were consistent across subjects and datasets. These findings indicate that the emotional meaning conveyed through natural language is represented in a graded and distributed manner in the human brain, beyond linguistic content alone.

1. [1] A. G. Huth, S. Nishimoto, A. T. Vu, and J. L. Gallant, “A continuous semantic space describes the representation of thousands of object and action categories across the human brain,” Neuron, Vol.76, No.6, pp. 1210-1224, 2012. https://doi.org/10.1016/j.neuron.2012.10.014
2. [2] A. S. Cowen and D. Keltner, “Self-report captures 27 distinct categories of emotion bridged by continuous gradients,” Proc. of the National Academy of Sciences, Vol.114, No.38, pp. E7900-E7909, 2017. https://doi.org/10.1073/pnas.1702247114
3. [3] A. S. Cowen, H. A. Elfenbein, P. Laukka, and D. Keltner, “Mapping 24 emotions conveyed by brief human vocalization,” American Psychologist, Vol.74, No.6, pp. 698-712, 2019. https://doi.org/10.1037/amp0000399
4. [4] A. S. Cowen and D. Keltner, “What the face displays: Mapping 28 emotions conveyed by naturalistic expression,” American Psychologist, Vol.75, No.3, pp. 349-364, 2020. https://doi.org/10.1037/amp0000488
5. [5] A. G. Huth, W. A. de Heer, T. L. Griffiths, F. E. Theunissen, and J. L. Gallant, “Natural speech reveals the semantic maps that tile human cerebral cortex,” Nature, Vol.532, No.7600, pp. 453-458, 2016. https://doi.org/10.1038/nature17637
6. [6] T. Naselaris, K. N. Kay, S. Nishimoto, and J. Gallant, “Encoding and decoding in fMRI,” NeuroImage, Vol.56, No.2, pp. 400-410, 2011. https://doi.org/10.1016/j.neuroimage.2010.07.073
7. [7] N. Koide-Majima, T. Nakai, and S. Nishimoto, “Distinct dimensions of emotion in the human brain and their representation on the cortical surface,” NeuroImage, Vol.222, Article No.117258, 2020. https://doi.org/10.1016/j.neuroimage.2020.117258.
8. [8] L. A. Wallenwein, S. N. L. Schmidt, J. Hass, and D. Mier, “Cross-modal decoding of emotional expressions in fMRI—Cross-session and cross-sample replication,” Imaging Neuroscience, Vol.2, Article No.imag-2-00289, 2024. https://doi.org/10.1162/imag_a_00289
9. [9] S. R. Batten, A. E. Hartle, L. S. Barbosa, B. Hadj-Amar, D. Bang, N. Melville, T. Twomey, J. P. White, A. Torres, X. Celaya et al., “Emotional words evoke region- and valence-specific patterns of concurrent neuromodulator release in human thalamus and cortex,” Cell Reports, Vol.44, No.1, Article No.115162, 2025. https://doi.org/10.1016/j.celrep.2024.115162
10. [10] R. Plutchik, “The emotions,” University Press of America, 1991.
11. [11] J. A. Russell and L. F. Barrett, “Core affect, prototypical emotional episodes, and other things called emotion: Dissecting the elephant,” J. of Personality and Social Psychology, Vol.76, No.5, pp. 805-819, 1999. https://doi.org/10.1037/0022-3514.76.5.805
12. [12] A. C. Weidman, C. M. Steckler, and J. L. Tracy, “The jingle and jangle of emotion assessment: Imprecise measurement, casual scale usage, and conceptual fuzziness in emotion research,” Emotion, Vol.17, No.2, pp. 267-295, 2017. https://doi.org/10.1037/emo0000226
13. [13] J. Phang, T. Févry, and S. R. Bowman, “Sentence encoders on stilts: Supplementary training on intermediate labeled-data tasks,” arXiv preprint, arXiv:1811.01088, 2018. https://doi.org/10.48550/arXiv.1811.01088
14. [14] V. Kocijan, A.-M. Cretu, O.-M. Camburu, Y. Yordanov, and T. Lukasiewicz, “A surprisingly robust trick for winograd schema challenge,” arXiv preprint, arXiv:1905.06290, 2019. https://doi.org/10.48550/arXiv.1905.06290
15. [15] S. Bhattasali, J. Brennan, W.-M. Luh, B. Franzluebbers, and J. Hale, “The alice datasets: fMRI & EEG observations of natural language comprehension,” Proc. of the Twelfth Language Resources and Evaluation Conf., pp. 120-125, 2020.
16. [16] J. Li, S. Bhattasali, S. Zhang, B. Franzluebbers, W.-M. Luh, R. N. Spreng, J. R. Brennan, Y. Yang, C. Pallier, and J. Hale, “Le petit prince multilingual naturalistic fMRI corpus,” Scientific data, Vol.9, No.1, Article No.530, 2022. https://doi.org/10.1038/s41597-022-01625-7
17. [17] C. Caucheteux, A. Gramfort, and J.-R. King, “GPT-2’s activations predict the degree of semantic comprehension in the human brain,” bioRxiv, 2021. https://doi.org/10.1101/2021.04.20.440622.
18. [18] E. T. Rolls, W. Cheng, and J. Feng, “The orbitofrontal cortex: Reward, emotion and depression,” Brain Communications, Vol.2, No.2, Article No.fcaa196, 2020. https://doi.org/10.1093/braincomms/fcaa196
19. [19] M. A. J. Apps, M. F. S. Rushworth, and S. W. C. Chang, “The anterior cingulate gyrus and social cognition: Tracking the motivation of others,” Neuron, Vol.90, No.4, pp. 692-707, 2016. https://doi.org/10.1016/j.neuron.2016.04.018
20. [20] E. T. Rolls, “Emotion, motivation, decision-making, the orbitofrontal cortex, anterior cingulate cortex, and the amygdala,” Brain Structure and Function, Vol.228, No.5, pp. 1201-1257, 2023. https://doi.org/10.1007/s00429-023-02644-9
21. [21] I. Fried, C. L. Wilson, K. A. MacDonald, and E. J. Behnke, “Electric current stimulates laughter,” Nature, Vol.391, No.6668, p. 650, 1998. https://doi.org/10.1038/35536
22. [22] M. Belyk, S. Brown, J. Lim, and S. A. Kotz, “Convergence of semantics and emotional expression within the IFG pars orbitalis,” NeuroImage, Vol.156, pp. 240-248, 2017. https://doi.org/10.1016/j.neuroimage.2017.04.020
23. [23] S. A. Abdel-Ghaffar, A. G. Huth, M. D. Lescroart, D. Stansbury, J. L. Gallant, and S. J. Bishop, “Occipital-temporal cortical tuning to semantic and affective features of natural images predicts associated behavioral responses,” Nature Communications, Vol.15, No.1, Article No.5531, 2024. https://doi.org/10.1038/s41467-024-49073-8
24. [24] C. S. Carver and E. Harmon-Jones, “Anger is an approach-related affect: Evidence and implications,” Psychological Bulletin, Vol.135, No.2, pp. 183-204, 2009. https://doi.org/10.1037/a0013965
25. [25] D. Mobbs, P. Petrovic, J. L. Marchant, D. Hassabis, N. Weiskopf, B. Seymour, R. J. Dolan, and C. D. Frith, “When fear is near: Threat imminence elicits prefrontal-periaqueductal gray shifts in humans,” Science, Vol.317, No.5841, pp. 1079-1083, 2007. https://doi.org/10.1126/science.1144298
26. [26] D. W. Grupe and J. B. Nitschke, “Uncertainty and anticipation in anxiety: An integrated neurobiological and psychological perspective,” Nature Reviews Neuroscience, Vol.14, No.7, pp. 488-501, 2013. https://doi.org/10.1038/nrn3524
27. [27] P. Kuppens, Z. Oravecz, and F. Tuerlinckx, “Feelings change: Accounting for individual differences in the temporal dynamics of affect,” J. of Personality and Social Psychology, Vol.99, No.6, pp. 1042-1060, 2010. https://doi.org/10.1037/a0020962
28. [28] D. Keltner and J. Haidt, “Approaching awe, a moral, spiritual, and aesthetic emotion,” Cognition and Emotion, Vol.17, No.2, pp. 297-314, 2003. https://doi.org/10.1080/02699930302297
29. [29] G. D. Sherman and J. Haidt, “Cuteness and disgust: The humanizing and dehumanizing effects of emotion,” Emotion Review, Vol.3, No.3, pp. 245-251, 2011. https://doi.org/10.1177/1754073911402396
30. [30] L. F. Barrett, “Solving the emotion paradox: Categorization and the experience of emotion,” Personality and Social Psychology Review, Vol.10, No.1, pp. 20-46, 2006. https://doi.org/10.1207/s15327957pspr1001_2
31. [31] K. A. Lindquist, T. D. Wager, H. Kober, E. Bliss-Moreau, and L. F. Barrett, “The brain basis of emotion: A meta-analytic review,” Behavioral and Brain Sciences, Vol.35, No.3, pp. 121-143, 2012. https://doi.org/10.1017/S0140525X11000446
32. [32] Y. Luo and I. Kobayashi, “Brainlm: Enhancing brain encoding and decoding capabilities with applications in multilingual learning,” J. Adv. Comput. Intell. Intell. Inform., Vol.29, No.4, pp. 754-767, 2025. https://doi.org/10.20965/jaciii.2025.p0754