Research Paper:
Electroencephalography Emotion Recognition Based on Rhythm Information Entropy Extraction
Zhen-Tao Liu*1,*2,*3, , Xin Xu*1,*2,*3, Jinhua She*4 , Zhaohui Yang*5, and Dan Chen*1,*2,*3
*1School of Automation, China University of Geosciences
No.388 Lumo Road, Hongshan District, Wuhan, Hubei 430074, China
*2Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems
No.388 Lumo Road, Hongshan District, Wuhan, Hubei 430074, China
*3Engineering Research Center of Intelligent Technology for Geo-Exploration, Ministry of Education
No.388 Lumo Road, Hongshan District, Wuhan, Hubei 430074, China
*4School of Engineering, Tokyo University of Technology
1404-1 Katakura, Hachioji, Tokyo 192-0982, Japan
*5Department of Rehabilitation, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology
No.1277 Jiefang Road, Jianghan District, Wuhan, Hubei 430022, China
Corresponding author
Electroencephalography (EEG) is a physiological signal directly generated by the central nervous system. Brain rhythm is closely related to a person’s emotional state and is widely used for EEG emotion recognition. In previous studies, the rhythm specificity between different brain channels was seldom explored. In this paper, the rhythm specificity of brain channels is studied to improve the accuracy of EEG emotion recognition. Variational mode decomposition is used to decompose rhythm signals and enhance features, and two kinds of information entropy, i.e., differential entropy (DE) and dispersion entropy (DispEn) are extracted. The rhythm being used to get the best result of single channel emotion recognition is selected as the representative rhythm, and the remove one method is employed to obtain rhythm information entropy feature. In the experiment, the DEAP database was used for EEG emotion recognition in valence-arousal space. The results showed that the best result of rhythm DE feature classification in the valence dimension is 77.04%, and the best result of rhythm DispEn feature classification in the arousal dimension is 79.25%.
- [1] X. Li et al., “Exploring EEG features in cross-subject emotion recognition,” Frontiers in Neuroscience, Vol.12, Article No.162, 2018. https://doi.org/10.3389/fnins.2018.00162
- [2] A. Iyer, S. S. Das, R. Teotia, S. Maheshwari, and R. R. Sharma, “CNN and LSTM based ensemble learning for human emotion recognition using EEG recordings,” Multimedia Tools and Applications, Vol.82, No.4, pp. 4883-4896, 2023. https://doi.org/10.1007/s11042-022-12310-7
- [3] D. Li, L. Xie, Z. Wang, and H. Yang, “Brain emotion perception inspired EEG emotion recognition with deep reinforcement learning,” IEEE Trans. on Neural Networks and Learning Systems, 2023. https://doi.org/10.1109/tnnls.2023.3265730
- [4] T. Alotaiby, F. E. A. El-Samie, S. A. Alshebelli, and I. Ahmad, “A review of channel selection algorithms for EEG signal processing,” EURASIP J. on Advances in Signal Processing, Vol.2015, Article No.66, 2015. https://doi.org/10.1186/s13634-015-0251-9
- [5] A. Al-Nafjan, M. Hosny, Y. Al-Ohali, and A. Al-Wabil, “Review and classification of emotion recognition based on EEG brain-computer interface system research: A systematic review,” Applied Sciences, Vol.7, No.12, Article No.1239, 2017. https://doi.org/10.3390/app7121239
- [6] M. Niknazar, S. R. Mousavi, B. V. Vahdat, and M. Sayyah, “A new framework based on recurrence quantification analysis for epileptic seizure detection,” IEEE J. of Biomedical and Health Informatics, Vol.17, No.3, pp. 572-578, 2013. https://doi.org/10.1109/JBHI.2013.2255132
- [7] S.-J. Choi and B.-G. Kang, “Prototype design and implementation of an automatic control system based on a BCI,” Wireless Personal Communications, Vol.79, No.4, pp. 2551-2563, 2014. https://doi.org/10.1007/s11277-014-1861-5
- [8] W. Zheng, “Multichannel EEG-based emotion recognition via group sparse canonical correlation analysis,” IEEE Trans. on Cognitive and Developmental Systems, Vol.9, No.3, pp. 281-290, 2017. https://doi.org/10.1109/TCDS.2016.2587290
- [9] M. L. R. Menezes et al., “Towards emotion recognition for virtual environments: An evaluation of EEG features on benchmark dataset,” Personal and Ubiquitous Computing, Vol.21, No.6, pp. 1003-1013, 2017. https://doi.org/10.1007/s00779-017-1072-7
- [10] K. S. Kamble and J. Sengupta, “EVNCERS: An integrated eigenvector centrality-variational nonlinear chirp mode decomposition-based EEG rhythm separation for automatic emotion recognition,” IEEE Sensors J., Vol.23, No.18, pp. 21661-21669, 2023. https://doi.org/10.1109/JSEN.2023.3304891
- [11] J. W. Li, S. Barma, S. H. Pun, M. I. Vai, and P. U. Mak, “Emotion recognition based on EEG brain rhythm sequencing technique,” IEEE Trans. on Cognitive and Developmental Systems, Vol.15, No.1, pp. 163-174, 2023. https://doi.org/10.1109/TCDS.2022.3149953
- [12] G. Rodríguez-Bermúdez and P. J. García-Laencina, “Analysis of EEG signals using nonlinear dynamics and chaos: A review,” Applied Mathematics & Information Sciences, Vol.9, No.5, pp. 2309-2321, 2015. https://doi.org/10.12785/amis/090512
- [13] Y. Lu et al., “Entropy-based pattern learning based on singular spectrum analysis components for assessment of physiological signals,” Complexity, Vol.2020, Article No.4625218, 2020. https://doi.org/10.1155/2020/4625218
- [14] N. Zhuang et al., “Emotion recognition from EEG signals using multidimensional information in EMD domain,” BioMed Research Int., Vol.2017, Article No.8317357, 2017. https://doi.org/10.1155/2017/8317357
- [15] Y. Zhang, X. Ji, and S. Zhang, “An approach to EEG-based emotion recognition using combined feature extraction method,” Neuroscience Letters, Vol.633, pp. 152-157, 2016. https://doi.org/10.1016/j.neulet.2016.09.037
- [16] A. Mert and A. Akan, “Emotion recognition from EEG signals by using multivariate empirical mode decomposition,” Pattern Analysis and Applications, Vol.21, No.1, pp. 81-89, 2018. https://doi.org/10.1007/s10044-016-0567-6
- [17] Z. Wu and N. E. Huang, “Ensemble empirical mode decomposition: A noise-assisted data analysis method,” Advances in Adaptive Data Analysis, Vol.1, No.1, pp. 1-41, 2009. https://doi.org/10.1142/S1793536909000047
- [18] J.-R. Yeh, J.-S. Shieh, and N. E. Huang, “Complementary ensemble empirical mode decomposition: A novel noise data analysis method,” Advances in Adaptive Data Analysis, Vol.2, No.2, pp. 135-156, 2010. https://doi.org/10.1142/S1793536910000422
- [19] M. A. Colominas, G. Schlotthauer, and M. E. Torres, “Improved complete ensemble EMD: A suitable tool for biomedical signal processing,” Biomedical Signal Processing and Control, Vol.14, pp. 19-29, 2014. https://doi.org/10.1016/j.bspc.2014.06.009
- [20] M. Zhang, B. Hu, X. Zheng, and T. Li, “A novel multidimensional feature extraction method based on VMD and WPD for emotion recognition,” 2020 IEEE Int. Conf. on Bioinformatics and Biomedicine (BIBM), pp. 1216-1220, 2020. https://doi.org/10.1109/BIBM49941.2020.9313220
- [21] P. Pandey and K. R. Seeja, “Subject independent emotion recognition from EEG using VMD and deep learning,” J. of King Saud University – Computer and Information Sciences, Vol.34, No.5, pp. 1730-1738, 2022. https://doi.org/10.1016/j.jksuci.2019.11.003
- [22] S. K. Khare and V. Bajaj, “An evolutionary optimized variational mode decomposition for emotion recognition,” IEEE Sensors J., Vol.21, No.2, pp. 2035-2042, 2021. https://doi.org/10.1109/JSEN.2020.3020915
- [23] W.-L. Chu, M.-W. Huang, B.-L. Jian, and K.-S. Cheng, “Analysis of EEG entropy during visual evocation of emotion in schizophrenia,” Annals of General Psychiatry, Vol.16, Article No.34, 2017. https://doi.org/10.1186/s12991-017-0157-z
- [24] J. S. Richman and J. R. Moorman, “Physiological time-series analysis using approximate entropy and sample entropy,” American J. of Physiology – Heart and Circulatory Physiology, Vol.278, No.6, pp. H2039-H2049, 2000. https://doi.org/10.1152/ajpheart.2000.278.6.H2039
- [25] C. Bandt and B. Pompe, “Permutation entropy: A natural complexity measure for time series,” Physical Review Letters, Vol.88, No.17, Article No.174102, 2002. https://doi.org/10.1103/PhysRevLett.88.174102
- [26] Y. Xia et al., “Application of permutation entropy and permutation min-entropy in multiple emotional states analysis of RRI time series,” Entropy, Vol.20, No.3, Article No.148, 2018. https://doi.org/10.3390/e20030148
- [27] K. Tung, P.-K. Liu, Y.-C. Chuang, S.-H. Wang, and A.-Y. A. Wu, “Entropy-assisted multi-modal emotion recognition framework based on physiological signals,” 2018 IEEE-EMBS Conf. on Biomedical Engineering and Sciences (IECBES), pp. 22-26, 2018. https://doi.org/10.1109/IECBES.2018.8626634
- [28] H. Azami, A. Fernández, and J. Escudero, “Refined multiscale fuzzy entropy based on standard deviation for biomedical signal analysis,” Medical & Biological Engineering & Computing, Vol.55, No.11, pp. 2037-2052, 2017. https://doi.org/10.1007/s11517-017-1647-5
- [29] B. Fadlallah, B. Chen, A. Keil, and J. Príncipe, “Weighted-permutation entropy: A complexity measure for time series incorporating amplitude information,” Physical Review E, Vol.87, No.2, Article No.022911, 2013. https://doi.org/10.1103/physreve.87.022911
- [30] H. Azami et al., “Multiscale fluctuation-based dispersion entropy and its applications to neurological diseases,” IEEE Access, Vol.7, pp. 68748-68733, 2019. https://doi.org/10.1109/ACCESS.2019.2918560
- [31] X.-W. Ding, Z.-T. Liu, D.-Y. Li, Y. He, and M. Wu, “Electroencephalogram emotion recognition based on dispersion entropy feature extraction using random over-sampling imbalanced data processing,” IEEE Trans. on Cognitive and Developmental Systems, Vol.14, No.3, pp. 882-891, 2022. https://doi.org/10.1109/TCDS.2021.3074811
- [32] S. Liu et al., “Subject-independent emotion recognition of EEG signals based on dynamic empirical convolutional neural network,” IEEE/ACM Trans. on Computational Biology and Bioinformatics, Vol.18, No.5, pp. 1710-1721, 2020. https://doi.org/10.1109/TCBB.2020.3018137
- [33] H. Li, C. Qing, X. Xu, and T. Zhang, “A novel DE-PCCM feature for EEG-based emotion recognition,” 2017 Int. Conf. on Security, Pattern Analysis, and Cybernetics (SPAC), pp. 389-393, 2017. https://doi.org/10.1109/SPAC.2017.8304310
- [34] V. M. Joshi and R. B. Ghongade, “Optimal number of electrode selections for EEG-based emotion recognition using linear formulation of differential entropy,” Biomedical and Pharmacology J., Vol.13, No.2, pp. 645-653, 2020. https://doi.org/10.13005/bpj/1928
- [35] Y.-J. Liu et al., “Real-time movie-induced discrete emotion recognition from EEG signals,” IEEE Trans. on Affective Computing, Vol.9, No.4, pp. 550-562, 2017. https://doi.org/10.1109/TAFFC.2017.2660485
- [36] S.-J. Hu, Z.-T. Liu, and X.-W. Ding, “Electroencephalogram emotion recognition using variational modal decomposition based dispersion entropy feature extraction,” 2021 40th Chinese Control Conf. (CCC), pp. 3323-3326, 2021. https://doi.org/10.23919/CCC52363.2021.9549884
- [37] J. Lian, Z. Liu, H. Wang, and X. Dong, “Adaptive variational mode decomposition method for signal processing based on mode Characteristic,” Mechanical Systems and Signal Processing, Vol.107, pp. 53-77, 2018. https://doi.org/10.1016/j.ymssp.2018.01.019
- [38] V. Gupta, M. D. Chopda, and R. B. Pachori, “Cross-subject emotion recognition using flexible analytic wavelet transform from EEG signals,” IEEE Sensors J., Vol.19, No.6, pp. 2266-2274, 2018. https://doi.org/10.1109/JSEN.2018.2883497
- [39] S. Koelstra et al., “DEAP: A database for emotion analysis; using physiological signals,” IEEE Trans. on Affective Computing, Vol.3, No.1, pp. 18-31, 2012. https://doi.org/10.1109/T-AFFC.2011.15
- [40] P. Arnau-González, M. Arevalillo-Herráez, and N. Ramzan, “Fusing highly dimensional energy and connectivity features to identify affective states from EEG signals,” Neurocomputing, Vol.244, pp. 81-89, 2017. https://doi.org/10.1016/j.neucom.2017.03.027
- [41] Y. Ding, N. Robinson, S. Zhang, Q. Zeng, and C. Guan, “TSception: Capturing temporal dynamics and spatial asymmetry from EEG for emotion recognition,” IEEE Trans. on Affective Computing, Vol.14, No.3, pp. 2238-2250, 2023. https://doi.org/10.1109/TAFFC.2022.3169001
- [42] C. Li, B. Chen, Z. Zhao, N. Cummins, and B. W. Schuller, “Hierarchical attention-based temporal convolutional networks for EEG-based emotion recognition,” 2021 IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), pp. 1240-1244, 2021. https://doi.org/10.1109/ICASSP39728.2021.9413635
- [43] N. Zhuang et al., “Emotion recognition from EEG signals using multidimensional information in EMD domain,” BioMed Research Int., Vol.2017, Article No.8317357, 2017. https://doi.org/10.1155/2017/8317357
- [44] Z.-T. Liu, S.-J. Hu, J. She, Z. Yang, and X. Xu, “Electroencephalogram Emotion recognition using combined features in variational mode decomposition domain,” IEEE Trans. on Cognitive and Developmental Systems, Vol.15, No.3, pp. 1595-1604, 2023. https://doi.org/10.1109/TCDS.2022.3233858
- [45] Y. Huang, J. Yang, S. Liu, and J. Pan, “Combining facial expressions and electroencephalography to enhance emotion recognition,” Future Internet, Vol.11, No.5, Article No.105, 2019. https://doi.org/10.3390/fi11050105
- [46] Z.-M. Wang, S.-Y. Hu, and H. Song, “Channel selection method for EEG emotion recognition using normalized mutual information,” IEEE Access, Vol.7, pp. 143303-143311, 2019. https://doi.org/10.1109/ACCESS.2019.2944273
- [47] Q. Li, Y. Song, J. Zhang, and V. S. Sheng, “Multiclass imbalanced learning with one-versus-one decomposition and spectral clustering,” Expert Systems with Applications, Vol.147, Article No.113152, 2020. https://doi.org/10.1016/j.eswa.2019.113152
- [48] H. He and E. A. Garcia, “Learning from imbalanced data,” IEEE Trans. on Knowledge and Data Engineering, Vol.21, No.9, pp. 1263-1284, 2009. https://doi.org/10.1109/TKDE.2008.239
- [49] W. Lu, H. Hou, and J. Chu, “Feature fusion for imbalanced ECG data analysis,” Biomedical Signal Processing and Control, Vol.41, pp. 152-160, 2018. https://doi.org/10.1016/j.bspc.2017.11.010
This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.