Deep learning has achieved significant advancements in natural language processing. However, applying these methods to languages with complex morphological and syntactic structures—such as Russian—remains challenging. To address these challenges, this paper presents an optimized sentiment analysis model, GNN–BERT–AE, specifically designed for the Russian language. The model integrates graph neural networks (GNNs) with the contextualized embeddings of bidirectional encoder representations from transformers (BERT), enabling it to capture both syntactic dependencies and nuanced semantic information inherent in the Russian language. Whereas GNN excels in modeling the intricate word dependencies within the language, the contextualized representations of BERT provide a deep understanding of the text, improving the ability of the model to accurately interpret sentiments. The model further incorporates traditional feature extraction techniques—bag of words and term frequency–inverse document frequency—to preprocess text and emphasize critical features for sentiment analysis. To further enhance these features, a self-encoder clustering algorithm is employed, enabling the identification of latent patterns and improving the sensitivity of the model to subtle sentiment variations. The final phase of the model involves sentiment classification, categorizing emotions based on the enriched feature set. Experimental results showed that the GNN–BERT–AE model outperformed existing models—CNN–Transformer, RNN–LSTM–GRU, and Text–BiLSTM–CNN—on Russian social media datasets, achieving 1.25% to 3.1% accuracy improvements. These results highlight the robustness of the model and its significant potential for advancing sentiment analysis in the Russian language, particularly in handling complex linguistic features.

1. [1] S. Divya R., A. K. Desai, and V. Dave, “Artificial intelligence for human learning & behaviour change,” Int. J. of Advanced Science and Computer Applications, Vol.4, No.2, 2025. https://doi.org/10.47679/ijasca.v4i2.68
2. [2] N. Nosiel, S. Andriyanto, and M. S. Hasibuan, “Application of nave Bayes algorithm for SMS spam classification using orange,” Int. J. of Advanced Science and Computer Applications, Vol.1, No.1, pp. 16-24, 2022. https://doi.org/10.47679/ijasca.v1i1.5
3. [3] M. Rabi and M. Amrouche, “Convolutional Arabic handwriting recognition system based BLSTM-CTC using WBS decoder,” Int. J. of Advanced Science and Computer Applications, Vol.4, No.1, 2024. https://doi.org/10.47679/ijasca.v3i2.52
4. [4] H.-A. Goh, C.-K. Ho, and F. S. Abas, “Front-end deep learning web apps development and deployment: A review,” Applied Intelligence, Vol.53, No.12, pp. 15923-15945, 2023. https://doi.org/10.1007/s10489-022-04278-6
5. [5] F. Qiu et al., “Predicting students’ performance in e-learning using learning process and behaviour data,” Scientific Reports, Vol.12, Article No.453, 2022. https://doi.org/10.1038/s41598-021-03867-8
6. [6] A. A. Movassagh et al., “Artificial neural networks training algorithm integrating invasive weed optimization with differential evolutionary model,” J. of Ambient Intelligence and Humanized Computing, Vol.14, No.5, pp. 6017-6025, 2023. https://doi.org/10.1007/s12652-020-02623-6
7. [7] I. Kansizoglou, L. Bampis, and A. Gasteratos, “An active learning paradigm for online audio-visual emotion recognition,” IEEE Trans. on Affective Computing, Vol.13, No.2, pp. 756-768, 2022. https://doi.org/10.1109/TAFFC.2019.2961089
8. [8] E. Hsu, I. Malagaris, Y.-F. Kuo, R. Sultana, and K. Roberts, “Deep learning-based NLP data pipeline for EHR-scanned document information extraction,” JAMIA Open, Vol.5, No.2, Article No.00ac045, 2022. https://doi.org/10.1093/jamiaopen/ooac045
9. [9] D. Khurana, A. Koli, K. Khatter, and S. Singh, “Natural language processing: State of the art, current trends and challenges,” Multimedia Tools and Applications, Vol.82, No.3, pp. 3713-3744, 2023. https://doi.org/10.1007/s11042-022-13428-4
10. [10] D. P. Singh and B. Kaushik, “A systematic literature review for the prediction of anticancer drug response using various machine-learning and deep-learning techniques,” Chemical Biology & Drug Design, Vol.101, No.1, pp. 175-194, 2023. https://doi.org/10.1111/cbdd.14164
11. [11] M. F. Bashir et al., “Context-aware emotion detection from low-resource Urdu language using deep neural network,” ACM Trans. on Asian and Low-Resource Language Information Processing, Vol.22, No.5, Article No.131, 2023. https://doi.org/10.1145/3528576
12. [12] D. Ruan, J. Wang, J. Yan, and C. Gühmann, “CNN parameter design based on fault signal analysis and its application in bearing fault diagnosis,” Advanced Engineering Informatics, Vol.55, Article No.101877, 2023. https://doi.org/10.1016/j.aei.2023.101877
13. [13] M. Méndez, M. G. Merayo, and M. Núñez, “Long-term traffic flow forecasting using a hybrid CNN-BiLSTM model,” Engineering Applications of Artificial Intelligence, Vol.121, Article No.106041, 2023. https://doi.org/10.1016/j.engappai.2023.106041
14. [14] G. Brauwers and F. Frasincar, “A general survey on attention mechanisms in deep learning,” IEEE Trans. on Knowledge and Data Engineering, Vol.35, No.4, pp. 3279-3298, 2023. https://doi.org/10.1109/TKDE.2021.3126456
15. [15] P. Ray and A. Chakrabarti, “A mixed approach of deep learning method and rule-based method to improve aspect level sentiment analysis,” Applied Computing and Informatics, Vol.18, Nos.1-2, pp. 163-178, 2022. https://doi.org/10.1016/j.aci.2019.02.002
16. [16] P. Dominic et al., “Multilingual sentiment analysis using deep-learning architectures,” 5th Int. Conf. on Smart Systems and Inventive Technology, pp. 1077-1083, 2023. https://doi.org/10.1109/ICSSIT55814.2023.10060993
17. [17] V. Moshkin, A. Konstantinov, N. Yarushkina, and A. Dyrnochkin, “Comparison of different neural networks in sentiment analysis of social media data,” 2021 Int. Conf. on Information Technology and Nanotechnology, 2021. https://doi.org/10.1109/ITNT52450.2021.9649048
18. [18] A. I. Kanev, G. A. Savchenko, I. A. Grishin, D. A. Vasiliev, and E. M. Duma, “Sentiment analysis of multilingual texts using machine learning methods,” 2022 Conf. of Russian Young Researchers in Electrical and Electronic Engineering, pp. 326-331, 2022. https://doi.org/10.1109/ElConRus54750.2022.9755568
19. [19] D. S. Bakanov and A. V. Kupriyanov, “Designing an algorithm for annotating Russian-language text data of social media using transfer learning,” IX Int. Conf. on Information Technology and Nanotechnology, 2023. https://doi.org/10.1109/ITNT57377.2023.10139023
20. [20] J. Cheng, M. Sadiq, O. A. Kalugina, S. A. Nafees, and Q. Umer, “Convolutional neural network based approval prediction of enhancement reports,” IEEE Access, Vol.9, pp. 122412-122424, 2021. https://doi.org/10.1109/ACCESS.2021.3108624
21. [21] Y. HaCohen-Kerner, “Survey on profiling age and gender of text authors,” Expert Systems with Applications, Vol.199, Article No.117140, 2022. https://doi.org/10.1016/j.eswa.2022.117140
22. [22] D. Zhukov and J. Perova, “A model for analyzing user moods of self-organizing social network structures based on graph theory and the use of neural networks,” 3rd Int. Conf. on Control Systems, Mathematical Modeling, Automation and Energy Efficiency, pp. 319-322, 2021. https://doi.org/10.1109/SUMMA53307.2021.9632203
23. [23] M. Grootendorst, “BERTopic: Neural topic modeling with a class-based TF-IDF procedure,” arXiv:2203.05794, 2022. https://doi.org/10.48550/arXiv.2203.05794
24. [24] L. Phan et al., “CoTexT: Multi-task learning with code-text transformer,” arXiv:2105.08645, 2021. https://doi.org/10.48550/arXiv.2105.08645
25. [25] L. Wang, W. Wang, and X. Cheng, “Bimodal emotion recognition model for speech-text based on Bi-LSTM-CNN,” Computer Engineering and Applications, Vol.58, No.4, pp. 192-197, 2022 (in Chinese).
26. [26] M. Rafiepour and J. S. Sartakhti, “CTRAN: CNN-Transformer-based network for natural language understanding,” Engineering Applications of Artificial Intelligence, Vol.126, Part C, Article No.107013, 2023. https://doi.org/10.1016/j.engappai.2023.107013
27. [27] Z. Wang, S. Kim, and I. Joe, “An improved LSTM-based failure classification model for financial companies using natural language processing,” Applied Sciences, Vol.13, No.13, Article No.7884, 2023. https://doi.org/10.3390/app13137884
28. [28] A. He and M. Abisado, “Text sentiment analysis of Douban film short comments based on BERT-CNN-BiLSTM-Att model,” IEEE Access, Vol.12, pp. 45229-45237, 2024. https://doi.org/10.1109/ACCESS.2024.3381515
29. [29] X. Bi and T. Zhang, “Pedagogical sentiment analysis based on the BERT-CNN-BiGRU-attention model in the context of intercultural communication barriers,” PeerJ Computer Science, Vol.10, Article No.e2166, 2024. https://doi.org/10.7717/peerj-cs.2166