Top > JACIII > Fine-Grained Sentiment Recognition Model for English Writing T ...

single-jc.php

JACIII Vol.30 No.4 pp. 1150-1162
(2026)
doi: 10.20965/jaciii.2026.p1150

Research Paper:

Views over last 60 days: 228

Fine-Grained Sentiment Recognition Model for English Writing Texts Based on BERT-CNN-BiGRU

Zhigang Peng† ORCID Icon

Jingdezhen University
No.3 Fuliang Avenue, Fuliang County, Jingdezhen, Jiangxi 333000, China

†Corresponding author

Received:
January 7, 2026
Accepted:
February 20, 2026
Published:
July 20, 2026
Creative Commons License
Keywords:
BERT, BiGRU, CNN, English writing, fine-grained emotion
Abstract

English writing texts often show complex semantic layers, implicit emotional expression, and strong temporal dependence, which leads to limitations in semantic modeling and restricted feature extraction in existing methods. To address this issue, the study constructed a fine-grained sentiment recognition model that integrates bidirectional encoder representations from transformers (BERT), bidirectional gated recurrent unit (BiGRU), convolutional neural network (CNN), and an attention mechanism. BERT was used to generate context-aware semantic representations and improve overall semantic understanding of the text. BiGRU was applied to capture bidirectional temporal dependencies and describe the dynamic evolution of emotions in discourse. CNN was employed to extract phrase-level local emotional features and enhance the detection of emotion-triggering segments. The attention mechanism was introduced to highlight key emotional information and improve feature discriminability. On this basis, a gating fusion strategy was used to dynamically integrate multi-source features, and a multi-task learning framework was incorporated. The model performed emotion intensity prediction while conducting multi-class emotion classification. In this way, fine-grained sentiment modeling was achieved from both category and intensity perspectives. The results indicate that the proposed model achieves excellent performance across several datasets, with overall accuracy and F1-score remaining above 91%, ROC-AUC reaching up to 0.944, and recognition rates for all emotion categories staying above 84%, clearly outperforming existing mainstream approaches. Ablation results demonstrated that all six modules contributed to performance improvement. On the SST-2 and IMDB datasets, the complete model achieved an accuracy of 93% and an AUC of 0.97. In summary, this research offers a compact, stable, and adaptable solution for emotion modeling in English texts, with both theoretical and practical value.

Close full text
Cite this article as:
Z. Peng, “Fine-Grained Sentiment Recognition Model for English Writing Texts Based on BERT-CNN-BiGRU,” J. Adv. Comput. Intell. Intell. Inform., Vol.30 No.4, pp. 1150-1162, 2026.
Data files:

1. Introduction

With the rapid growth of artificial intelligence and natural language processing, emotion recognition research has gradually extended from social media and review texts to the field of education 1. English writing texts, as structured language with rich semantic hierarchy, contain implicit emotional information and play an important role in intelligent writing evaluation and emotional intervention in learning 2. In recent years, sentiment analysis has evolved rapidly from traditional machine learning to deep learning methods 3. Early approaches mainly relied on handcrafted features and sentiment lexicons, which made it difficult to capture complex semantic dependencies and contextual relationships 4. Later, deep learning models such as convolutional neural network (CNN) and recurrent neural networks were widely applied to extract semantic and temporal features related to emotion 5,6. However, although CNN and RNN-based models have shown effectiveness in sentiment analysis, they still present structural limitations. CNN relies on convolution kernels with fixed window sizes for local feature extraction. Its receptive field mainly focuses on phrase-level semantics and cannot fully capture global semantic information across sentences or paragraphs. Meanwhile, RNN and its variants (such as long short-term memory (LSTM) and gated recurrent unit (GRU)) can model sequential dependencies. Yet they are prone to gradient decay and information compression when processing long texts. This issue gradually weakens long-distance dependency features and reduces the stability of long-range semantic interactions. In English writing texts, emotional expression often involves cross-sentence relations, semantic progression, or emotional transitions (e.g., although \(\dots\) but \(\dots\)). The sentiment polarity is therefore not determined by isolated local words. Instead, it depends on the overall discourse semantics. Consequently, models that rely only on local feature extraction or single-direction temporal modeling have difficulty capturing complex emotional structures in a fine-grained manner.

Recently, pre-trained language models such as bidirectional encoder representations from transformers (BERT) and robustly optimized BERT pretraining approach (RoBERTa) have greatly improved the ability of semantic representation, making contextual modeling in emotion recognition tasks more accurate 7. Meanwhile, the development of multi-task learning, attention mechanisms, and feature fusion strategies has provided new directions for fine-grained emotion modeling 8,9. Despite these advancements, emotion recognition in English writing still faces three major challenges. First, deep semantic hierarchies and implicit emotional expressions make it difficult to identify local emotional cues. Second, emotional changes depend on contextual dynamics, which are hard to capture through time-sequence modeling. Third, models often suffer from feature redundancy and information imbalance in multi-class emotion tasks.

To overcome the above limitations in global semantic modeling, long-range temporal dependency capture, and multi-class feature redundancy, this study proposed a fine-grained English writing sentiment recognition model that integrates BERT, CNN, bidirectional gated recurrent unit (BiGRU), an attention mechanism, and a multi-task learning framework. The model used BERT for contextual semantic representation, BiGRU to capture bidirectional temporal dependencies, and an attention mechanism to emphasize critical emotional segments. The CNN module extracted phrase-level local emotion features, while a gated fusion layer dynamically integrated multiple feature sources to suppress noise. The multi-task learning module jointly predicted emotion categories and intensity, achieving multi-level fusion of semantic, local, and temporal features.

The main innovations and contributions of this study are as follows:

  1. (1)

    A dual-channel structure combining BERT-based semantic modeling and BiGRU-Attention mechanisms was proposed to enhance contextual sensitivity in fine-grained emotion recognition.

  2. (2)

    A gated fusion layer was designed to dynamically balance global, local, and temporal features, improving the robustness of the model.

  3. (3)

    A multi-task learning framework was constructed to predict emotion category and intensity simultaneously.

  4. (4)

    The proposed method was evaluated on several public datasets, and the results showed that it achieved higher accuracy and generalization than mainstream models.

2. Related Work

2.1. Pre-Trained Semantic Modeling

In recent years, emotion analysis methods based on pre-trained language models have significantly improved text semantic representation and contextual understanding. Khadhraoui et al. introduced the BERT model, which captures contextual dependencies through a bidirectional Transformer and provides deep semantic features for emotion recognition 10. Gardazi et al. designed a multi-layer semantic attention network based on BERT for student composition emotion recognition and achieved better identification of implicit emotions 11. Nkhata et al. combined BERT with semantic contrastive learning to identify implicit emotions more accurately and improved generalization in long sentence structures 12. Tan et al. enhanced cross-sentence consistency in RoBERTa, enabling the model to maintain semantic coherence across longer texts 13. Although these studies made progress in semantic understanding, they were mostly limited to sentence-level modeling. They lacked the ability to capture dynamic emotional transitions and temporal patterns in paragraph-level emotion structures. Therefore, this study incorporated the BiGRU module into BERT-based semantic modeling to bridge the gap between semantic and temporal representation.

2.2. CNN-Based Local Feature Modeling

Local convolutional structures perform well in capturing phrase-level emotional cues. Chen et al. applied multi-scale convolution kernels to extract detailed local emotional and semantic features 14. Zhang et al. proposed a CNN-Transformer hybrid framework that combines local perception from convolution with global dependency modeling from the transformer for multi-layer emotion recognition in complex texts 15. Yang et al. introduced dynamic convolution, allowing kernels to adjust their receptive fields adaptively and improving model stability under varying contexts 16. Zhang et al. added a channel attention module to CNN layers to focus on emotion-relevant regions 17. Although these methods improved local feature representation, they often suffered from redundant information and lacked coordination with global semantics. To address this limitation, this study added a gated fusion mechanism after the CNN layer to dynamically balance local and global features and reduce the shortcomings of single-feature convolution in multi-context scenarios.

2.3. BiGRU-Based Temporal Modeling

The BiGRU effectively captures temporal dependencies in emotional expression and has shown strong performance in multi-turn dialogue and long-text modeling. Zhu et al. proposed a BiGRU-Attention model that integrates attention to strengthen the representation of key emotional fragments 18. Ma et al. developed a lightweight residual BiGRU that reduces parameter size while maintaining contextual coherence 19. Jin et al. designed a multi-scale temporal attention mechanism that enables BiGRU to capture long-range dependencies more effectively in lengthy texts 20. Na et al. employed a multi-channel BiGRU to track emotion evolution dynamically 21. However, traditional BiGRU models are mostly used for single-task classification and fail to integrate semantic, local, and global information comprehensively. They also experience information decay when modeling emotion drift in long texts. To overcome these issues, this study employed a BERT-BiGRU hybrid architecture to combine pre-trained semantic features with bidirectional temporal modeling, improving robustness in long-text and cross-sentence emotion recognition.

2.4. Attention Mechanism and Multi-Task Learning

The attention mechanism has been widely used for selecting key emotional features and helps models focus on important emotional cues and core semantics. Liu et al. proposed a hierarchical attention mechanism that distinguishes emotion changes within and across sentences, improving the detection of emotional shifts 22. Foggia et al. introduced a multi-task learning framework for English writing emotion recognition, achieving joint prediction of emotion category and intensity 23. Kocak et al. presented a gated attention fusion model that dynamically weighted multi-source features and enhanced stability in complex semantic conditions 24. Zhang et al. combined multi-task learning with regression to predict emotion intensity and enrich emotion representation 25. Although attention and multi-task learning improved discrimination and generalization, most fusion models still lacked a unified feature coordination strategy, which led to redundancy and gradient conflict. Based on these studies, this research developed a multi-task learning framework that combines attention and gated dynamic fusion to achieve coordinated optimization of semantic, temporal, and local features, producing a more robust fine-grained emotion recognition model.

3. Method

3.1. Overall Model Architecture

This study developed a deep learning model that integrates BERT, CNN, BiGRU, and an attention mechanism to achieve fine-grained sentiment recognition for English writing texts. In this work, “fine-grained sentiment” is defined as a joint modeling framework that includes multi-class emotion recognition and emotion intensity prediction. At the category level, emotion labels were divided into multiple detailed classes according to each dataset, which differs from traditional binary sentiment analysis. At the intensity level, an emotion intensity prediction subtask was introduced, where emotional strength was modeled as a continuous variable. Through this dual characterization of category and intensity, fine-grained modeling of emotional expression in English writing was achieved. The model adopted a serial integration strategy. It organically combined deep semantic encoding, local feature extraction, and bidirectional temporal modeling. The overall architecture illustrates the complete data flow from raw text input to the final emotion category output, as shown in Fig. 1.

The model mainly consisted of five functional layers:

  1. (1)

    BERT semantic encoding layer: The original English text was first tokenized. Position embeddings and segment embeddings were then added before being fed into the pretrained BERT model. The multi-head self-attention mechanism in the transformer captured deep bidirectional contextual information and produced semantic-rich word vector sequences.

  2. (2)

    CNN local feature extraction layer: The word vectors generated by BERT were passed to the convolution layer. Multiple convolution kernels with different sizes were used to capture local phrase patterns in the text. Max-pooling was then applied to select the most salient emotional trigger signals and form the local feature sequence.

  3. (3)

    BiGRU temporal modeling layer: The BiGRU layer consisted of forward and backward GRU networks. They processed the feature sequence in two opposite directions. This structure effectively captured long-distance dependencies in the text. The bidirectional hidden states were concatenated to form a representation containing global logic and temporal evolution information.

  4. (4)

    Attention mechanism layer: To further emphasize key emotional information that contributes to classification, an attention mechanism was introduced to aggregate the BiGRU outputs. Attention weights were calculated for the hidden states, followed by weighted summation. This process generated a context vector focused on important semantic targets.

  5. (5)

    Output classification layer: The final context vector was mapped into a high-dimensional space through a fully connected layer. A softmax activation function was then applied to output the probability distribution of fine-grained sentiment categories.

Through this modular and fully integrated design, the framework jointly modeled global semantics, local details, and temporal dependencies. As a result, the model improved the discrimination accuracy for implicit emotions in complex English essays.

figure

Fig. 1. Overall architecture of the fine-grained sentiment recognition model.

3.2. Semantic and Temporal Modeling Based on BERT-BiGRU with Attention Mechanism

Based on the overall architecture described in Section 3.1, the study first constructed a semantic and temporal modeling module using the BERT layer and the BiGRU layer. In English writing texts, emotional information often depends on the contextual connection between words and the temporal features of sentence structures. Emotional tendencies are not only found in individual words but also reflected in semantic flow and syntactic changes 26. Traditional sentiment analysis methods have difficulty capturing long-distance dependencies and deep semantic information, which limits their ability to understand and recognize hidden emotions in English writing 27. To achieve this goal, the module employed the BERT model to perform contextual semantic encoding of the input text and generated word vector representations with global context awareness. Then, the BERT output is passed into the BiGRU network to capture the bidirectional dependencies within the text sequence, which helps reveal the dynamic changes of emotion across the sentence. After the BiGRU layer, an attention mechanism is added to assign different weights to features at various positions, enabling the model to focus on words and phrases that are more important for emotional expression.

The BERT semantic modeling layer handles text input representation and preprocessing steps, segmenting and tokenizing the original English writing text \(T=\{w_1,w_2,\dots,w_n\}\) into the triplet format required by BERT, as shown in Eq. 1:

\begin{equation} \label{eq:1} H=\mathrm{BERT}(x,s,m), \end{equation}
where \(H=[h_1,h_2,\dots,h_n]\in \mathbb{R}^{n\times d}\) represents the token-level contextual representation output by BERT, \(n\) is the sequence length and \(d\) is the hidden dimension; \(x\) refers to the index sequence of words in the vocabulary; \(s\) indicates the segment identifiers (used to distinguish sentence pairs); and \(m\) is the attention mask used to ignore padding tokens. This BERT representation provides context-aware embeddings for each word and serves as the input foundation for the downstream sequence modeling and emotion recognition tasks. After preprocessing, the input text is passed into the BERT model. The model generates token embeddings using the transformer and multi-head attention mechanism, as shown in Eq. 2 28:
\begin{equation} \label{eq:2} \textrm{Attention}(Q,K,V)=\textrm{softmax}\left(\dfrac{\textit{QK}}{\sqrt{d_k}} \right)V, \end{equation}
where \(Q,K,V\in \mathbb{R}^{n\times d_k}\) denotes the query, key, and value matrices, respectively. \(d_k\) is the scaling factor used to stabilize training, and \(\textrm{softmax}(\cdot)\) is applied to calculate attention weights for each token. The token representations output by BERT are averaged to obtain the sentence-level representation vector, as defined in Eq. 3:
\begin{equation} \label{eq:3} h_{\mathrm{sentence}}=\dfrac{1}{n}\sum_{i=1}^n h_i, \end{equation}
where \(n\) denotes the sequence length, which is the total number of tokens (words or subwords) in the input text. \(h_{\textrm{sentence}}\in \mathbb{R}^d\) is the sentence-level vector representation, and \(h_i\) represents the contextual embedding of the \(i\)-th token from BERT output. A BiGRU network is then used for sequence modeling, with its structure shown in Fig. 2.
figure

Fig. 2. BiGRU network structure diagram.

Figure 2 illustrates the process of constructing bidirectional sequence modeling based on the BERT output using BiGRU. In the figure, \(h_{t}\) denotes the combined hidden state output of the bidirectional GRU at time step; \(\overrightarrow{h_t}\) and \(\overleftarrow{h_t}\) represent the hidden states of the forward and backward GRU units at time step \(t\); \(\overrightarrow{h_{t-1}}\) and \(\overrightarrow{h_{t+1}}\) indicate the hidden states of the forward GRU at time steps \(t-1\) and \(t+1\); \(\overleftarrow{h_{t-1}}\) and \(\overleftarrow{h_{t+1}}\) represent the hidden states of the backward GRU at time steps \(t-1\) and \(t+1\); \(x_{t-1}\), \(x_t\), and \(x_{t+1}\) denote the input vectors at time steps \(t-1\), \(t\), and \(t+1\), which correspond to the token representations generated by BERT; \(\sigma\) refers to the sigmoid activation function, and \(\tanh\) represents the hyperbolic tangent activation function. The output representation from BERT is fed into both forward and backward GRU units to capture contextual dependencies in both directions. The final hidden states from both directions are then combined to provide dynamic support for modeling emotional evolution trends. Specifically, the BERT output \(H\) is input into the forward GRU unit to model the sequence dependency, as shown in Eq. 4:

\begin{equation} \label{eq:4} \left\{\begin{aligned} \overrightarrow{h_t} &=\textrm{GRU}_f\left(h_t,\overrightarrow{h_{t-1}}\right), \\ \overleftarrow{h_t} &=\textrm{GRU}_b\left(h_t,\overleftarrow{h_{t+1}}\right), \\ \end{aligned}\right. \end{equation}
where \(\textrm{GRU}_f\) and \(\textrm{GRU}_b\) denote the forward and backward gated recurrent units, respectively. It helps model the syntactic logic in English writing, such as structures like “although \(\dots\) but \(\dots\).” The bidirectional hidden states are concatenated to obtain the final temporal representation, as shown in Eq. 5:
\begin{equation} \label{eq:5} h_t^{\textrm{BiGRU}}=\left[\overrightarrow{h_t};\overleftarrow{h_t}\right], \end{equation}
where \(h_t^{\text{BiGRU}}\in \mathbb{R}^{2d}\) is the concatenated output of the forward and backward GRU. The notation \([\,\cdot\,;\,\cdot\,]\) denotes vector concatenation along the feature dimension. After obtaining the sequence representation, an attention mechanism is further applied so that the model can automatically focus on the fragments that are more discriminative in emotional expression, as shown in Eq. 6:
\begin{equation} \label{eq:6} \left\{\begin{aligned} \alpha_t &=\dfrac{\exp\left(\tanh\left(W_a h_t+b_a\right)\right)}{\displaystyle\sum_{i=1}^T \exp\left(\tanh\left(W_a h_i+b_a\right)\right)},\\ v &=\displaystyle\sum_{i=1}^T \alpha_t h_t, \\ \end{aligned}\right. \end{equation}
where \(W_a\) and \(b_a\) are the parameters of the attention layer. \(\alpha_t\) is the attention weight at time \(t\), and \(v\) is the context vector after weighting. \(T\) refers to the sequence length, namely the number of tokens in the input text. Through the attention mechanism, the model not only has the ability of global semantic and sequence modeling, but also concentrates on key emotional words or short phrases, which improves the recognition effect of fine-grained emotions in complex writing texts.

3.3. Fine-Grained Emotion Modeling Based on CNN and Gating Mechanism

After completing global semantic and temporal modeling in Section 3.2, the study addressed the insufficient capture of local emotional features by introducing a CNN feature extraction layer and a gating fusion mechanism. For example, in an expression such as “not very happy,” the sentiment polarity is mainly determined by the local phrase rather than the overall sentence structure. To overcome this limitation, CNN was used to specifically extract local emotional features. Combined with the dual-level modeling framework defined in Section 3.1, the gating mechanism and multi-task learning were further integrated to achieve more targeted fine-grained sentiment recognition. The detailed structure of the CNN module in modeling local emotional cues illustrates the process from feature extraction to abstraction, as shown in Fig. 3.

figure

Fig. 3. Schematic diagram of CNN structure.

Figure 3 presents the CNN module process, from the input layer to the emotion classification output 29. Specifically, the structure first applies convolution and pooling operations to the input vector sequence to gradually extract local contextual semantic features and form hierarchical phrase representations. Then, a fully connected layer compresses and abstracts the features, which are finally mapped to the emotional classification output unit. This process plays a key role in capturing phrase-level emotional fragments and effectively enhances the model’s fine-grained recognition capability. To further depict the specific mechanism of the convolution kernel in extracting emotional phrases, its local sliding operation is defined in Eq. 7:

\begin{equation} \label{eq:8} c_i=f\left(W_c\cdot H_{i:i+k-1}+b_c\right), \end{equation}
where \(H_{i:i+k-1}\) represents the continuous vector segment from position \(i\) to \(i+k-1\) in the BERT output sequence \(H\in \mathbb{R}^{T\times d}\); \(W_c\in \mathbb{R}^{k\times d}\) is the convolution kernel weight matrix, \(b_c\) is the bias, and \(f(\cdot)\) is the nonlinear function. \(c_i\) denotes the local feature value of the convolution window at position iii. Through this operation, the model can detect local emotional signals of length \(k\). The output is then processed by max pooling to obtain the strongest activation value as the local feature representation, as shown in Eq. 8:
\begin{equation} \label{eq:9} z=\max\left(c_1,c_2,\dots,c_{T-k+1}\right), \end{equation}
where \(z\) represents the maximum value among all convolution window outputs, serving as the final local emotional feature. Although BERT provides global semantic representation, and BiGRU captures sequential dependencies, CNN complements local information. However, simple concatenation of different features may introduce redundancy and noise, especially in long texts or complex expressions. To address this, a gating fusion mechanism is designed, where the feature weight is dynamically adjusted, as shown in Eq. 9:
\begin{equation} \label{eq:10} g=\sigma\left(W_g\left[s;z;f_{\textit{seq}}\right]+b_g\right), \end{equation}
where \([s;z;f_{\textit{seq}}]\) denotes the concatenation of BERT global semantic vector \(s\), CNN local feature \(z\), and BiGRU sequential feature \(f_{\textit{seq}}\); \(\sigma(\cdot)\) is the sigmoid function, \(W_g\) is the weight matrix, and \(b_g\) is the bias. This calculation gives the importance weight of each feature channel. The fused feature is expressed in Eq. 10:
\begin{equation} \label{eq:11} u=g\odot \left[s;z;f_{\textit{seq}}\right], \end{equation}
where \(\odot\) represents element-wise multiplication, and \(u\) is the final fused feature. Unlike simple concatenation, the gating mechanism can dynamically suppress irrelevant features and highlight important ones, allowing local and sequential information to combine effectively. If simple concatenation is applied, the model may overlook weak but useful cues. For example, “slightly sad” and “very sad” both contain the word “sad,” but the difference lies in intensity. To further improve fine-grained recognition, a multi-task learning strategy is used. Along with emotion classification, intensity prediction is added. The classification output is shown in Eq. 11:
\begin{equation} \label{eq:12} \hat{y}=\textrm{softmax}\left(W_c u+b_c\right), \end{equation}
where \(\hat{y}\) is the predicted distribution, \(W_c\) and \(b_c\) are the weight and bias of the classifier. The emotion intensity prediction is given in Eq. 12:
\begin{equation} \label{eq:13} \hat{r}=W_r u+b_r, \end{equation}
where \(\hat{r}\) is the predicted emotion intensity, and \(W_r\) and \(b_r\) are the weight and bias of the regression layer. The classification and regression losses are shown in Eq. 13:
\begin{equation} \label{eq:14} \left\{\begin{aligned} \mathcal{L}_{\textit{cls}} &=-\sum_{i=1}^N y_i\log \left(\hat{y}_i\right), \\ \mathcal{L}_{\textit{reg}} &=\dfrac{1}{N}\sum_{i=1}^N \left(r_i-\hat{r}_i\right)^2,\\ \end{aligned} \right. \end{equation}
where \(\mathcal{L}_{\textit{cls}}\) and \(\mathcal{L}_{\textit{reg}}\) denote the classification loss and regression loss, \(r_i\) is the true intensity, \(\hat{r}_i\) is the predicted value, \(y_i\) is the ground-truth label, and \(N\) is the total number of samples. The joint optimization objective is expressed in Eq. 14:
\begin{equation} \label{eq:15} \mathcal{L}=\lambda_1\mathcal{L}_{\textit{cls}}+\lambda_2\mathcal{L}_{\textit{reg}}, \end{equation}
where \(\lambda_1\) and \(\lambda_2\) are weight parameters. With this combined loss, the model keeps the accuracy of classification while learning the weak continuous information of emotional intensity, thereby improving the discriminative ability and generalization in fine-grained emotion recognition.

Table 1. Sample distribution of emotion categories datasets.

figure

4. Resuls and Discussion

4.1. Datasets and Experimental Settings

To systematically evaluate the generalization ability and robustness of the proposed model under different text structures and sentiment granularity scenarios, three public datasets were selected for experiments: SST-2, IMDB, and Emotion. These datasets cover short-text sentiment classification, long-text emotion modeling, and multi-class fine-grained emotion recognition tasks, respectively, ensuring a comprehensive assessment. To improve the comparability of category numbers and sample sizes across datasets and to reduce the impact of class imbalance, balanced subsets were constructed based on the official datasets. Specifically, random sampling with a unified ratio was performed for each dataset, and the data were divided into training and testing sets at a ratio of \(8{:}2\). The final sample distribution is presented in Table 1.

(1) SST-2 and IMDB datasets

The original SST-2 and IMDB datasets are binary sentiment corpora containing only Positive and Negative labels. To meet the requirements of the multi-class fine-grained sentiment task, the study expanded the emotion categories and re-annotated the samples while keeping the original texts unchanged. Based on predefined emotion semantics, the samples were reassigned into five categories: Negative, Neutral, Positive, Surprise, and Sadness. To ensure objectivity and consistency, unified annotation criteria were defined. Negative indicates clear negative polarity. Positive represents affirmative or favorable evaluation. Neutral refers to weak or objective statements. Surprise denotes unexpected reactions. Sadness reflects low-mood emotional states. For samples containing multiple emotions, the dominant polarity was used for labeling. The re-annotation was independently completed by two annotators with linguistic backgrounds. Disagreements were resolved through discussion. Annotation agreement was evaluated using Cohen’s \(\kappa\). The \(\kappa\) value was 0.82 for SST-2 and 0.80 for IMDB, indicating high reliability. SST-2 was mainly used to evaluate semantic modeling on short texts, whereas IMDB was used to assess contextual dependency modeling in long documents.

(2) Emotion dataset

The Emotion dataset is a multi-class emotion corpus with six original labels: Joy, Sadness, Fear, Anger, Disgust, and Surprise. Based on the official data, random sampling was performed to build a size-controlled experimental subset. The data were then split into training and testing sets using the same ratio. Because this dataset contains more detailed emotional expressions, it is suitable for evaluating the model’s performance and generalization in multi-class fine-grained emotion recognition.

To ensure fair comparison across datasets, all texts underwent a unified preprocessing pipeline: (1) text cleaning to remove special symbols and redundant spaces; (2) tokenization and indexing; (3) construction of BERT-compatible token IDs, segment IDs, and attention masks; (4) truncation or padding to a fixed maximum length. All models were trained under the same experimental platform and hyperparameter settings. During training, 10% of the training set was randomly split as a validation set for parameter tuning and hyperparameter selection. Final performance was reported on the independent test set. The experimental setup includes Ubuntu 20.04 as the operating system, with Python 3.10 and PyTorch 2.0 for model implementation. Training is conducted on a server equipped with an NVIDIA RTX A6000 GPU. The optimizer is Adam with an initial learning rate of 1e-4. The batch size is set to 32, the maximum number of training epochs is 20, and the dropout rate is 0.3. To ensure reliability and comparability, all quantitative results were calculated as the mean of five independent runs. Line charts and curves display representative runs for visualization purposes.

figure

Fig. 4. Comparison of accuracy and loss convergence. (a) Accuracy comparison; (b) loss convergence.

figure

Fig. 5. Feature distribution and classification performance. (a) t-SNE visualization; (b) confusion matrix.

4.2. Validation of Semantic and Temporal Modeling with BERT-BiGRU

To verify the effectiveness of the proposed BERT-BiGRU model in the task of emotion recognition in English writing, experiments are conducted on training and testing sets. Evaluation is performed using two metrics: accuracy and loss. The compared models include BERT-only (using only the BERT encoder) and BiGRU (applying bidirectional GRU for sequence modeling). Fig. 4 presents the corresponding results.

From Fig. 4(a), it can be observed that all three models show a rapid increase in the initial stage. The proposed method becomes stable after about 10 epochs and keeps the highest accuracy (0.95) during training, which is clearly higher than 0.8 for BERT-only and 0.7 for BiGRU. Fig. 4(b) shows the loss convergence process. The proposed method converges quickly after the 5th epoch, and the final loss on the test set remains at 0.23, while BERT-only and BiGRU converge only to 0.30 and 0.34, showing faster convergence and lower error for the proposed method. These results demonstrate the high accuracy and training efficiency of the proposed model in emotion recognition tasks. To further evaluate the model’s ability to distinguish emotional features and classify them accurately, feature separability and large-scale sample classification performance are analyzed. Fig. 5 displays the experimental outcomes.

Figure 5(a) shows the result of t-distributed stochastic neighbor embedding (t-SNE), samples from the same category form clear clusters, and different categories show distinct boundaries, indicating that the model captures separable emotional features in the semantic space. Fig. 5(b) presents a confusion matrix based on 5,000 labeled samples from the SST-2 dataset. The overall accuracy across five emotion classes is 84.6%. Among them, Negative (86.0%) and Sadness (85.4%) are recognized with the highest accuracy, while Neutral achieved 81.7%, which is slightly lower but still acceptable. The results confirm the model’s stable performance and effective multi-class emotion classification ability.

To evaluate the effectiveness of the proposed BERT-BiGRU in semantic and temporal modeling, RoBERTa, XLNet, and Enhanced Representation through Knowledge Integration 2.0 (ERNIE 2.0) were used as baseline methods. Performance was assessed on SST-2, IMDB, and Emotion datasets. The evaluation metrics included matthews correlation coefficient (MCC) and expected calibration error (ECE). The results are presented in Table 2.

Table 2. Performance comparison across datasets (mean \(\pm\) std over 5 runs).

figure

As shown in Table 2, the proposed model outperformed the baselines across all three datasets. On SST-2, the model achieved an MCC of 0.884 and reduced ECE to 0.032, better than RoBERTa (0.861, 0.047) and XLNet (0.858, 0.051), indicating stronger robustness in short-text sentiment classification. On the IMDB long-text dataset, the model also obtained the best MCC of 0.887 and significantly lowered ECE to 0.029, demonstrating more reliable predictions under complex contexts. For the multi-class Emotion dataset, the model maintained a high MCC of 0.881 with an ECE of only 0.031, showing clear advantages over ERNIE 2.0 (0.860, 0.047). Overall, BERT-BiGRU achieved superior performance in classification accuracy, class balance, and probability calibration, confirming the rationality and effectiveness of combining semantic and temporal modelin.

4.3. Validation of Fine-Grained Emotion Recognition with BERT-CNN-BiGRU

In order to evaluate how well the proposed fusion model classifies emotions, the receiver operating characteristic (ROC) curve is used for comparison analysis. The comparison includes three baseline models: BERT with gated recurrent unit (BERT-GRU), TextCNN, and LSTM. The ROC curve and area under the curve (AUC) are selected as evaluation metrics. Two widely used sentiment datasets, SST-2 and IMDB, are employed for experimentation. The results are shown in Fig. 6.

In Fig. 6(a), the ROC curve of the proposed method lies entirely above those of the other three models (BERT-GRU, TextCNN, and LSTM) on the SST-2 dataset. The AUC reached 0.97, which is higher than BERT-GRU (0.93), TextCNN (0.89), and LSTM (0.86), indicating stronger discrimination in short-text sentiment recognition. Similarly, in Fig. 6(b), the AUC of the proposed model is 0.95 on the IMDB dataset, outperforming BERT-GRU (0.91), TextCNN (0.88), and LSTM (0.85). The model also shows better TPR under low FPR conditions (FPR \(<0.2\)), further demonstrating robustness and adaptability to long-text tasks.

To verify the effectiveness of each component in the proposed framework, an ablation study was conducted. BERT, BiGRU, CNN, the attention mechanism, the gating fusion module, and the multi-task learning component (including emotion intensity prediction) were removed or replaced one by one. Comparative evaluations were performed on the SST-2 and IMDB datasets. The evaluation metrics included accuracy, F1-score, and recall, and the results are summarized in Table 3.

figure

Fig. 6. ROC curve comparison. (a) ROC curve on SST-2; (b) ROC curve on IMDB.

Table 3. Ablation results of proposed modules (mean\(\pm\)std over 5 runs).

figure
figure

Fig. 7. Performance comparison with five models. (a) Performance comparison on SST-2; (b) performance comparison on IMDB.

As shown in Table 3, the full model achieved the best performance on both datasets. Specifically, on SST-2, the accuracy reached 93%, the F1-score was 91%, the recall achieved 92%, and the AUC was 0.975. On IMDB, the accuracy was also 93%, while the F1-score and recall were 90% and 91%, respectively. When CNN or BiGRU was removed, the AUC on both datasets decreased by 2%–5%. The accuracy and F1-score dropped by about 2–5 percentage points on average. This result indicates that both local feature extraction and temporal modeling played important roles in capturing phrase-level patterns and emotion evolution. When the attention module was removed, the AUC of SST-2 and IMDB declined to 0.930 and 0.922, respectively. The average accuracy decreased by about 5 percentage points. This finding suggests that the attention mechanism helped the model focus on key emotional information and improved recognition performance. Removing BERT caused the most significant degradation. The AUC of SST-2 and IMDB fell to 0.915 and 0.908, respectively. Both accuracy and F1-score dropped by more than 8 percentage points. This result confirms the fundamental role of deep semantic representation. When BiGRU was replaced with a unidirectional GRU, the AUC decreased by 5.5%. This further demonstrates that bidirectional modeling is more effective for capturing contextual dependencies. After removing the gating mechanism, the AUC of SST-2 and IMDB decreased to 0.947 and 0.941. The accuracy and F1-score dropped by about 3 percentage points on average. This indicates that the gating module effectively suppressed redundant information and highlighted key features. When multi-task learning was disabled, the AUC of SST-2 and IMDB declined to 0.952 and 0.945. The recall decreased by approximately 2–3 percentage points. This result suggests that emotion intensity prediction provided additional benefits for distinguishing ambiguous emotional boundaries. Overall, BERT, BiGRU, and CNN mainly supported global semantic modeling, temporal dependency learning, and local feature extraction, respectively. The gating fusion and multi-task strategy further improved the robustness and fine-grained recognition capability of the proposed model.

Table 4. Model evaluation on SST-2, IMDB, and Emotion (mean\(\pm\)std over 5 runs).

figure

In order to check how well the model performs and adapts in different sentiment classification tasks, the SST-2 and IMDB datasets are used again. Four performance metrics are selected for comparison with five advanced models: BERT-GRU, TextCNN, LSTM, RoBERTa-GRU (robustly optimized BERT pretraining approach with GRU), and BERT-BiLSTM (BERT with bidirectional LSTM). These models represent a range of common methods from traditional deep learning to transformer-based approaches. The findings are illustrated in Fig. 7.

Figure 7(a) shows that on the SST-2 dataset, the proposed model achieved over 92% in accuracy, recall, F1-score, and precision, with overall performance clearly surpassing the compared models, confirming its precision advantage in short-text sentiment classification. In Fig. 7(b), even with the more complex IMDB dataset, the model maintains leading performance, achieving 93% accuracy and 93% F1-score, outperforming RoBERTa-GRU (88%, 87%) and BERT-BiLSTM (86%, 85%). These results confirmed the model’s generalization ability. Overall, the proposed method consistently shows high discrimination and robustness across different text types.

To further verify the effectiveness of the proposed model in fine-grained emotion recognition, it was compared with three advanced methods, namely XLNet-BiGRU-Attention, RoBERTa-BiLSTM-Attention, and Multi-XLNet-RCNN, on three datasets. The evaluation metrics included F1-macro, ROC-AUC (micro), PR-AUC (micro), and quadratic weighted kappa (QWK). In particular, QWK is used to measure the consistency between model predictions and human annotations, with higher values indicating stronger agreement. The results are reported in Table 4.

From the results in Table 4, it can be seen that the proposed model achieved better performance than the comparison models on all three datasets. On SST-2 and IMDB, the F1-macro reached 92.0%, while ROC-AUC (micro) and PR-AUC (micro) exceeded 0.94 and 0.92, respectively. The QWK was close to 0.90, outperforming the baselines. On the Emotion dataset, although the task was more challenging, the proposed model still obtained the highest values across the four metrics, with an F1-macro of 89.3%, ROC-AUC of 0.935, PR-AUC of 0.918, and QWK of 0.884. These results confirm that the model has stronger generalization and label consistency in multi-class and fine-grained emotion recognition tasks.

4.4. Strengths and Limitations of the Experimental Results

Overall, the proposed BERT-CNN-BiGRU framework outperformed the baseline methods on multiple datasets and evaluation metrics. It showed strong classification accuracy, good probability calibration, and high label agreement. The ablation study further confirmed the role of each component. BERT provided deep semantic representations and formed the foundation of performance improvement. BiGRU and the attention mechanism enhanced temporal dependency modeling. CNN helped capture phrase-level emotional cues. The gating fusion and multi-task strategy improved robustness and fine-grained discrimination. Stable results across short texts, long texts, and multi-class tasks indicate good generalization ability.

Despite these advantages, several limitations remain. Although overall performance was strong, some fluctuation across emotion categories was observed in the multi-class task. This suggests that complex emotion discrimination still needs improvement. In addition, the multi-module architecture increases computational cost. Current experiments were conducted mainly on public datasets. Further validation in real educational writing scenarios is still required.

5. Conclusion

The growing demand for intelligent writing evaluation and emotional understanding in education has driven research on fine-grained emotion recognition in English writing texts. This study developed a BERT-CNN-BiGRU integrated framework to address the challenges of insufficient semantic modeling and limited temporal feature extraction. By combining attention, gated fusion, and multi-task learning, the model achieved joint modeling of semantic, temporal, and local features. Compared with conventional architectures, the proposed method demonstrated stronger capability in semantic comprehension, feature aggregation, and contextual dependency modeling. Experimental verification showed that the framework provides a robust and generalizable solution for fine-grained emotion recognition in English writing, reflecting both higher precision and stability across diverse datasets.

Despite the notable improvements in accuracy, robustness, and calibration performance, several limitations remain. The relatively limited number of emotion categories may constrain fine-grained representation, while the absence of external affective knowledge and multimodal information may restrict semantic richness. Moreover, as discussed in Section 4.4, performance fluctuations across emotion categories, relatively high computational complexity, and the lack of validation in real educational writing scenarios still warrant further investigation. Future research will explicitly address these limitations. First, expanding the emotion category system or constructing a hierarchical emotion taxonomy may further improve the modeling of complex emotional expressions. Second, integrating affective lexicons or knowledge-enhanced pre-trained models could strengthen semantic understanding. Third, incorporating multimodal information (e.g., text, speech, and facial expressions) may enable more comprehensive emotion modeling. Fourth, exploring lightweight architectures and domain-adaptive learning strategies is expected to improve computational efficiency and cross-domain generalization. Finally, future work will involve validation on real-world educational writing datasets to further verify the practical applicability and robustness of the proposed model. Availability of Data and Materials The datasets generated during and/or analyzed during the current study available from the corresponding author on reasonable request. Competing Interests The authors have no competing interests to declare that are relevant to the content of this article. Authors’ Contributions The author solely conceived the research idea, designed the methodology, performed the experiments, analyzed the data, and wrote the manuscript.

References
  1. [1] B. Wan, P. Wu, C. K. Yeo, and G. Li, “Emotion-cognitive reasoning integrated BERT for sentiment analysis of online public opinions on emergencies,” Inf. Process. Manag., Vol.61, No.2, Article No.103609, 2024. https://doi.org/10.1016/j.ipm.2023.103609
  2. [2] I. Üveges and O. Ring, “HunEmBERT: A fine-tuned BERT-model for classifying sentiment and emotion in political communication,” IEEE Access, Vol.11, pp. 60267-60278, 2023. https://doi.org/10.1109/ACCESS.2023.3285536
  3. [3] G. Singh, D. Brahma, P. Rai, and A. Modi, “Text-based fine-grained emotion prediction,” IEEE Trans. Affect. Comput., Vol.15, No.2, pp. 405-416, 2023. https://doi.org/10.1109/TAFFC.2023.3298405
  4. [4] D. Kim and P. Kang, “Cross-modal distillation with audio–text fusion for fine-grained emotion classification using BERT and Wav2vec 2.0,” Neurocomputing, Vol.506, pp. 168-183, 2022. https://doi.org/10.1016/j.neucom.2022.07.035
  5. [5] Y. He, J. Chen, D. Antonyrajah, and I. Horrocks, “BERTMap: A BERT-based ontology alignment system,” Proc. AAAI Conf. Artif. Intell., Vol.36, No.5, pp. 5684-5691, 2022. https://doi.org/10.1609/aaai.v36i5.20510
  6. [6] T. Han, Z. Zhang, M. Ren et al., “Text emotion recognition based on XLNet-BiGRU-Att,” Electronics, Vol.12, No.12, Article No.2704, 2023. https://doi.org/10.3390/electronics12122704
  7. [7] J. Deng and Y. Liu, “Research on sentiment analysis of online public opinion based on RoBERTa–BiLSTM–attention model,” Appl. Sci., Vol.15, No.4, Article No.2148, 2025. https://doi.org/10.3390/app15042148
  8. [8] Y. Xu and J. Wang, “Design and implementation of information system for vocational education quality assessment based on CNN-BiGRU,” Syst. Soft Comput., Vol.7, Article No.200359, 2025. https://doi.org/10.1016/j.sasc.2025.200359
  9. [9] J. Alghamdi, Y. Lin, and S. Luo, “Enhancing hierarchical attention networks with CNN and stylistic features for fake news detection,” Expert Syst. Appl., Vol.257, Article No.125024, 2024. https://doi.org/10.1016/j.eswa.2024.125024
  10. [10] M. Khadhraoui, H. Bellaaj, M. B. Ammar, H. Hamam, and M. Jmaiel, “Survey of BERT-base models for scientific text classification: COVID-19 case study,” Appl. Sci., Vol.12, No.6, Article No.2891, 2022. https://doi.org/10.3390/app12062891
  11. [11] N. M. Gardazi, A. Daud, M. K. Malik et al., “BERT applications in natural language processing: A review,” Artif. Intell. Rev., Vol.58, Article No.166, 2025. https://doi.org/10.1007/s10462-025-11162-5
  12. [12] G. Nkhata, S. Gauch, U. Anjum et al., “Fine-tuning BERT with bidirectional LSTM for fine-grained movie reviews sentiment analysis,” arXiv preprint, arXiv:2502.20682, 2025. https://doi.org/10.48550/arXiv.2502.20682
  13. [13] K. L. Tan, C. P. Lee, K. S. M. Anbananthen et al., “RoBERTa-LSTM: A hybrid model for sentiment analysis with transformer and recurrent neural network,” IEEE Access, Vol.10, pp. 21517-21525, 2022. https://doi.org/10.1109/ACCESS.2022.3152828
  14. [14] J. Chen and L. Pan, “An AI-based cross-language aspect-level sentiment analysis model using English corpus,” Eng. Rep., Vol.6, No.12, Article No.e12969, 2024. https://doi.org/10.1002/eng2.12969
  15. [15] J. Zhang, C. Li, G. Liu, M. Min, C. Wang, J. Li, Y. Wang, H. Yan, Z. Zuo, W. Huang, and H. Chen, “A CNN-transformer hybrid approach for decoding visual neural activity into text,” Comput. Methods Programs Biomed., Vol.214, Article No.106586, 2022. https://doi.org/10.1016/j.cmpb.2021.106586
  16. [16] F. Yang, “CNN-transformer triple classification of news text based on multiple strategy optimizations,” Procedia Comput. Sci., Vol.266, pp. 891-898, 2025. https://doi.org/10.1016/j.procs.2025.08.111
  17. [17] J. Zhang, J. Wu, H. Wang, Y. Wang, and Y. Li, “Cloud detection method using CNN based on cascaded feature attention and channel attention,” IEEE Trans. Geosci. Remote Sens., Vol.60, Article No.4104717, 2021. https://doi.org/10.1109/TGRS.2021.3120752
  18. [18] Q. Zhu, X. Jiang, and R. Ye, “Sentiment analysis of review text based on BiGRU-attention and hybrid CNN,” IEEE Access, Vol.9, pp. 149077-149088, 2021. https://doi.org/10.1109/ACCESS.2021.3118537
  19. [19] P. Ma, G. Li, H. Zhang, C. Wang, and X. Li, “Prediction of remaining useful life of rolling bearings based on multiscale efficient channel attention CNN and bidirectional GRU,” IEEE Trans. Instrum. Meas., Vol.73, Article No.2508413, 2024. https://doi.org/10.1109/TIM.2023.3347787
  20. [20] Z. Jin and K. Jia, “A temporal multi-scale hybrid attention network for sleep stage classification,” Med. Biol. Eng. Comput., Vol.61, pp. 2291-2303, 2023. https://doi.org/10.1007/s11517-023-02808-z
  21. [21] L. Na, T. Cao, S. Bai, and D. Li, “A parallel two-channel emotion classification method for Chinese text,” IEEE Access, Vol.12, pp. 39533-39548, 2024. https://doi.org/10.1109/ACCESS.2024.3350190
  22. [22] X. Liu, S. Li, and M. Wang, “Hierarchical attention-based multimodal fusion network for video emotion recognition,” Comput. Intell. Neurosci., Vol.2021, No.1, Article No.5585041, 2021. https://doi.org/10.1155/2021/5585041
  23. [23] P. Foggia, A. Greco, A. Saggese, and M. Vento, “Multi-task learning on the edge for effective gender, age, ethnicity and emotion recognition,” Eng. Appl. Artif. Intell., Vol.118, Article No.105651, 2023. https://doi.org/10.1016/j.engappai.2022.105651
  24. [24] A. Kocak, E. Erdem, and A. Erdem, “A gated fusion network for dynamic saliency prediction,” IEEE Trans. Cogn. Dev. Syst., Vol.14, No.3, pp. 995-1008, 2021. https://doi.org/10.1109/TCDS.2021.3094974
  25. [25] T. Zhang, C. Liu, X. Liu, Y. Liu, L. Meng, L. Sun, W. Jiang, F. Zhang, J. Zhao, and Q. Jin, “Multi-task learning framework for emotion recognition in-the-wild,” Eur. Conf. Comput. Vis., pp. 143-156, 2022. https://doi.org/10.1007/978-3-031-25075-0_11
  26. [26] Q. Na, X. Li, Y. Wang, J. Li, Y. Yang, and H. Zhang, “A pre-training method inspired by large language model for power named entity recognition,” Proc. 5th Int. Conf. Internet Things, Autom. Artif. Intell., pp. 308-312, 2023. https://doi.org/10.1145/3653081.3653131
  27. [27] R. Bensoltane and T. Zaki, “Combining BERT with TCN-BiGRU for enhancing Arabic aspect category detection,” J. Intell. Fuzzy Syst., Vol.44, No.3, pp. 4123-4136, 2023. https://doi.org/10.3233/JIFS-221214
  28. [28] A. Kamal, P. Mohankumar, and V. K. Singh, “IMFinE: An integrated BERT-CNN-BiGRU model for mental health detection in financial context on textual data,” Proc. 19th Int. Conf. Nat. Lang. Process. (ICON), pp. 139-148, 2022. https://aclanthology.org/2022.icon-main.19/
  29. [29] X. Bi and T. Zhang, “Pedagogical sentiment analysis based on the BERT-CNN-BiGRU-attention model in the context of intercultural communication barriers,” PeerJ Comput. Sci., Vol.10, Article No.e2166, 2024. https://doi.org/10.7717/peerj-cs.2166

Creative Commons License  This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Jul. 19, 2026