Sentences are composed of words, phrases, and clauses. The relationship between them is usually tree-like. In the hierarchical structure of the sentence, the dependency relationships between different components affect the syntactic structure. Syntactic structure is very important for understanding the meaning of the whole sentence. However, the gated recursive unit (GRU) models cannot fully encode hierarchical syntactic dependencies, which leads to its poor performance in various natural language tasks. In this paper, a model called relative syntactic distance bidirectional gated recursive unit (RSD-BiGRU) is constructed to capture syntactic structure dependencies. The model modifies the gating mechanism in GRU through relative syntactic distance. It also offers a transformation gate to model the syntactic structure more directly. Embedding sentence meanings with sentence structure dependency into dense vectors. This model is used to conduct semantic similarity experiments on the QQP and SICK datasets. The results show that the sentence representation obtained by RSD-BiGRU model contains more semantic information. This is helpful for semantic similarity analysis tasks.

1. [1] M. Han et al., “A survey on the techniques, applications, and performance of short text semantic similarity,” Concurrency and Computation Practice and Experience, Vol.33, No.5, Article No.e5971, 2021. https://doi.org/10.1002/cpe.5971
2. [2] A. O. N. Rene, K. Okuhara, and T. Matsui, “Natural language generation system for knowledge acquisition based on patent database,” J. Adv. Comput. Intell. Intell. Inform., Vol.26, No.2, pp. 160-168, 2022. https://doi.org/10.20965/jaciii.2022.p0160
3. [3] S. Wang and J. Jiang, “Machine comprehension using match-LSTM and answer pointer,” arXiv: 1608.07905, 2016. https://doi.org/10.48550/arXiv.1608.07905
4. [4] J. Kleenankandy and Abdul Nozeer K. A., “An enhanced Tree-LSTM architecture for sentence semantic modeling using typed dependencies,” Information Processing & Management, Vol.57, No.6, Article No.102362, 2020. https://doi.org/10.1016/j.ipm.2020.102362
5. [5] Y. Zhou, C. Liu, and Y. Pan, “Modelling sentence pairs with tree-structured attentive encoder,” Proc. of the 26th Int. Conf. on Computational Linguistics (COLING 2016): Technical Papers, pp. 2912-2922, 2016.
6. [6] I. Arroyo-Fernández et al., “Unsupervised sentence representations as word information series: Revisiting TF-IDF,” Computer Speech & Language, Vol.56, pp. 107-129, 2019. https://doi.org/10.1016/j.csl.2019.01.005
7. [7] G. Varelas et al., “Semantic similarity methods in wordNet and their application to information retrieval on the web,” Proc. of the 7th Annual ACM Int. Workshop on Web Information and Data Management (WIDM’05), pp. 10-16, 2005. https://doi.org/10.1145/1097047.1097051
8. [8] J. Chambua et al., “Tensor factorization method based on review text semantic similarity for rating prediction,” Expert Systems with Applications, pp. 629-638, 2018. https://doi.org/10.1016/j.eswa.2018.07.059
9. [9] X. Li et al., “Text similarity measurement with semantic analysis,” Int. J. of Innovative Computing, Information and Control, Vol.13, No.5, pp. 1693-1708, 2017.
10. [10] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation, pp. 1735-1780, 1997. https://doi.org/10.1162/neco.1997.9.8.1735
11. [11] K. Cho et al., “Learning phrase representations using RNN encoder-decoder for statistical machine translation,” arXiv: 1406.1078, 2014. https://doi.org/10.48550/arXiv.1406.1078
12. [12] Y. Shen et al., “Neural language modeling by jointly learning syntax and lexicon,” arXiv: 1711.02013, 2017. https://doi.org/10.48550/arXiv.1711.02013
13. [13] K. S. Tai, R. Socher, and C. D. Manning, “Improved semantic representations from tree-structured long short-term memory networks,” arXiv: 1503.00075, 2015. https://doi.org/10.48550/arXiv.1503.00075
14. [14] C. Manning et al., “The Stanford CoreNLP natural language processing toolkit,” Proc. of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 55-60, 2014. https://doi.org/10.3115/v1/P14-5010
15. [15] J. Pennington, R. Socher, and C. Manning, “GloVe: Global vectors for word representation,” Proc. of the 2014 Conf. on Empirical Methods in Natural Language Processing, pp. 1532-1543, 2014. https://doi.org/10.3115/v1/D14-1162