single-jc.php

JACIII Vol.28 No.1 pp. 179-185
doi: 10.20965/jaciii.2024.p0179
(2024)

Research Paper:

Semantic Similarity Analysis via Syntax Dependency Structure and Gate Recurrent Unit

Qiao Kang*, Jing Kan**, Fangyan Dong*, and Kewei Chen*,†

*Faculty of Mechanical Engineering & Mechanics, Ningbo University
No.818 Fenghua Road, Jiangbei District, Ningbo, Zhejiang 315211, China

Corresponding author

**Advanced Institute of Information Technology, Peking University
Hangzhou Bay Wisdom Valley, No.233 Yonghui Road, Xiaoshan District, Hangzhou, Zhejiang 311215, China

Received:
January 31, 2023
Accepted:
September 19, 2023
Published:
January 20, 2024
Keywords:
semantic similarity, GRU, relative syntactic distance, syntactic structure, natural language processing (NLP)
Abstract

Sentences are composed of words, phrases, and clauses. The relationship between them is usually tree-like. In the hierarchical structure of the sentence, the dependency relationships between different components affect the syntactic structure. Syntactic structure is very important for understanding the meaning of the whole sentence. However, the gated recursive unit (GRU) models cannot fully encode hierarchical syntactic dependencies, which leads to its poor performance in various natural language tasks. In this paper, a model called relative syntactic distance bidirectional gated recursive unit (RSD-BiGRU) is constructed to capture syntactic structure dependencies. The model modifies the gating mechanism in GRU through relative syntactic distance. It also offers a transformation gate to model the syntactic structure more directly. Embedding sentence meanings with sentence structure dependency into dense vectors. This model is used to conduct semantic similarity experiments on the QQP and SICK datasets. The results show that the sentence representation obtained by RSD-BiGRU model contains more semantic information. This is helpful for semantic similarity analysis tasks.

Cite this article as:
Q. Kang, J. Kan, F. Dong, and K. Chen, “Semantic Similarity Analysis via Syntax Dependency Structure and Gate Recurrent Unit,” J. Adv. Comput. Intell. Intell. Inform., Vol.28 No.1, pp. 179-185, 2024.
Data files:
References
  1. [1] M. Han et al., “A survey on the techniques, applications, and performance of short text semantic similarity,” Concurrency and Computation Practice and Experience, Vol.33, No.5, Article No.e5971, 2021. https://doi.org/10.1002/cpe.5971
  2. [2] A. O. N. Rene, K. Okuhara, and T. Matsui, “Natural language generation system for knowledge acquisition based on patent database,” J. Adv. Comput. Intell. Intell. Inform., Vol.26, No.2, pp. 160-168, 2022. https://doi.org/10.20965/jaciii.2022.p0160
  3. [3] S. Wang and J. Jiang, “Machine comprehension using match-LSTM and answer pointer,” arXiv: 1608.07905, 2016. https://doi.org/10.48550/arXiv.1608.07905
  4. [4] J. Kleenankandy and Abdul Nozeer K. A., “An enhanced Tree-LSTM architecture for sentence semantic modeling using typed dependencies,” Information Processing & Management, Vol.57, No.6, Article No.102362, 2020. https://doi.org/10.1016/j.ipm.2020.102362
  5. [5] Y. Zhou, C. Liu, and Y. Pan, “Modelling sentence pairs with tree-structured attentive encoder,” Proc. of the 26th Int. Conf. on Computational Linguistics (COLING 2016): Technical Papers, pp. 2912-2922, 2016.
  6. [6] I. Arroyo-Fernández et al., “Unsupervised sentence representations as word information series: Revisiting TF-IDF,” Computer Speech & Language, Vol.56, pp. 107-129, 2019. https://doi.org/10.1016/j.csl.2019.01.005
  7. [7] G. Varelas et al., “Semantic similarity methods in wordNet and their application to information retrieval on the web,” Proc. of the 7th Annual ACM Int. Workshop on Web Information and Data Management (WIDM’05), pp. 10-16, 2005. https://doi.org/10.1145/1097047.1097051
  8. [8] J. Chambua et al., “Tensor factorization method based on review text semantic similarity for rating prediction,” Expert Systems with Applications, pp. 629-638, 2018. https://doi.org/10.1016/j.eswa.2018.07.059
  9. [9] X. Li et al., “Text similarity measurement with semantic analysis,” Int. J. of Innovative Computing, Information and Control, Vol.13, No.5, pp. 1693-1708, 2017.
  10. [10] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation, pp. 1735-1780, 1997. https://doi.org/10.1162/neco.1997.9.8.1735
  11. [11] K. Cho et al., “Learning phrase representations using RNN encoder-decoder for statistical machine translation,” arXiv: 1406.1078, 2014. https://doi.org/10.48550/arXiv.1406.1078
  12. [12] Y. Shen et al., “Neural language modeling by jointly learning syntax and lexicon,” arXiv: 1711.02013, 2017. https://doi.org/10.48550/arXiv.1711.02013
  13. [13] K. S. Tai, R. Socher, and C. D. Manning, “Improved semantic representations from tree-structured long short-term memory networks,” arXiv: 1503.00075, 2015. https://doi.org/10.48550/arXiv.1503.00075
  14. [14] C. Manning et al., “The Stanford CoreNLP natural language processing toolkit,” Proc. of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 55-60, 2014. https://doi.org/10.3115/v1/P14-5010
  15. [15] J. Pennington, R. Socher, and C. Manning, “GloVe: Global vectors for word representation,” Proc. of the 2014 Conf. on Empirical Methods in Natural Language Processing, pp. 1532-1543, 2014. https://doi.org/10.3115/v1/D14-1162

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Jun. 18, 2024