single-jc.php

JACIII Vol.30 No.6 pp. 1782-1798
(2026)

Research Paper:

Topic Modelling for Biomedical Literature Retrieval via a Dual-Stream Embedding Framework

Tao Tao* ORCID Icon, Amena Mahmoud** ORCID Icon, and Eiko Takaoka* ORCID Icon

*Graduate School of Science and Technology, Sophia University
7-1 Kioi-cho, Chiyoda-ku, Tokyo 102-8554, Japan

**Department of Computer Science, Faculty of Computers and Information, Kafrelsheikh University
El-Geish Street, Kafrelsheikh 33516, Egypt

Received:
January 8, 2026
Accepted:
June 5, 2026
Published:
November 20, 2026
Keywords:
information retrieval, large language models, medical information systems, semantic search, topic models
Abstract

The rapid expansion of biomedical literature has created substantial challenges in retrieving and synthesizing relevant knowledge, particularly for novice and interdisciplinary users. Conventional keyword-based methods often lack semantic depth, whereas traditional topic modelling techniques struggle to capture domain-specific contexts and rarely integrate structured metadata, such as medical subject headings (MeSH). To address these limitations, this study introduces a dual-stream embedding framework that integrates unstructured textual content with curated biomedical metadata using transformer-based encoders and a parameterized fusion mechanism. The architecture combines contextual semantics from article titles and abstracts with expert-annotated MeSH descriptors and incorporates a large language model to generate concise and human-readable topic labels. Experimental results show that three of the four evaluated embedding configurations outperformed the single-stream baseline and that the dual-stream fusion surpassed all ablation models built from the same inputs. The best configuration, Dual PubMedBERT, achieved the highest overall topic modelling quality, improving the combined measure of topic coherence and topic diversity by 7.3% relative to the baseline. These results demonstrate the effectiveness of integrating contextual textual embeddings with structured biomedical metadata using the proposed fusion mechanism. This approach is scalable, reproducible, and domain-adaptable, providing both a methodological advancement in semantic topic modelling and a practical foundation for the development of biomedical information retrieval systems, semantic search engines, and thematic exploration tools across knowledge-intensive domains.

Dual-stream fusion of text and MeSH

Dual-stream fusion of text and MeSH

Cite this article as:
T. Tao, A. Mahmoud, and E. Takaoka, “Topic Modelling for Biomedical Literature Retrieval via a Dual-Stream Embedding Framework,” J. Adv. Comput. Intell. Intell. Inform., Vol.30 No.6, pp. 1782-1798, 2026.
Data files:
References
  1. [1] Q. Jin, R. Leaman, and Z. Lu, “PubMed and beyond: biomedical literature search in the age of artificial intelligence,” eBioMedicine, Vol.100, Article No.104988, 2024. https://doi.org/10.1016/j.ebiom.2024.104988
  2. [2] H. C. Huurdeman and J. Kamps, “Designing Multistage Search Systems to Support the Information Seeking Process,” W. T. Fu and H. van Oostendorp (Eds.), “Understanding and Improving Information Search,” pp. 113-137, Springer, 2020. https://doi.org/10.1007/978-3-030-38825-6_7
  3. [3] D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent Dirichlet allocation,” J. of Machine Learning Research, Vol.3, pp. 993-1022, 2003.
  4. [4] D. M. Blei, “Probabilistic topic models,” Communications of the ACM, Vol.55, Issue 4, pp. 77-84, 2012. https://doi.org/10.1145/2133806.2133826
  5. [5] S. Kavvadias, G. Drosatos, and E. Kaldoudi, “Supporting topic modeling and trends analysis in biomedical literature,” J. of Biomedical Informatics, Vol.110, Article No.103574, 2020. https://doi.org/10.1016/j.jbi.2020.103574
  6. [6] H. Kilicoglu, “Biomedical text mining for research rigor and integrity: tasks, challenges, directions,” Briefings in Bioinformatics, Vol.19, Issue 6, pp. 1400-1414, 2018. https://doi.org/10.1093/bib/bbx057
  7. [7] T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient Estimation of Word Representations in Vector Space,” arXiv:1301.3781, 2013. https://doi.org/10.48550/arXiv.1301.3781
  8. [8] A. B. Dieng, F. J. R. Ruiz, and D. M. Blei, “Topic Modeling in Embedding Spaces,” Trans. of the Association for Computational Linguistics, Vol.8, pp. 439-453, 2020. https://doi.org/10.1162/tacl_a_00325
  9. [9] D. Angelov, “Top2Vec: Distributed Representations of Topics,” arXiv:2008.09470, 2020. https://doi.org/10.48550/arXiv.2008.09470
  10. [10] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol.1 (Long and Short Papers), pp. 4171-4186, 2019. https://doi.org/10.18653/v1/N19-1423
  11. [11] I. Beltagy, K. Lo, and A. Cohan, “SciBERT: A Pretrained Language Model for Scientific Text,” Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing and the 9th Int. Joint Conf. on Natural Language Processing (EMNLP-IJCNLP), pp. 3615-3620, 2019. https://doi.org/10.18653/v1/D19-1371
  12. [12] J. Lee, W. Yoon, S. Kim, D. Kim, S. Kim, C. H. So, and J. Kang, “BioBERT: A pre-trained biomedical language representation model for biomedical text mining,” Bioinformatics, Vol.36, Issue 4, pp. 1234-1240, 2020. https://doi.org/10.1093/bioinformatics/btz682
  13. [13] M. Grootendorst, “BERTopic: Neural topic modeling with a class-based TF-IDF procedure,” arXiv:2203.05794, 2022. https://doi.org/10.48550/arXiv.2203.05794
  14. [14] Y. Gu, R. Tinn, H. Cheng, M. Lucas, N. Usuyama, X. Liu, T. Naumann, J. Gao, and H. Poon, “Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing,” ACM Trans. on Computing for Healthcare, Vol.3, Issue 1, Article No.2, 2022. https://doi.org/10.1145/3458754
  15. [15] N. Reimers and I. Gurevych, “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,” arXiv:1908.10084, 2019. https://doi.org/10.48550/arXiv.1908.10084
  16. [16] Z. Yu, E. Bernstam, T. Cohen, B. C. Wallace, and T. R. Johnson, “Improving the utility of MeSH® terms using the TopicalMeSH representation,” J. of Biomedical Informatics, Vol.61, pp. 77-86, 2016. https://doi.org/10.1016/j.jbi.2016.03.013
  17. [17] D. Mimno, H. Wallach, E. Talley, M. Leenders, and A. McCallum, “Optimizing Semantic Coherence in Topic Models,” Proc. of the 2011 Conf. on Empirical Methods in Natural Language Processing, pp. 262-272, 2011.
  18. [18] O. Lezhnina, “Depression, anxiety, and burnout in academia: Topic modeling of PubMed abstracts,” Frontiers in Research Metrics and Analytics, Vol.8, Article No.1271385, 2023. https://doi.org/10.3389/frma.2023.1271385
  19. [19] Y. Wang, R. Benavides, L. Diatchenko, A. V. Grant, and Y. Li, “A graph-embedded topic model enables characterization of diverse pain phenotypes among UK biobank individuals,” iScience, Vol.25, Issue 6, Article No.104390, 2022. https://doi.org/10.1016/j.isci.2022.104390
  20. [20] M. Röder, A. Both, and A. Hinneburg, “Exploring the Space of Topic Coherence Measures,” Proc. of the 8th ACM Int. Conf. on Web Search and Data Mining, pp. 399-408, 2015. https://doi.org/10.1145/2684822.2685324

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Nov. 20, 2026