Research Paper:
Topic Modelling for Biomedical Literature Retrieval via a Dual-Stream Embedding Framework
Tao Tao*
, Amena Mahmoud**
, and Eiko Takaoka*

*Graduate School of Science and Technology, Sophia University
7-1 Kioi-cho, Chiyoda-ku, Tokyo 102-8554, Japan
**Department of Computer Science, Faculty of Computers and Information, Kafrelsheikh University
El-Geish Street, Kafrelsheikh 33516, Egypt
The rapid expansion of biomedical literature has created substantial challenges in retrieving and synthesizing relevant knowledge, particularly for novice and interdisciplinary users. Conventional keyword-based methods often lack semantic depth, whereas traditional topic modelling techniques struggle to capture domain-specific contexts and rarely integrate structured metadata, such as medical subject headings (MeSH). To address these limitations, this study introduces a dual-stream embedding framework that integrates unstructured textual content with curated biomedical metadata using transformer-based encoders and a parameterized fusion mechanism. The architecture combines contextual semantics from article titles and abstracts with expert-annotated MeSH descriptors and incorporates a large language model to generate concise and human-readable topic labels. Experimental results show that three of the four evaluated embedding configurations outperformed the single-stream baseline and that the dual-stream fusion surpassed all ablation models built from the same inputs. The best configuration, Dual PubMedBERT, achieved the highest overall topic modelling quality, improving the combined measure of topic coherence and topic diversity by 7.3% relative to the baseline. These results demonstrate the effectiveness of integrating contextual textual embeddings with structured biomedical metadata using the proposed fusion mechanism. This approach is scalable, reproducible, and domain-adaptable, providing both a methodological advancement in semantic topic modelling and a practical foundation for the development of biomedical information retrieval systems, semantic search engines, and thematic exploration tools across knowledge-intensive domains.
Dual-stream fusion of text and MeSH
1. Introduction
Efficient thematic discovery rather than simple keyword matching has become a central bottleneck in biomedical search. The scale and growth of PubMed exacerbate this difficulty, with tens of millions of records and more than one million new entries per year 1. This information overload poses particular challenges for novice biomedical researchers and for those entering the field from interdisciplinary backgrounds. Unlike experienced domain experts who may possess well-defined search heuristics, these users often struggle to articulate precise information needs and navigate the vast biomedical knowledge space. In the early stages of research, identifying a suitable entry point and narrowing it to a relevant thematic area require substantial time and cognitive effort. As highlighted by Huurdeman and Kamps 2, a key bottleneck in information retrieval is supporting users during the initial exploratory phase, helping them clarify their objectives and converge on a focused area of inquiry. This persistent issue underscores the need for intelligent retrieval systems that go beyond keyword-based searches and guide users through the conceptual landscape of biomedical literature.
Topic modelling, an unsupervised machine-learning approach designed to extract latent thematic structures from text corpora, has emerged as a promising solution for biomedical literature retrieval and organization. However, existing approaches often fail to adequately integrate the structured biomedical knowledge available through curated metadata, such as medical subject headings (MeSH). These structured metadata encodes significant expert knowledge about article content, which, if effectively integrated, can enhance the semantic richness and specificity of generated topics. Thus, there is a clear need for topic modelling methods that combine unstructured textual content with structured domain knowledge to produce more semantically coherent and distinct topics suitable for practical biomedical information retrieval.
To address this gap, we propose a dual-stream embedding framework built on the BERTopic pipeline that integrates textual embeddings (titles and abstracts) and structured MeSH term embeddings using a parameterized weighted fusion mechanism. This mechanism leverages two adjustable parameters, \(\alpha\) and \(\beta\), which control the relative importance of text versus MeSH embeddings and of major versus minor MeSH terms, respectively. Through a systematic evaluation involving various pretrained embedding model combinations (general-purpose and biomedical-specific), we identified configurations that improve topic modelling quality, quantified by topic coherence (TC) and topic diversity (TD). The primary objectives of this study are threefold: (i) to systematically evaluate the effectiveness of the dual-stream embedding framework for improving biomedical topic modelling quality; (ii) to examine the impact of embedding model choices and parameter configurations on performance; and (iii) to demonstrate the framework’s practical utility as a foundational component for biomedical literature retrieval systems with applications in semantic search, thematic organization, and user-centered discovery tools.
The novelty of this study is best understood by comparing it with two lines of prior research. Existing BERTopic extensions operate on a single embedding stream and treat all input text homogeneously. Even when metadata, such as MeSH terms, are included, they are concatenated into the same textual sequence and lose their distinct status as expert-curated knowledge. Multimodal topic modelling methods typically fuse sources through simple concatenation or fixed-weight averaging, which offers no explicit control over the relative contribution of each source. Our framework departs from both: it keeps unstructured text and structured MeSH metadata as two separate embedding streams and introduces a two-layer parameterized fusion (\(\alpha\) for the text–MeSH balance, and \(\beta\) for the major–minor MeSH weighting) that makes the contribution of each knowledge source explicit and tunable. To the best of our knowledge, this is the first topic modelling framework to treat curated biomedical metadata as an independent, separately weighted embedding stream rather than as an auxiliary text. A detailed methodological comparison with representative prior approaches is presented in Section 6.
Our experiments and comparisons with baseline methods and ablation studies confirmed that the proposed dual-stream embedding framework improves TC and diversity, with the Dual PubMedBERT configuration achieving \(\mathrm{TC} = 0.6172\), \(\mathrm{TD} = 0.7458\), and \(\mathrm{TC}+\mathrm{TD} = 1.3443\), a 7.3% improvement over the baseline (\(\mathrm{TC}+\mathrm{TD} = 1.2530\)). These findings indicate that integrating structured biomedical metadata with contextualized textual embeddings enriches thematic discovery and interpretation, benefiting biomedical informatics applications, such as semantic search, thematic literature summarization, and evidence synthesis.
The remainder of this paper is organized as follows. Section 2 reviews related studies on topic modelling, neural embeddings, and biomedical domain applications. Section 3 describes the dataset, preprocessing, and dual-stream embedding framework. Section 4 presents the experimental design, and Section 5 presents the experimental results and analysis. Section 6 discusses the limitations and future directions, and Section 7 concludes the paper.
3. Methodology and Proposed Framework
3.1. System Workflow Overview
An overview of the proposed topic modelling framework is presented in Fig. 1. The architecture comprises five modular stages: data preparation, dual-stream embedding, topic modelling using BERTopic, topic labeling using an LLM, and evaluation and analysis.

Fig. 1. System workflow of the proposed dual-stream topic modelling framework.
In the data preparation stage, article titles, abstracts, and MeSH descriptors are extracted from PubMed records. Each MeSH term is associated with a binary MajorTopicYN flag that indicates whether the term represents a major or minor topic. Pretrained transformer models, including MiniLM, PubMedBERT, and BioBERT, are loaded for downstream embedding generation. In the dual-stream embedding stage, two parallel embedding vectors are computed: one for unstructured biomedical text (titles and abstracts) and the other for structured MeSH descriptors. The MeSH stream is processed through a first-layer fusion mechanism, which applies a weighting scheme controlled by parameter \(\beta\) to emphasize major MeSH terms over minor ones. The resulting weighted MeSH embedding is then combined with textual embedding via a second-layer fusion mechanism governed by parameter \(\alpha\), which balances the contributions of textual and MeSH streams to form the final document representation. In the topic modelling stage, the fused embeddings are fed into the BERTopic pipeline. Dimensionality reduction is performed using UMAP, followed by clustering using HDBSCAN. Topic representations are generated via CountVectorizer and class-based TF-IDF (c-TF-IDF) and are further refined via maximal marginal relevance (MMR) to improve interpretability by promoting keyword diversity. In the topic labeling stage, the top-ranked keywords from each topic are extracted and used to prompt an LLM to generate concise, human-readable topic titles. This step improves accessibility for users without domain expertise and supports downstream applications, such as thematic dashboards and visualizations. In the evaluation and analysis stage, a grid search is conducted over the \(\alpha\) and \(\beta\) values to explore the parameter space. For each configuration, TC and TD are computed to assess the semantic quality of the generated topics. These results were compared with those of single-stream and ablation baselines to validate the performance improvements introduced by the dual-layer fusion mechanism.
3.2. Dataset
The dataset used in this study comprises 3,000 biomedical article records retrieved from the PubMed database. Each record contains the article title, abstract, publication year, PubMed ID (PMID), and associated MeSH terms. To ensure comparability with existing methods and enable a controlled baseline comparison, we adopted the exact dataset and preprocessing methodology used in the study “Depression, anxiety, and burnout in academia: topic modeling of PubMed abstracts” 18. This ensures that any observed differences in topic quality or interpretability are attributable to the modelling framework rather than to variations in data sourcing or processing. The PubMed records were structured according to the MEDLINE format, which includes metadata such as titles, abstracts, publication dates, and manually assigned MeSH. Each MeSH term was accompanied by a MajorTopicYN flag, indicating whether the term represented a primary subject focus (MajorTopic\(\mathrm{YN}=\mathrm{Y}\)) or a secondary/supporting concept (MajorTopic\(\mathrm{YN}=\mathrm{N}\)). In this study, we utilize this binary distinction to inform the \(\beta\)-weighted fusion of MeSH embeddings, thereby emphasizing core medical topics during the integration process.
3.3. Data Preprocessing
Minimal preprocessing was required because of the design of the embedding framework. Specifically, we employed models from the Sentence Transformers library, which utilizes pretrained transformer-based encoders that are robust to raw, unprocessed text inputs 15. Unlike traditional bag-of-words topic models, which often require lower casing, lemmatization, stopword removal, or token filtering, transformer-based sentence embeddings inherently capture contextual semantics without relying on such preprocessing steps. This aligns with the findings from prior studies that show that modern sentence-embedding models can operate effectively on unaltered text, preserving more nuanced linguistic and biomedical structures 19. Specifically, the subword (WordPiece/BPE) tokenization used by these encoders decomposes rare or morphologically complex biomedical terms into known subunits, such that lemmatization and aggressive token filtering are unnecessary. Conversely, stopword removal and lowercasing would discard casing and function-word cues that the transformer’s contextual attention exploits and could therefore degrade rather than improve the embeddings. As such, our pipeline does not apply additional text normalization procedures beyond basic format verification.
The preprocessing workflow transforms raw PubMed records into structured document representations suitable for the dual-stream embedding framework. Each record undergoes systematic extraction of the core components (title, abstract, MeSH terms, PMID, and publication year), followed by format verification and MeSH metadata structuring. Critically, the MajorTopicYN flag associated with each MeSH term is preserved during this stage, enabling the subsequent parameterized fusion mechanism to distinguish between major and minor topics. The complete preprocessing procedure is presented in Algorithm 1.

The preprocessing algorithm employs three helper functions to ensure data quality: (i) Verify_Format performs minimal text cleaning by removing null characters and stripping leading/trailing whitespace while preserving the original text structure; (ii) Has_Required_Fields validates document completeness by ensuring that title, abstract, and MeSH terms are all nonempty (\(\text{document.title} \neq \emptyset \land \text{document.abstract} \neq \emptyset \land \text{document.mesh}\_\text{terms} \neq \emptyset\)); and (iii) Concatenate combines the title, abstract, and MeSH descriptors into a unified full-text representation using space separators.
3.4. Dual-Stream Embedding Framework
The proposed dual-stream embedding framework was designed to integrate unstructured textual content and structured biomedical metadata through two parallel embedding channels: a textual stream and MeSH stream. In this study, both the embedding inputs and BERTopic document inputs were strictly derived from the full-text combination (Title + Abstract + MeSH). We employed models from the Sentence Transformers library to generate sentence-level embeddings from this unified input. This choice was motivated by the relatively long and syntactically rich nature of biomedical abstracts, for which sentence-level transformers have demonstrated strong performance in capturing contextual semantics and discourse-level coherence 15. Additionally, sentence transformer models offer superior robustness in downstream integration scenarios, making them well-suited for incorporation into real-world biomedical search systems. For the MeSH stream, which represents the structured domain-specific descriptors associated with each article, we explored two types of embeddings:
-
・
A Sentence Transformer model pretrained on biomedical corpora (NeuML/pubmedbert-base embedding)
-
・
The original BioBERT model (dmis-lab/biobert-v1.1) with mean pooling was applied.
We compared which embedding strategy better captured the semantics of MeSH terms, which are typically composed of concise biomedical phrases or terminology.
3.5. Parameterized Weighted Fusion Mechanism
To effectively integrate unstructured textual content and structured biomedical metadata, a two-layer parameterized weighted fusion mechanism was developed. This design governs the combination of (i) sentence-level embeddings derived from article titles and abstracts (the text stream) and (ii) embeddings generated from MeSH descriptors (the MeSH stream).
Two scalar parameters control the fusion process:
-
・
Text–MeSH fusion weight (\(\boldsymbol{\alpha}\)): It adjusts the relative importance of text embeddings.
-
・
Major–Minor MeSH weighting parameter (\(\boldsymbol{\beta}\)): It adjusts the relative per-term weight assigned to major versus minor MeSH terms based on the MajorTopicYN flag.
Both \(\alpha\) and \(\beta\) are tunable within the range [0.05, 0.95] with a step size of 0.05.
First-layer fusion: intra-MeSH term weighting (\(\boldsymbol{\beta}\)). Each article’s MeSH terms are first categorized as either major or minor using the MajorTopicYN flag. Each term is encoded using a biomedical transformer model (for example, BioBERT or PubMedBERT) and assigned a weight based on its importance:
-
・
Major topic: weight \(= \beta\)
-
・
Minor topic: weight \(= 1 - \beta\)
The weights are then normalized, and the weighted average embedding for MeSH terms is computed as follows:
Second-layer fusion: text–MeSH stream integration (\(\boldsymbol{\alpha}\)). The final document-level embedding is constructed by blending the MeSH embedding and the text embedding through the parameter \(\alpha\), as follows:
If an article does not contain MeSH terms, only the text embedding is used. All articles in the dataset used in this study contained MeSH descriptors.
Implementation. The fusion mechanism was implemented as follows:
-
1.
Extract MeSH descriptors and mark each as major or minor.
-
2.
Encode each descriptor with a pretrained biomedical encoder to obtain embeddings.
-
3.
Assign weights \(\beta\) (major) and \((1-\beta)\) (minor), normalize the weights, and compute the weighted sum of MeSH embeddings.
-
4.
Compute the final embedding as \(\alpha\cdot\text{text}\_\text{embedding} + (1-\alpha)\cdot\text{mesh}\_\text{embedding}\) when MeSH exists; otherwise use the text_embedding.
The complete fusion logic is formalized in Algorithm 2.

3.6. Design Rationale and Benefits
This dual-stream embedding framework offers several advantages:
-
1)
Semantic complementarity: This method integrates contextual semantics from free text with structured domain knowledge from curated vocabularies, thereby providing a more comprehensive representation of document content.
-
2)
Fine-grained control: Adjustable parameters (\(\alpha\) and \(\beta\)) enable the framework to adapt to varying corpus characteristics and information retrieval objectives, allowing task-specific optimization.
-
3)
Interpretability: The explicit separation and weighting of text and MeSH components enhance model transparency, facilitating an understanding of the retrieval mechanism and promoting user trust.
It is worth clarifying that in the current framework, MeSH descriptors are deliberately treated as independent terms; each descriptor is encoded individually, and the descriptors are aggregated by a weighted average, without explicitly modelling the MeSH tree hierarchy, synonym (entry-term) mappings, or subheading qualifiers. This is considered a design choice rather than an oversight. First, the contextual encoders used here (PubMedBERT and BioBERT) are pretrained on large biomedical corpora and, therefore, already capture a substantial degree of semantic relatedness between hierarchically or synonymously linked descriptors. Thus, part of the structural signal is recovered implicitly through the embedding space. Secondly, relying only on descriptor strings and the binary MajorTopicYN flag keeps the method dependent solely on the fields present in every MEDLINE record, making it broadly applicable and avoiding dependence on the full MeSH ontology being available and correctly parsed. Third, and most importantly, this minimal formulation isolates the contribution of the dual-stream architecture from the orthogonal question of how richly the MeSH structure is exploited, yielding a clean and controlled comparison against the baseline. We nonetheless recognize that the MeSH hierarchy, qualifiers, and synonym relations encode additional expert knowledge that a simple average cannot represent; exploiting them explicitly is a promising direction, which we return to in Section 6.
3.7. Topic Modelling Pipeline

Fig. 2. Modular architecture of the BERTopic pipeline.
To model latent thematic structures in the biomedical literature, we adopted the BERTopic framework 13, a modular and extensible pipeline that integrates state-of-the-art sentence embeddings with clustering and keyword extraction. BERTopic is particularly well-suited for biomedical domain applications because of its ability to generate interpretable topics from high-dimensional contextual embeddings via a structured and visually interpretable workflow (see Fig. 2).
To ensure rigorous and fair experimental comparisons, we strictly replicated the full modelling pipeline and hyperparameter configurations from the target baseline study 18. This ensures that all observed differences in topic quality stem solely from our proposed embedding strategies, rather than from downstream modelling variations.
The BERTopic pipeline, illustrated in Fig. 2, consists of the following key components:
-
・
Sentence-level embeddings were generated using our dual-stream embedding framework and served as input to the topic modelling pipeline.
-
・
Dimensionality reduction was performed using UMAP with the parameters set to \(\texttt{n}\_\texttt{neighbors}=15\), \(\texttt{n}\_\texttt{components}=5\), and \(\texttt{min}\_\texttt{dist}=0.0\). These are the standard BERTopic defaults: \(\texttt{n}\_\texttt{neighbors}=15\) balances the preservation of local and global structures, \(\texttt{min}\_\texttt{dist}=0.0\) produces tightly packed embeddings that favour downstream density-based clustering, and \(\texttt{n}\_\texttt{components}=5\) retains sufficient variance while mitigating the curse of dimensionality for HDBSCAN.
-
・
Document clustering was conducted using HDBSCAN, which was configured with \(\texttt{min}\_\texttt{cluster}\_\texttt{size}=30\) and \(\texttt{min}\_\texttt{samples}=10\), enabling the discovery of semantically coherent clusters without predefining the number of topics. \(\texttt{min}\_\texttt{cluster}\_\texttt{size}=30\) was adopted from the baseline study to ensure a directly comparable number of topics in the 3,000-document corpus, large enough to suppress spurious microclusters yet small enough to retain fine-grained themes, and \(\texttt{min}\_\texttt{samples}=10\) provides a conservative noise threshold that limits over-fragmentation.
-
・
Tokenization and keyword extraction rely on CountVectorizer and the ClassTfidfTransformer (c-TF-IDF) to compute discriminative topic representations.
-
・
To promote topic interpretability and reduce keyword redundancy, MMR was applied with a diversity parameter of 0.2.
-
・
No modifications were made to any of the above components, preserving complete alignment with the baseline.
This strict adherence to the baseline modelling setup isolates the impact of the dual-stream embedding framework under equivalent downstream conditions.
3.8. Evaluation Metrics Overview
To evaluate topic modelling quality, we adopted two widely used metrics: TC and TD. TC quantifies the semantic consistency of the top-ranked words within a topic, whereas TD measures the distinctiveness between topics by assessing vocabulary overlap. These metrics were selected to ensure comparability with the baseline study 18 and are widely used in the neural topic modelling literature 8,17,20.
4. Experiment Design
Experiments systematically explored the following:
-
1)
Four embedding model combinations (see Table 1).
-
2)
A comprehensive grid search of \(\alpha\) and \(\beta\) parameters across specified ranges.
| Combinations name | Text stream embedding model | MeSH stream embedding model | Description |
| Dual PubMedBERT | NeuML/pubmedbert-base-embeddings | NeuML/pubmedbert-base-embeddings | Both streams use the biomedical PubMedBERT encoder. |
| Hybrid PubMedBERT-BioBERT | NeuML/pubmedbert-base-embeddings | dmis-lab/biobert-v1.1 | Hybrid model: PubMedBERT for the text stream and BioBERT for the MeSH stream. |
|
Dual MiniLM (The baseline’s model) |
all-MiniLM-L6-v2 | all-MiniLM-L6-v2 | Both streams use the general-purpose all-MiniLM-L6-v2 encoder adopted by the baseline study. |
| Dual BioBERT | dmis-lab/biobert-v1.1 | dmis-lab/biobert-v1.1 | Both streams use the biomedical BioBERT encoder. |
The preliminary experiments explored various text input strategies (Abstract, Text, AbMeSH, and Full). However, all the final main experiments reported in this study exclusively adopted the full-text combination (Title + Abstract + MeSH) for both the embedding and BERTopic doc inputs, ensuring semantic alignment and eliminating input inconsistency.
Computational cost and deployment considerations. The \(\alpha\)–\(\beta\) grid search evaluates \(19 \times 19 = 361\) parameter combinations per model over the range \([0.05, 0.95]\) with a step of \(0.05\). Although transformer encoders are the most expensive components of the pipeline, each document’s text and MeSH-term embeddings are computed and cached only once, and every grid cell then recomputes only the weighted fusion—an \(\mathcal{O}(d)\) vector operation per document, where \(d\) is the embedding dimension, followed by the BERTopic clustering stage (UMAP and HDBSCAN). Therefore, the grid search is dominated by lightweight clustering rather than repeated neural encoding, which is an offline, one-time tuning cost. During deployment, once \(\alpha\) and \(\beta\) are fixed, embedding a new document requires only a single text encoding, a single MeSH encoding, and the \(\mathcal{O}(d)\) fusion, adding negligible overhead over a standard single-stream BERTopic pipeline. The search is also embarrassingly parallel across grid cells, so wall-clock time scales inversely with the available compute.
4.1. Ablation Study Design
We designed a controlled ablation study to isolate the contribution of architectural fusion from the raw information volume. Specifically, all ablation configurations use a single encoder model (all-MiniLM-L6-v2, same as the baseline) and avoid any fusion mechanism (no \(\alpha\)–\(\beta\) weighting). Instead, multiple input sources are concatenated into a single textual stream encoded via a single-stream embedding process. This allows us to evaluate whether a comparable topic modelling performance can be achieved without structural fusion, given the same input information.
4.2. Experimental Setup
We evaluated four configurations, each constructed using different combinations of biomedical content (Title, Abstract, and MeSH). Under all settings, the input components were concatenated into a single text sequence and embedded in the baseline model (all-MiniLM-L6-v2). This avoids architectural fusion or weight tuning and ensures that any performance differences arise solely from the nature of the input data.
-
・
A1 – Abstract only
-
・
A2 – Title + Abstract
-
・
A3 – Abstract + MeSH
-
・
A4 – Title + Abstract + MeSH (Full input)
These configurations are then compared to two reference models:
-
1)
Baseline: Using only the abstract as the input, the all-MiniLM-L6-v2 model was used, following previous studies.
-
2)
Best dual-stream fusion model with the all-MiniLM-L6-v2 model: The highest-performing configuration from the \(\alpha\)–\(\beta\) tuned dual-stream experiments.
The A1 (Abstract only) configuration replicates the baseline settings used in previous studies. To ensure methodological rigor, we re-ran this configuration independently using our devices.
4.3. Embedding Procedure
For each configuration, the relevant text components were concatenated into a single string, which was then passed through the all-MiniLM-L6-v2 encoder to produce document embedding. The resulting embeddings were used directly as inputs to the BERTopic pipeline (UMAP, HDBSCAN, and c-TF-IDF) without modification or fusion. This ensures alignment with the baseline processing while avoiding the additional complexity of multistream fusion.
4.4. Evaluation Metrics
To assess the topic modelling performance, we adopted two widely used evaluation metrics: TC and TD. These metrics were selected to ensure consistency with the baseline study 18, thereby enabling a direct comparison of the topic modelling results.
4.4.1. TC
TC quantifies the semantic consistency of a topic by evaluating the degree of co-occurrence among its top-ranked terms. Specifically, we use normalized pointwise mutual information (NPMI) over a reference corpus to compute coherence scores following standard methodologies in the literature 17,20.
To aid interpretability, we provide the following definitions for each variable in the evaluation metrics.
In Eq. (3), the TC score quantifies the semantic consistency among the top-ranked keywords for a given topic. Specifically:
-
・
where \(N\) denotes the number of top terms considered per topic (for example, \(N = 10\)).
-
・
where \(w_i\) and \(w_j\) are the \(i\)-th and \(j\)-th words among the top \(N\) words, respectively.
-
・
where NPMI\((w_i, w_j)\) is the normalized pointwise mutual information between words \(w_i\) and \(w_j\), computed over a reference corpus (for example, PubMed abstracts).
Equation (3) averages the pairwise NPMI scores across all unique word pairs in a topic, yielding a coherence score in the range \([-1, 1]\), where higher values indicate stronger semantic associations and better topic consistency.
4.4.2. TD
TD assesses the distinctiveness between topics by computing the proportion of the unique top-\(N\) words across all topics. Higher TD values indicate greater thematic breadth and reduced redundancy. This metric has been adopted in neural topic modelling literature as a complementary measure of coherence 8.
In Eq. (4), the TD score measures the distinctiveness of topics by evaluating the vocabulary overlap:
-
・
\(K\) is the total number of topics.
-
・
where \(\text{Top}_N(\text{Topic}_k)\) refers to the set of the top \(N\) words in the \(k\)-th topic.
-
・
The numerator \(\left|\bigcup_{k=1}^{K} \text{Top}_N(\text{Topic}_k)\right|\) counts the total number of unique terms across all top \(N\) word sets.
-
・
The denominator \(K \times N\) is the maximum possible number of distinct words if all the topics are entirely disjoint.
The resulting TD value ranges from 0 to 1, where higher values indicate greater topic distinctiveness and reduced redundancy across topics.
These two metrics evaluate topic quality by capturing the intratopic semantic consistency and intertopic separation, which are essential for interpretability and downstream usability in biomedical literature applications.
4.5. Comparison Protocol
All the ablation results were evaluated using the same metrics as those used in the main study: TC, TD, and their sum (\(\mathrm{TC}+\mathrm{TD}\)). Each configuration is compared with
-
・
the baseline model (Abstract only)
-
・
the best-performing dual-stream framework model (\(\alpha\)–\(\beta\) tuned, all-MiniLM-L6-v2 model).
This isolates the effect of information richness from structural innovation and validates whether multistream weighted integration is necessary to improve topic quality.
4.6. Topic Label Enhancement with a LLM
To improve the interpretability of the discovered topics, we integrated a post-processing step using an LLM via API calls. Although BERTopic provides keyword-based topic representations, these keyword sets are often fragmented or require expert interpretation. To generate human-readable, semantically coherent topic titles suitable for academic use, we employed the Qwen-14B-Instruct model using the API.
Each topic’s top keywords (obtained using BERTopic’s c-TFIDF ranking) were used as inputs for a structured prompt. The prompt instructed the model to generate a concise and accurate English title that summarized the topic meaningfully, avoiding simple keyword repetition and direct translation. This approach aims to simulate expert-level summarization and provide titles suitable for inclusion in academic papers, visualizations, and topic-based exploration tools.
The generation pipeline included:
-
・
cleaning and verifying the keyword format (ensuring a valid list of terms);
-
・
constructing a prompt template that contextualizes the generation task for the model;
-
・
iterative generation using Qwen2.5-14B-instruct-1M with a controlled decoding configuration (\(\mathrm{temperature}=0.3\), \(\mathrm{top}\_p=0.95\));
-
・
the output is stored in the experimental result table alongside the raw keyword representations.
An example prompt used:
You are an expert in academic topic summarization. Please generate a concise, accurate, human-readable English title summarizing the following keywords: cancer, therapy, chemotherapy.
Requirements:
- 1.
Use one short sentence.
- 2.
Focus on meaning, not literal translation or keyword listing.
- 3.
The output should be English, be academic sounding, and be suitable for reports or visualizations.
- 4.
Do not include quotes or numbering.
Please generate the topic title
This LLM-based topic labelling enhances downstream interpretability and provides a more intuitive interface for nonexpert users in literature exploration and clinical research.
Reliability and alternative labelling strategies. Two points should be emphasized regarding the role and reliability of this component. First, LLM-generated titles are a presentation-layer enhancement only: the quantitative topic-quality metrics (TC and TD) are computed exclusively from the c-TF-IDF keyword distributions, such that the labelling step does not influence any reported score and cannot inflate the comparison against the baseline. Second, the main reliability risk of LLM labelling is hallucinations or overgeneralization. We mitigate this by (i) constraining the prompt to summarize only the keywords supplied by BERTopic, without introducing external concepts, and (ii) using low-temperature decoding (\(\mathrm{temperature} =0.3\)), which yields stable, near-deterministic titles across runs. Compared with alternative labelling strategies such as raw c-TF-IDF keyword lists (the BERTopic default), KeyBERT-based keyphrase extraction, or selecting the most representative document title, the LLM approach produces fluent, self-contained titles that require no expert post-interpretation at the cost of dependence on an external model. Because the labelling layer is fully decoupled from the embedding and clustering stages, it can be replaced by any of these alternatives without affecting the dual-stream results.
5. Results
5.1. Overall Experimental Outcomes
Three of the four dual-stream embedding configurations outperformed the baseline, and dual-stream fusion surpassed every single-stream ablation model built from the same inputs. Among all configurations, the Dual PubMedBERT model achieved the highest overall topic modelling quality, with a TC of 0.6172, TD of 0.7458, and combined TC+TD score of 1.3443, representing a 7.3% improvement over the baseline (\(\mathrm{TC}+\mathrm{TD} = 1.2530\)). These results demonstrate the effectiveness of integrating contextual textual embeddings with structured biomedical metadata using the proposed fusion mechanism.
To provide a more intuitive view of the parameter landscape, we visualized the \(\alpha\)–\(\beta\) grid search results via three separate heatmaps.

Fig. 3. Heatmap of topic coherence (TC) scores across the \(\alpha\)–\(\beta\) parameter grid under the Dual PubMedBERT configuration.
Figure 3 shows the distribution of TC across different \(\alpha\)–\(\beta\) combinations. The highest TC value of 0.6172 was achieved at \(\alpha=0.40\) and \(\beta=0.65\), indicating that emphasizing textual information alongside strongly weighted major MeSH terms leads to more semantically consistent topic clusters.

Fig. 4. Heatmap of topic diversity (TD) scores across the \(\alpha\)–\(\beta\) parameter grid under the Dual PubMedBERT configuration.
Figure 4 illustrates the variation in TD. The highest TD score of 0.7458 occurred at \(\alpha=0.45\) and \(\beta=0.40\), suggesting that balanced weighting between textual and MeSH inputs fosters broader thematic coverage and reduced keyword overlap across topics.

Fig. 5. Heatmap of combined topic coherence and diversity scores across the \(\alpha\)–\(\beta\) parameter grid under the Dual PubMedBERT configuration.
Figure 5 presents the combined \(\mathrm{TC}+\mathrm{TD}\) scores, reflecting the trade-off between intratopic coherence and intertopic diversity. The optimal joint performance (\(\mathrm{TC}+\mathrm{TD} = 1.3443\)) was also observed at \(\alpha=0.45\) and \(\beta=0.40\), confirming the robustness of this fusion configuration.
Blank regions in the heatmaps denote \(\alpha\)–\(\beta\) combinations that produced an abnormally low number of clusters, typically fewer than five, making coherence and diversity scores unreliable. These were excluded from the visualizations to ensure clarity and interpretability.
5.2. Model-Specific Performance
The dual-stream embedding topic modelling framework demonstrated significant improvements over the baseline models across all evaluated metrics. The detailed performance of each model is summarized in Table 2.
| Model | Best TC | Best TD | Best TC+TD | Improvement |
| (\(\boldsymbol\alpha\), \(\boldsymbol\beta\)) | (\(\boldsymbol\alpha\), \(\boldsymbol\beta\)) | (\(\boldsymbol\alpha\), \(\boldsymbol\beta\)) | (vs. baseline) | |
| Dual PubMedBERT | 0.6172 | 0.7458 | 1.3443 | +7.3% |
| (0.40, 0.65) | (0.45, 0.40) | (0.45, 0.40) | ||
| Hybrid | 0.6038 | 0.7333 | 1.3247 | +5.7% |
| PubMedBERT-BioBERT | (0.15, 0.15) | (0.10, 0.25) | (0.30, 0.35) | |
| Dual MiniLM | 0.6076 | 0.7143 | 1.3005 | +3.8% |
| (The Baseline’s model) | (0.15, 0.20) | (0.10, 0.35) | (0.20, 0.35) | |
| Dual BioBERT | 0.5826 | 0.7000 | 1.2331 | \(-1.6\%\) |
| (0.45, 0.65) | (0.10, 0.15) | (0.45, 0.80) | ||
| Baseline | 0.5360 | 0.7170 | 1.2530 | – |

Fig. 6. Performance comparison of dual-stream model variants across TC, TD, and combined scores. Dual PubMedBERT achieves the highest combined score (\(\mathrm{TC}+\mathrm{TD} = 1.3443\)) with optimal parameters \(\alpha=0.45\) and \(\beta=0.40\). Data source: Table 2.
Table 2 shows that, on the combined \(\mathrm{TC}+\mathrm{TD}\) metric, three of the four dual-stream configurations outperform the baseline, while Dual BioBERT falls slightly below it. Notably, the Dual PubMedBERT model achieved the highest performance, with clear improvements in both coherence and diversity over the baseline. As illustrated in Fig. 6, Dual PubMedBERT outperforms all other variants, achieving the highest combined score (\(\mathrm{TC}+\mathrm{TD} = 1.3443\)) at optimal parameters \(\alpha=0.45\) and \(\beta=0.40\). The hybrid configuration demonstrates competitive performance with \(\mathrm{TC}+\mathrm{TD} = 1.3247\), while Dual BioBERT exhibits the lowest performance (\(\mathrm{TC}+\mathrm{TD} = 1.2331\)), even performing worse than the baseline by \(1.6\%\). This validates the importance of both model selection and proper hyperparameter tuning in the dual-stream framework.
To complement the aggregate metrics, we examined the statistical reliability of coherence improvement at the topic level. Per-TC (\(c_v\)) scores were computed for the optimal Dual PubMedBERT configuration (\(\alpha=0.45\), \(\beta=0.40\); 24 topics) and the single-stream baseline (all-MiniLM-L6-v2 on abstracts; 24 topics); these per-topic scores are obtained by topic-level decomposition and are not on the same scale as—and therefore not directly comparable to—the corpus-aggregate TC values reported in Table 2. The dual-stream model attained a higher mean per-TC (0.597 vs. 0.568). In a pairwise comparison, a randomly selected dual-stream topic exceeded a randomly selected baseline topic in coherence \(56\%\) of the time. A one-sided Mann–Whitney U test did not reach the conventional significance threshold (\(p=0.25\)). This is an expected consequence of the small number of topics (\(n=24\)) and the substantial intertopic variance characteristic of topic models: topic-level inference is inherently low-powered, which is why topic-modelling studies, including the baseline study, conventionally evaluate performance through aggregate TC and TD rather than per-topic hypothesis testing. The consistent direction of the per-topic difference combined with the improvement in both aggregate metrics in the optimal configuration indicates that the gain reflects a systematic shift in the coherence distribution rather than a single high-scoring topic.
5.3. Optimal Parameters and Robustness
A comprehensive parameter analysis identified the optimal parameter ranges for the dual-stream fusion mechanism, which were concentrated around moderate values:
-
・
\(\alpha\) (text vs. MeSH weighting): 0.15–0.45
-
・
\(\beta\) (major vs. minor MeSH weighting): 0.15–0.80
These parameter ranges indicated a balanced integration of textual content and structured biomedical knowledge, which significantly enhanced both the coherence and diversity of the discovered topics.
Compared with the baseline, the dual-stream models maintained reliable performance across a broad range of parameter settings, with the best configurations clustered in the moderate-value region of the \(\alpha\)–\(\beta\) grid rather than at isolated extreme points, indicating that the framework is not unduly sensitive to fine parameter choices.
| Topic ID | Top keywords | LLM-generated title |
| 0 | students, nursing, medical, learning, clinical, student, education, anxiety, simulation, burnout | Addressing Anxiety and Burnout in Nursing Students through Clinical Simulation-Based Medical Education |
| 1 | health, mental, students, depression, prevalence, symptoms, data, factors, study, numerical | Numerical Study on the Prevalence of Depression Symptoms in Mental Health Among Students |
| 2 | covid19, pandemic, health, mental, students, anxiety, depression, pandemics, stress, study | Mental Health Impact of Anxiety and Depression on Students During the COVID-19 Pandemic |
| 3 | intervention, mindfulness, trial, treatment, group, stress, anxiety, control, therapy, randomized | Randomized Trial of Mindfulness Intervention for Stress and Anxiety in Treatment Groups |
| 4 | burnout, residents, residency, resident, training, medical, physicians, trainees, internship, survey | Burnout among medical residents and trainees during internship: a survey analysis |
5.4. Enhanced Interpretability Through Topic Labelling
Applying the LLM-based topic-labelling procedure described above to the topics discovered by the best-performing configuration produced concise, human-readable titles instead of raw keyword lists.
Table 3 illustrates five representative examples of LLM-generated titles. Each title corresponds to a unique topic and demonstrates how natural language summarization can bridge the gap between machine-generated keywords and human comprehension. The results show that LLM-based postprocessing significantly improves topic accessibility and reduces the cognitive effort required to interpret and organize thematic outputs.
5.5. Ablation Study Results
To isolate the effectiveness of the proposed dual-stream fusion mechanism, we designed a set of ablation experiments using a fixed single-stream architecture without parameterized integration. Each configuration concatenated different combinations of textual and metadata inputs and encoded them via the baseline model (all-MiniLM-L6-v2) without applying any \(\alpha\)–\(\beta\) fusion.
| Experiment ID | Input configuration | Input components | TC | TD | TC+TD |
| A1 | Abstract only | Abstract | 0.5428 | 0.7136 | 1.2564 |
| A2 | Text | Title + Abstract | 0.5355 | 0.6667 | 1.2022 |
| A3 | AbMeSH | Abstract + MeSH | 0.5274 | 0.6087 | 1.1361 |
| A4 | Full | Title + Abstract + MeSH | 0.5451 | 0.6174 | 1.1625 |
| Baseline | Abstract only | Abstract | 0.5360 | 0.7170 | 1.2530 |
| Dual-stream | Full + \(\alpha\)–\(\beta\) fusion | Title + Abstract + MeSH | 0.6047 | 0.6958 | 1.3005 |

Fig. 7. Ablation study results showing the progressive performance variation across different input configurations (A1–A4) without parameterized fusion, compared to the baseline and the full dual-stream system with fusion. The dual-stream configuration achieves a \(3.8\%\) improvement over the baseline (\(\mathrm{TC}+\mathrm{TD}\): 1.3005 vs. 1.2530). Data source: Table 4.
All the experiments used the same random seed and embedding model to ensure strict comparability. A1 replicates the original “Abstract only” configuration used in the baseline study. We reran this experiment independently on our own computing infrastructure and obtained results that are extremely close to the reported baseline \(\mathrm{TC} = 0.5428\) vs. 0.5360 and \(\mathrm{TD} = 0.7136\) vs. 0.7170 (see Table 4). These minor discrepancies are likely due to differences in computational environments (for example, hardware-specific floating-point precision, GPU parallelism effects) rather than methodological inconsistencies.
Figure 7 demonstrates the impact of different input components on topic modelling performance. The ablation configurations (A1–A4) showed inconsistent improvements, with A2, A3, and A4 performing worse than the baseline. A closer look at the metric breakdown explains why, across A2–A4, TC remains roughly stable (\(0.527\)–\(0.545\)), whereas TD deteriorates sharply, from \(0.7136\) (A1) to \(0.6087\) (A3). Therefore, degradation is driven almost entirely by loss of diversity. This is a direct consequence of naive concatenation: appending MeSH descriptors as plain text injects a small, controlled, and heavily reused vocabulary (for example, generic descriptors such as Humans or Female) that the class-based TF-IDF cannot sufficiently downweight, inflating keyword overlap across topics and collapsing TD; appending titles adds further boilerplate phrasing with the same effect. Because flat concatenation cannot distinguish a document’s core (major) descriptors from incidental (minor) descriptors, generic term noise accumulates rather than being suppressed. The dual-stream design avoids this failure mode by encoding MeSH in a separate embedding space and weighting the major against minor terms such that generic descriptors never contaminate the keyword extraction stage. Consequently, the full dual-stream configuration with parameterized \(\alpha\)–\(\beta\) fusion achieves TC+TD = 1.3005, a \(3.8\%\) improvement over the baseline, confirming that the key innovation lies in the weighted fusion mechanism rather than in the inclusion of additional metadata.
Importantly, none of the ablation configurations, despite using the same input components, matched the performance of the dual-stream architecture. The proposed full fusion model yielded the highest \(\mathrm{TC}+\mathrm{TD}\) (1.3005), outperforming both the best ablation result (A1, \(\mathrm{TC}+\mathrm{TD} = 1.2564\)) and the baseline (1.2530). This confirms that simply adding information (for example, full input or MeSH terms) is not sufficient; the performance gains come from the structured, weighted integration of unstructured and structured knowledge, not from input volume alone.
5.6. Effect of the \(\beta\) Weighting Scheme
It should be noted that the per-term weighting of \(\beta\) (Eq. (1)) makes the effective document-level contribution of the major-descriptor group depend on the number of major and minor terms in each record. With \(M\) major and \(m\) minor descriptors, the effective contribution of the major group under the per-term formulation is \(M\beta/(M\beta+m(1-\beta))\) rather than \(\beta\) itself. In the present corpus, records carry on average substantially more minor than major descriptors (approximately twelve versus two), such that the effective major-group contribution is consistently lower than the nominal \(\beta\). The parameter \(\beta\) therefore acts as a monotonic control that increasingly emphasizes major descriptors as it grows, rather than as a literal document-level contribution ratio.
To assess whether this distinction affects topic quality, we compared the per-term formulation with a group-normalised alternative,
6. Discussion
6.1. Key Findings and Implications
The experimental results demonstrated the effectiveness of the proposed dual-stream embedding framework for biomedical topic modelling. Three of the four dual-stream configurations outperformed the single-stream baseline on the combined \(\mathrm{TC}+\mathrm{TD}\) metric, with the best configuration, Dual PubMedBERT, achieving \(\mathrm{TC}+\mathrm{TD} = 1.3443\), an improvement of 7.3%. One exception, Dual BioBERT, fell 1.6% below the baseline, indicating that the benefit of dual-stream fusion is conditional on an embedding model whose representation space is well-matched to the task: the framework amplifies a suitable encoder rather than compensating for an unsuitable one.
The dual-stream design provides two key benefits. First, it enables semantic complementarity between contextual embeddings, which capture linguistic nuances and discourse structures, and MeSH descriptors, which reflect domain-specific expert annotations. Second, the parameterized weighted fusion mechanism introduces fine-grained control over the contribution of each type of information. The grid search showed that moderate \(\alpha\) and \(\beta\) values yielded the best performance, underscoring the need for balanced integration rather than dominance of either stream.
Crucially, the ablation study confirms that the gains stem from structured integration rather than additional information alone. All single-stream ablation variants, despite using the same Title + Abstract + MeSH content, performed worse than the dual-stream fusion; the A4 configuration achieved only \(\mathrm{TC}+\mathrm{TD} = 1.1625\), well below the fused counterpart at 1.3005 using the same encoder. Therefore, architectural integration, not input volume, is a critical factor. By explicitly encoding and weighting narrative content and curated metadata as separate streams, the framework addresses a limitation of earlier transformer-based methods such as BERTopic, which treat all textual inputs homogeneously. The LLM-based topic-title generation further improves interpretability, which is particularly valuable for nonexpert users such as students and interdisciplinary researchers who require transparent thematic access to biomedical knowledge. The framework’s modular embedding, fusion, and topic-modelling stages also make it adaptable to other knowledge-intensive domains without major architectural changes.
6.3. Limitations and Future Work
Despite these encouraging results, this study has several limitations that point toward promising directions for future work.
First, the experiments were conducted on a single relatively small dataset of 3,000 PubMed abstracts selected to match the conditions of the reference baseline study. Although this design enables a clean and controlled benchmark, it has three consequences for the strength of our conclusions. (i) Generalizability: A single thematic corpus may not represent the full heterogeneity of biomedical literature, such that the specific optimal \(\alpha\)–\(\beta\) values reported here should be regarded as dataset-dependent rather than universal. (ii) External validation: Because all results are obtained on one corpus, we cannot yet claim that the observed improvements transfer to independent datasets or other biomedical subdomains; external validation on separate corpora is needed before the framework is adopted in production retrieval systems. (iii) Overfitting risk: The \(\alpha\) and \(\beta\) parameters were selected by grid search on the same corpus on which they are evaluated, which may optimistically bias the reported optima. Reassuringly, the high-performing configurations form a broad, contiguous region of the \(\alpha\)–\(\beta\) grid (\(\alpha \in [0.15, 0.45]\), \(\beta \in [0.15, 0.80]\)) rather than isolated spikes, which suggests the effect is robust rather than the product of a single overfitted cell; nevertheless, a held-out tuning/evaluation split would provide a stronger guarantee. Future research should therefore evaluate the scalability and robustness of the framework on larger and more diverse biomedical corpora, including documents that lack MeSH annotations, for which automatic MeSH tagging or prompt-based LLM generation could be explored, and adopt a train/validation/test protocol for parameter selection. The framework is expected to scale gracefully because, as noted in Section 4, document embeddings are computed once, and the fusion adds only an \(\mathcal{O}(d)\) operation per document.
Second, the \(\alpha\) and \(\beta\) parameters were tuned manually via grid search. Although this revealed interpretable and high-performance configurations, it introduces manual overhead and limits adaptability. As part of future development, we aim to implement an automated gating mechanism that dynamically adjusts the fusion weights from document features or a downstream optimization objective, thereby improving usability and generalizability in real-world biomedical information retrieval scenarios.
Third, as discussed in Section 3, the current framework treats MeSH descriptors as independent terms and does not explicitly exploit the MeSH knowledge organization system, its tree hierarchy, subheading qualifiers, or synonym (entry-term) mappings. These structures encode expert knowledge that a simple weighted average cannot capture. Hierarchically close descriptors can be smoothed toward shared ancestors, qualifiers can disambiguate a descriptor’s contextual role, and synonym mappings can consolidate lexical variants. Incorporating such information through hierarchy-aware aggregation, graph-based encoders over the MeSH tree, or qualifier-conditioned embeddings is a promising avenue for enriching structured streams and an important direction for future work.
Such advancements will facilitate their integration into interactive biomedical literature retrieval systems, enabling their practical deployment in clinical knowledge services, academic libraries, and research support platforms.
7. Conclusions
This study demonstrates that the proposed dual-stream embedding framework significantly enhances biomedical topic modelling by synergistically integrating unstructured textual data with structured metadata. Through systematic experimentation and rigorous ablation analysis, we confirmed that the observed improvements in TC and diversity are not merely attributable to increased input information but stem from the architecture’s structured, parameterized fusion of semantic and domain-specific knowledge. The fusion parameters (\(\alpha\) and \(\beta\)) allow fine-grained control over the contribution of each stream, and optimal values consistently yield higher-quality topics across multiple evaluation metrics.
Notably, the Dual PubMedBERT configuration achieved the highest combined topic quality (\(\mathrm{TC}+\mathrm{TD} = 1.3443\)), surpassing all baseline and ablation models. Even when the same input components (Title, Abstract, and MeSH) were used, single-stream models without fusion underperformed, reinforcing the necessity of structured integration. Furthermore, the incorporation of an LLM for post-hoc topic labelling significantly improved interpretability, transforming keyword lists into intuitive academic-style topic summaries suitable for downstream analyses.
Beyond its technical contributions, the framework is scalable, adaptable to other domains, and readily deployable in real-world retrieval systems, such as semantic search engines, thematic dashboards, and intelligent literature exploration tools. All source codes, processed datasets, and experimental configurations are openly available, enabling full reproducibility and supporting future extensions by the research community. These attributes position the framework as a robust and generalizable foundation for large-scale user-centered knowledge discovery systems across disciplines.
Data Availability The dataset used in this study was obtained from the publicly accessible PubMed database. The processed dataset and code supporting the findings of this study are available from the corresponding author upon reasonable request. All codes, processed datasets, and experimental results supporting the findings of this study are publicly available at https://github.com/taotao3614/Topic-Modeling-via-a-Dual-Stream-Embedding-Framework.
Conflicts of Interest The authors declare that they have no competing interests.
Funding This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.
Ethics Approval Not applicable. This study does not involve human participants, human data, or human tissue.
Acknowledgments
The authors would like to acknowledge the use of the AI language model ChatGPT for assistance in improving the clarity and fluency of the manuscript’s English writing. All intellectual content, analyses, and interpretations presented in this work remain solely those of the authors. The use of AI tools complied with the journal’s policies on AI-assisted writing.
- [1] Q. Jin, R. Leaman, and Z. Lu, “PubMed and beyond: biomedical literature search in the age of artificial intelligence,” eBioMedicine, Vol.100, Article No.104988, 2024. https://doi.org/10.1016/j.ebiom.2024.104988
- [2] H. C. Huurdeman and J. Kamps, “Designing Multistage Search Systems to Support the Information Seeking Process,” W. T. Fu and H. van Oostendorp (Eds.), “Understanding and Improving Information Search,” pp. 113-137, Springer, 2020. https://doi.org/10.1007/978-3-030-38825-6_7
- [3] D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent Dirichlet allocation,” J. of Machine Learning Research, Vol.3, pp. 993-1022, 2003.
- [4] D. M. Blei, “Probabilistic topic models,” Communications of the ACM, Vol.55, Issue 4, pp. 77-84, 2012. https://doi.org/10.1145/2133806.2133826
- [5] S. Kavvadias, G. Drosatos, and E. Kaldoudi, “Supporting topic modeling and trends analysis in biomedical literature,” J. of Biomedical Informatics, Vol.110, Article No.103574, 2020. https://doi.org/10.1016/j.jbi.2020.103574
- [6] H. Kilicoglu, “Biomedical text mining for research rigor and integrity: tasks, challenges, directions,” Briefings in Bioinformatics, Vol.19, Issue 6, pp. 1400-1414, 2018. https://doi.org/10.1093/bib/bbx057
- [7] T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient Estimation of Word Representations in Vector Space,” arXiv:1301.3781, 2013. https://doi.org/10.48550/arXiv.1301.3781
- [8] A. B. Dieng, F. J. R. Ruiz, and D. M. Blei, “Topic Modeling in Embedding Spaces,” Trans. of the Association for Computational Linguistics, Vol.8, pp. 439-453, 2020. https://doi.org/10.1162/tacl_a_00325
- [9] D. Angelov, “Top2Vec: Distributed Representations of Topics,” arXiv:2008.09470, 2020. https://doi.org/10.48550/arXiv.2008.09470
- [10] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol.1 (Long and Short Papers), pp. 4171-4186, 2019. https://doi.org/10.18653/v1/N19-1423
- [11] I. Beltagy, K. Lo, and A. Cohan, “SciBERT: A Pretrained Language Model for Scientific Text,” Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing and the 9th Int. Joint Conf. on Natural Language Processing (EMNLP-IJCNLP), pp. 3615-3620, 2019. https://doi.org/10.18653/v1/D19-1371
- [12] J. Lee, W. Yoon, S. Kim, D. Kim, S. Kim, C. H. So, and J. Kang, “BioBERT: A pre-trained biomedical language representation model for biomedical text mining,” Bioinformatics, Vol.36, Issue 4, pp. 1234-1240, 2020. https://doi.org/10.1093/bioinformatics/btz682
- [13] M. Grootendorst, “BERTopic: Neural topic modeling with a class-based TF-IDF procedure,” arXiv:2203.05794, 2022. https://doi.org/10.48550/arXiv.2203.05794
- [14] Y. Gu, R. Tinn, H. Cheng, M. Lucas, N. Usuyama, X. Liu, T. Naumann, J. Gao, and H. Poon, “Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing,” ACM Trans. on Computing for Healthcare, Vol.3, Issue 1, Article No.2, 2022. https://doi.org/10.1145/3458754
- [15] N. Reimers and I. Gurevych, “Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks,” arXiv:1908.10084, 2019. https://doi.org/10.48550/arXiv.1908.10084
- [16] Z. Yu, E. Bernstam, T. Cohen, B. C. Wallace, and T. R. Johnson, “Improving the utility of MeSH® terms using the TopicalMeSH representation,” J. of Biomedical Informatics, Vol.61, pp. 77-86, 2016. https://doi.org/10.1016/j.jbi.2016.03.013
- [17] D. Mimno, H. Wallach, E. Talley, M. Leenders, and A. McCallum, “Optimizing Semantic Coherence in Topic Models,” Proc. of the 2011 Conf. on Empirical Methods in Natural Language Processing, pp. 262-272, 2011.
- [18] O. Lezhnina, “Depression, anxiety, and burnout in academia: Topic modeling of PubMed abstracts,” Frontiers in Research Metrics and Analytics, Vol.8, Article No.1271385, 2023. https://doi.org/10.3389/frma.2023.1271385
- [19] Y. Wang, R. Benavides, L. Diatchenko, A. V. Grant, and Y. Li, “A graph-embedded topic model enables characterization of diverse pain phenotypes among UK biobank individuals,” iScience, Vol.25, Issue 6, Article No.104390, 2022. https://doi.org/10.1016/j.isci.2022.104390
- [20] M. Röder, A. Both, and A. Hinneburg, “Exploring the Space of Topic Coherence Measures,” Proc. of the 8th ACM Int. Conf. on Web Search and Data Mining, pp. 399-408, 2015. https://doi.org/10.1145/2684822.2685324
This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.