single-jc.php

JACIII Vol.30 No.6 pp. 1799-1815
(2026)

Review:

Vertical Domain Large Language Models: Applications, Challenges, Future Prospects

Yanling Chen*, Wujun Wang*, Jiyang Peng*, Jifa Xu*, Kang Jia**, Mengyao Liu*,†, and Zhongyun Tian*

*Inspur Electronic Information Industry Co., Ltd.
1036 Caoshanling East Road, Jinan, Shandong 250000, China

†Corresponding author

**Institute of Frontier and Interdisciplinary Science, Shandong University
72 Binhai Road, Qingdao, Shandong 260000, China

Received:
January 13, 2026
Accepted:
June 8, 2026
Published:
November 20, 2026
Keywords:
large language models, vertical domain applications, RAG, fine-tuning, agent
Abstract

Large language models (LLMs) have become a cornerstone of modern generative artificial intelligence, demonstrating remarkable performance across a wide range of tasks, including machine translation, question answering, and dialogue generation. However, most existing reviews primarily emphasize theoretical foundations and training methodologies, and offer limited insights into real-world application performance and domain-specific deployment patterns. To address this gap, we conducted a systematic literature review following PRISMA guidelines, synthesizing the application of LLMs across diverse vertical domains. The review examines system architectures, technical approaches, and performance benchmarks reported in the literature. In addition, it analyzes core challenges associated with LLM deployment, such as hallucinations, explainability, value alignment, and data quality, along with targeted solutions. Finally, it identifies emerging research opportunities and future directions, contributing to a more coherent, application-oriented understanding of LLM development and impact.

Outline structure of the article

Outline structure of the article

Cite this article as:
Y. Chen, W. Wang, J. Peng, J. Xu, K. Jia, M. Liu, and Z. Tian, “Vertical Domain Large Language Models: Applications, Challenges, Future Prospects,” J. Adv. Comput. Intell. Intell. Inform., Vol.30 No.6, pp. 1799-1815, 2026.
Data files:

1. Introduction

Amid rapid advances in artificial intelligence (AI), large language models (LLMs) have emerged as major drivers of technological innovation and breakthrough 1. Among them, ChatGPT 2, developed by OpenAI, has demonstrated remarkable natural language processing (NLP) capabilities through training on vast volumes of text and conversational data. It is capable of engaging in fluent multilingual conversations, accurately answering complex questions, and performing a wide range of professional tasks, such as content creation, planning, and code generation. The release of ChatGPT sparked global interest in LLM development, prompting major technology companies to enter the field and triggering a highly competitive “hundred-model race.” Over the past two years, progress in this field has accelerated rapidly. Models such as OpenAI’s ChatGPT series 3, Anthropic’s Claude series 4, and Google’s Gemini series 5 have been introduced, driving the field into an intensive phase of practical application and exploration. With continued advancements, leading global technology firms are shifting their focus from general large models (GLMs) to specialized development in domain-specific vertical applications. They are moving beyond parameter scaling and performance tuning to satisfy real-world application requirements. Unlike GLMs, which offer broad language understanding, vertical domain large language models (VDLLMs) are tailored to industry-specific demands and excel at adapting deeply to domain contexts for users.

Against this backdrop, to systematically review the current status, application logic, and technical framework of LLMs in vertical domains, this study presents a systematic literature review, aiming to achieve the following three main research objectives:

  1. 1.

    Examine how current VDLLMs integrate with real-world applications, clarify the nature of their connections, and identify key findings and controversies in existing research.

  2. 2.

    Assess how well LLMs integrate and adapt to technical systems, architectures, and tools in various vertical domains, summarize common integration patterns, and identify key structural limitations and implementation barriers.

  3. 3.

    Identify research gaps that impede a coherent methodological framework for VDLLMs across their application lifecycle, analyze key technical challenges, bottlenecks, and theoretical shortcomings, and provide actionable guidance for future research.

figure

Fig. 1. Overall structure of the paper.

Therefore, this review was guided by the following four research questions. Each question aimed to achieve one or more research objectives. RQ1 served the first objective, RQ2 and RQ3 supported the second objective, and RQ4 targeted the third objective.

  1. ・

    RQ1: What technologies, tools, and frameworks support the application of VDLLMs in various fields from 2020 to 2025?

  2. ・

    RQ2: How are models managed, architectured, and integrated using tools in different vertical domains?

  3. ・

    RQ3: What technical, organizational, and strategic challenges limit the application of VDLLMs?

  4. ・

    RQ4: How can future research directions help enhance the customized deployment and use of VDLLMs?

This study provides a comprehensive overview of the research on LLM in vertical domains through a systematic review based on the preferred reporting items for systematic reviews and meta-analyses (PRISMA) method, covering academic literature from 2020 to 2025. The findings are summarized using a comprehensive analytical framework and cross-industry case studies. The structure of this review is illustrated in Fig. 1. Section 2 describes the data preparation process for this study. Section 3 establishes the conceptual background and introduces the core technical foundations and architectures of LLMs and NLP. Section 4 systematically compares the existing research and clarifies the key research gaps that this study addresses. Section 5 explores core methodologies, including model training, model fine-tuning, base-model selection, knowledge distillation, and multi-agent system design. Section 6 discusses the evaluation and benchmarking strategies. Section 7 presents an analysis of the main challenges and mitigation measures. Finally, Section 8 summarizes the study and outlines future research directions.

2. Data Preparation

Following the clarification of the motivation and scope, we describe the methodology used to gather relevant literature for this analysis. This review draws on a systematic and extensive search conducted across several reputable databases, such as Web of Science and arXiv, complemented by additional searches using Google Scholar. To identify pertinent studies, we used various combinations of keywords, including “large language models,” “artificial intelligence,” and “autonomous research,” targeted at core application areas such as scientific research, healthcare, education, finance, and agriculture. The collected publications ranged from 2020 to 2025, a timeframe that aligns with the swift development and growing deployment of LLMs in specialized domains.

Before 2020, progress in NLP and knowledge-driven AI laid the groundwork for LLM application in vertical domains. LLMs such as BERT 6 use large-scale pretraining to improve performance in named entity recognition, relation extraction, and question answering, proving effective in general tasks and enabling adaptation to finance, law, healthcare, and industry. Integrating domain-specific knowledge graphs 7, particularly in finance and healthcare, highlights the value of structured reasoning for scientific discovery and decision support, offering a conceptual pathway for embedding domain knowledge into LLMs. However, early LLMs were mostly static and lacked reasoning, multi-agent coordination, and tool use, limiting their use in complex tasks such as financial risk assessment, multimodal medical analysis, and industrial fault diagnosis. Since 2023, LLMs, such as GPT-4, Claude 3.5, and Gemini 1.5 Pro, have advanced multimodal reasoning, long-context understanding, and function calling, enabling their integration with tools such as financial analytics and medical imaging systems. With improved instruction following and self-reflection, modern LLMs have evolved from passive tools to active systems capable of planning and cross-task collaboration, becoming core components in industrial applications. Currently, LLMs are used in customer service, clinical support, compliance auditing, and R&D assistance. This review focuses on research conducted from 2020 to 2025. The screening process involves two stages: first, non-academic content, such as news, blogs, and popular science, is excluded to ensure rigor; second, studies on “vertical applications of LLMs” are selected if they involve task planning, multi-agent coordination, or tool use in real-world settings. Priority was given to research with empirical results, such as improved accuracy or efficiency, or system innovations, such as fine-tuning methods and agent architectures, which provide a solid basis for assessing the progress and trends of domain-specific LLMs.

Instead of striving for a comprehensive list, this review focuses on a carefully selected set of high-impact representative studies that reflect the latest advancements and evolving directions for the application of LLMs within specific domains. The search and screening process followed the PRISMA 2020 guidelines and is illustrated in Fig. 2. A total of 3,836 articles were initially identified from Web of Science, arXiv, and Google Scholar. After removing 1,624 reviews, news reports, blog posts, and duplicate records, 2,212 unique records remained. These records were screened by title and abstract, and 1,376 studies that did not meet the inclusion criteria were excluded. After a second round of filtering based on research scope and core themes, a final sample of 632 peer-reviewed full-text articles was included in the analysis.

figure

Fig. 2. Overview of the literature search and screening process, which is applicable to the vertical domain large language model applications included in this review.

3. Background

After describing the data preparation and basic preliminaries above, this section establishes the conceptual foundation for the research by introducing NLP and LLMs, outlining the core architectures and representative models of LLMs, and providing a systematic basis for understanding their methodological frameworks.

3.1. Natural Language Processing

NLP 8 is a key research area within AI that employs computational techniques to deeply analyze and understand human language, thereby enabling the effective processing of linguistic information. The field primarily comprises two core components: natural language understanding (NLU) 9 and natural language generation (NLG) 10. NLU focuses on equipping machines with the capability to interpret the semantics, logical structure, and contextual nuances of human language, thereby allowing for accurate comprehension and appropriate responses to natural language inputs. In contrast, NLG involves transforming structured, nonlinguistic data, such as information from databases or visual charts, into coherent and contextually appropriate textual outputs that align with human communication patterns, thereby facilitating seamless interactions between AI systems and users. NLP has laid the foundation for the application of LLMs in vertical fields.

3.2. Large Language Models

Building on the fundamentals of NLP, LLMs represent the most advanced paradigms in modern language understanding and generation.

Pretrained language models (PLMs) 11 are neural network architectures trained on large-scale corpora that enable representations learned through general-purpose tasks to be effectively transferred to a wide range of downstream applications. The fundamental principle of this approach is to initialize model parameters with rich linguistic patterns embedded in massive datasets, followed by adaptation to specific tasks through fine-tuning or transfer learning techniques, thereby enhancing both model performance and training efficiency.

LLMs 12 represent an advanced evolution of PLMs, typically leveraging the Transformer architecture and scaling to hundreds of billions or even trillions of parameters, as exemplified by models such as PaLM 13, LLaMA 14, and GPT-4 15. In contrast to conventional PLMs, LLMs differ not only in their significantly larger parameter scales and vastly expanded training corpora, which lead to substantially enhanced language understanding and generation capabilities, but also, more critically, in the emergence of novel abilities when the scale exceeds a certain threshold. Specifically, LLMs exhibit “emergent capabilities” 16 that are absent in smaller models, arising from the complex interactions among model scale, model architecture, and data diversity.

3.3. Large Model Architecture

To understand how LLMs achieve such powerful capabilities, it is essential to examine their fundamental architectural design.

As deep learning advances, traditional models, such as convolutional neural networks 17 and recurrent neural networks (RNNs) 18, have shown limitations in processing sequential data. RNNs, which process tokens sequentially, struggle to capture the global context and limit computational parallelism. To address this, Attention Is All You Need introduced the Transformer architecture 19 based on an encoder–decoder framework that uses self-attention to model full-sequence dependencies. Self-attention allows each token to attend to all others, enabling effective long-range dependency modeling. Multi head attention improves the representation by computing attention across multiple subspaces in parallel and capturing complex semantic patterns. The Transformer framework includes three main variants: encoder-only, decoder-only, and encoder–decoder, as shown in Fig. 3, with a comparison in Table 1. These differ primarily in their attention mask configurations: encoder-only uses bidirectional attention for full context understanding; decoder-only applies autoregressive masking to ensure that predictions rely only on prior tokens, supporting causal language modeling; encoder–decoder incorporates cross-attention, allowing the decoder to attend to encoded inputs and coordinate generation with encoding.

figure

Fig. 3. Diagram of the Transformer architecture 19.

Table 1. Comparison among the three architectures of LLMs.
Architecture type Representative model Architecture features Primary applications
Encoder-only

BERT 20,

RoBERTa 21,

ALBERT 22

Composed of multiple encoders; excels at understanding text semantics and contextual relationships. NLU tasks: text classification, question-answering systems, sentiment analysis.
Decoder-only GPT 15, BLOOM 23, LLaMA 24 Composed of multiple decoders; excels at text generation with contextual coherence. Text generation tasks: open-domain text generation, dialogue systems, creative story creation.
Encoder–decoder

T5 25, mT5 26,

BART 27

Hybrid structure with encoder (input processing) and decoder (output generation); balances understanding and generation. Cross-lingual translation, abstractive text summarization, text-to-text transformation.
Table 2. Typical applications of LLMs across science research, medicine, education, finance, and agriculture domains.
Domain Typical cases Base model Core application scenarios
Science research Xiwu 32

LLaMA, LLaMA2,

Vicuna

A specialized LLM for HEP tasks, including question and answering, code generation, and instant knowledge updating.
Mephisto 33 GPT-4o, DeepSeek, Qwen A multi-agent LLM framework that emulates human reasoning for astronomical analysis, interfaces with SED models like CIGALE.
Chemma 34 LLaMA2-7B A finely tuned LLM for organic synthesis, excelling at retrosynthesis and yield prediction.
Medicine Med-PaLM 35 PaLM 2 Achieves state-of-the-art medical question answering performance, with answers clinically preferred over physicians.
MedChat 36 GPT-4.1 Multi-agent diagnostic framework combining specialized LLM agents, reduces hallucination risks.
LungDiag 37

BERT,

LightGBM

AI-powered NLP system for multi-disease respiratory diagnosis using EHRs; outperforms clinicians.
Education SimClass 38 GLM-4 Multi-agent classroom simulation framework exploring dynamic education scenarios.
BIDTrainer 39 GPT-4 LLM-driven educator with structured ontology, reasoning hints, and benchmarking assessment.
Finance BloombergGPT 40 BLOOM Specialized 50B LLM for finance with mixed data training and strong NLP competence.
InvestLM 41 LLaMA-65B Expert-level investment advice with strong generalization and long context handling.
Agriculture AgriGPT 42 Qwen3-8B High-quality Agri dataset; Tri-RAG enhances reasoning; comprehensive evaluation.
AgroLLM 43 Gemini-1.5, Mistral-7B Agriculture-specific chatbot using RAG for accurate educational farmer support.

4. Comparison with Existing Work

We reviewed several existing studies focusing on the application of LLMs in vertical domains.

This review explores the integration techniques and deployment methods of LLMs in vertical domains to enhance the efficiency and completeness of VDLLMs and provides technical guidance for their application in other fields. As the reliance on LLMs in vertical domains grows, issues such as model efficiency, data privacy, adversarial attacks, and hallucinations have received significant attention.

Several recent surveys have addressed different aspects of LLMs relevant to our work. First, systematic technical surveys examine methodological advances; for example, Lei et al. 28 offered a comprehensive overview of parameter-efficient fine-tuning techniques, detailing their implementation strategies and advantages in large-scale pretrained models. Second, growing research focuses on specialized NLP applications; notably, Gao et al. 29 reviewed recent progress in retrieval-augmented generation (RAG), providing practical insights into optimizing the retriever and generator components and their integration across stages. Third, emerging work emphasizes empirical validation in vertical domains; for instance, Vente et al. 30 showed that LLMs can support robust glaucoma screening in clinical settings, with a performance close to that of human experts. Fourth, increasing attention is given to rigorous evaluation frameworks; for example, Kumar and Mishra 31 systematically evaluated current LLM assessment methods through the lenses of reliability and robustness, laying a theoretical foundation for future improvements.

By carefully examining and summarizing these studies, we conclude that there is currently no review specifically dedicated to the application of LLMs in various vertical domains. Although some studies have explored the application of LLMs in specific domains, none has explicitly analyzed the application of LLMs in a comprehensive and integrated domain context. This prompted us to explore how the application of LLMs in vertical domains can enhance their reliability and provide a referential methodology.

5. VDLLM Methodology

Following the establishment of the foundational literature and conceptual background, we elaborate on the technical foundations and methodological frameworks for deploying LLMs in vertical domains. To demonstrate the practical implementation of VDLLMs built on diverse base models, Table 2 presents representative applications spanning five key vertical fields: scientific research, medicine, education, finance, and agriculture. For each application, we specify the developed system, its underlying base model, and core functional scenarios, illustrating how generic LLMs are tailored, adapted, and optimized to fulfill domain‑specific task requirements. To further elucidate the characteristics, differences, and tradeoffs of these core methodologies, a comprehensive comparison is presented in Table 3, which covers the core principles, advantages, limitations, and typical application domains.

Table 3. Comprehensive comparison of methodologies for vertical domain large language models.
Category Method Core principle Advantages and limitations Typical domains
Model training Full training Pretraining, SFT, RM, RL Strong domain fitting; high performance but extremely costly Healthcare, finance
MoE training Sparse gating; activate partial experts Efficient and scalable; unstable gate mechanism Multimodal medicine
Base models Pre-trained LLMs Use open/closed foundation models Stable and low-threshold; weak domain adaptation All vertical fields
Knowledge integration RAG Retrieve external knowledge for generation Reduces hallucinations; depends on retrieval quality Healthcare, law, finance
Fine-tuning Full fine-tuning Update all parameters with domain data Deep reasoning; risks catastrophic forgetting Scientific research
LoRA Update low-rank matrices only Lightweight and stable; slightly weaker expressiveness Most industries
Prompt tuning Optimize soft prompt embeddings Ultra-low cost; poor on complex tasks Education, simple question answering
Emergent ability CoT / ICL Stimulate reasoning via prompts Expert-level logic; uncontrollable behavior Legal, clinical decision
Model compression Knowledge distillation Transfer knowledge to small student model Edge-deployable; inevitable performance loss Agriculture, mobile health
Agent architecture Single agent Reason \(+\) act \(+\) memory Low latency; limited to simple tasks Customer service
Multi-agent Collaborative decision-making Handles complex tasks; high communication overhead Rare disease diagnosis, legal trial

5.1. Model Training

The construction of LLMs typically involves a four-stage process 44: pretraining 45, supervised fine-tuning (SFT), reward modeling (RM), and reinforcement learning (RL). During pretraining, models are trained on vast amounts of unlabeled text to acquire foundational linguistic knowledge, including syntactic structures, grammatical rules, and semantic representations. In the SFT stage, the model is further refined using high-quality task-specific labeled data, thereby enhancing its performance in the targeted applications. Subsequently, in RM, a reward function is learned from human feedback to evaluate the quality of the generated responses. Finally, in the RL stage, this reward model guides policy optimization through reinforcement learning, enabling the language model to generate more accurate, coherent, and contextually appropriate outputs.

During the model training process, large-scale corpora are cleaned and structured to improve data quality and diversity. Language modeling captures statistical linguistic features, whereas denoising autoencoding enhances semantic understanding and jointly encodes linguistic rules and domain knowledge into model parameters to support subsequent stages. Next, efficient parameter optimization is applied. Large-scale batch training improves computational efficiency and is combined with dynamic batch size scheduling, starting small and increasing as the stability improves, to maintain precision and avoid oscillations. Common optimizers include Adam 46, AdamW 47, and Adafactor 48, which adaptively adjust learning rates to accelerate convergence and ensure stable updates. As LLMs grow in parameters and data scales, training faces challenges owing to limited computational resources and memory efficiency. To overcome these, techniques such as ZeRO 49, 3D parallelism, and mixed-precision training 50 have been used to enable efficient and stable training. The model training process differs slightly across domains: medicine prioritizes privacy-preserving techniques (e.g., federated learning) for sensitive EHR data; finance emphasizes stability and interpretability in reward modeling to align with regulatory demands; and agriculture focuses on adapting to unstructured data, such as crop images and environmental sensor data.

figure

Fig. 4. MoE framework.

figure

Fig. 5. Process of domain-specific LLM building.

Table 4. Common basic LLMs.
Base models Model parameter Token Context length
LLaMA3 24 8B/70B 15T+ 8K
Baichuan2 54 7B/13B 1.2T/2.6T 4K
Qwen3 55 0.6B/1.7B/4B/8B/14B/32B 36T 32K/128K
ChatGLM3 56 6B/9B 1T/1.4T 2K/32K/8K/128K/1M
InternLM2 57 1.8B/7B/20B 1.6T 8K
Falcon 58 7B/40B 1.5T 2K

In recent years, the mixture of experts (MoE) 51 paradigm has shown significant advantages in terms of efficient pretraining. As shown in Fig. 4, the MoE core architecture comprises two key components. First, the sparse MoE layer 52 replaces the standard feed-forward network in Transformer blocks and contains multiple independent expert networks, typically lightweight FFNs, that process specific input types. These methods can be extended to hierarchical or complex structures. Its key innovation is dynamic selection; only a small subset of experts is activated per input token, enabling the efficient use of computational resources. Second, the gating network routes each token by generating a probability distribution across the experts. Depending on the task, tokens are assigned to one or more experts through soft or hard selection, which enables dynamic input-to-expert mapping. This allows the model to activate specialized units based on semantic features, preserving its capacity while minimizing redundant computations. For instance, the medical multimodal LLM Med-MoE 53, using this sparse activation, achieved 82.72%–91.98% accuracy in medical visual question answering and 58.55%–85.06% recall on open-ended questions, outperforming LLaVA-Med, while using only 30%–50% of a conventional Transformer’s parameters and significantly reducing training and inference costs.

Beyond the general training pipeline, the choice of base model plays a critical role in the development of effective VDLLMs.

5.2. Base Models

General LLMs are widely used in common NLP tasks such as text generation, translation, and sentiment analysis. VDLLMs, on the other hand, focus on specific industries and provide more specialized services by learning from domain-specific data. Currently, the construction of VDLLMs, as shown in Fig. 5, is usually based on PLMs for optimization, and further fine-tuning and alignment with high-quality domain data after cleaning to enhance their understanding and generation capabilities in specific scenarios. However, owing to the high cost of pretraining, most applications directly adopt existing base models and reduce the deployment threshold through fine-tuning. The common base models are listed in Table 4.

However, the application of base models faces knowledge gaps and accuracy issues. Therefore, in addition to the base models, techniques such as retraining, fine-tuning, tool usage, multimodal processing, and agents have been used to further enhance the functionality of VDLLMs. To enable LLMs to operate effectively and reliably in the vertical domain, researchers have adopted various key technologies to augment their capabilities and overcome their inherent limitations.

5.3. Retrieval-Augmented Generation

The core idea of RAG 59 is to effectively integrate language generation models with external knowledge sources. Although traditional LLMs exhibit strong generative capabilities across diverse tasks, their internal knowledge is constrained by the temporal limits of the training data, making it difficult to meet evolving information needs. Therefore, RAG addresses two core limitations of general LLMs: the obsolescence of knowledge owing to static pretraining and insufficient coverage of specific concepts in vertical domains. RAG enables LLMs to dynamically access the latest task-relevant information during the inference process, allowing them to generate contextually accurate and verifiable responses, thereby enhancing the accuracy and relevance of the output.

As shown in Fig. 6, the pipeline consists of three steps. First, in the indexing phase, documents are split into segments and converted into fixed-dimensional vectors using embedding techniques such as Text2Vec, enabling efficient semantic matching. The RAG chunking logic differs across domains; healthcare uses smaller chunks to focus on specific clinical details and citations, whereas finance adopts larger chunks to retain contextual relationships in market reports and regulatory documents. Next, the user query is encoded into a vector and matched against the vector database to retrieve the most contextually relevant content. Finally, the LLMs use the retrieved information as a supplementary context to generate more accurate, informative, and concise responses.

For instance, in the biomedical field, researchers often rely on RAG not only for evidence retrieval but also for tool integration. For example, SourceCheckup uses RAG to track whether statements generated by LLMs are supported by citations 60, thereby enabling citation-level fact-checking in medical quality assurance systems. Overall, RAG provides a scalable mechanism to reduce hallucinations, enhance credibility, and ensure the traceability of citations for LLMs in vertical domains.

5.4. Model Fine-Tuning

figure

Fig. 6. Workflow of RAG.

RAG expands the knowledge access scope of LLMs, whereas fine-tuning customizes their core behavior for specific domain reasoning. Although general LLMs demonstrate impressive language capabilities, their performance in specific vertical domains is often limited by domain knowledge and contextual understanding. Fine-tuning bridges this gap by adapting pre-trained models to specific domain tasks through additional supervised or instruction-based training on curated corpora. This process enhances the model’s understanding of reasoning patterns and task-specific discourse 61.

SFT is a core approach that includes instruction tuning 62, reinforcement learning from human feedback (RLHF) 63, and parameter-efficient fine-tuning (PEFT) 64. Instruction tuning uses datasets of instructions and their corresponding outputs to enhance the model’s ability to understand and follow task-specific directions. RLHF uses human-provided quality scores or rankings as rewards and applies reinforcement learning to optimize model outputs. PEFT methods, such as prompt tuning 65 and LoRA 66, update only a small fraction of the parameters. Prompt tuning reformulates inputs as cloze-style “prompts” to bridge the semantic gap between models and human reasoning, allowing high-quality outputs without full parameter updates and making efficient use of limited labeled data. Examples include Prefix Tuning 67 and P-Tuning 68. LoRA adjusts weight parameters via low-rank matrices, efficiently acquiring task-specific knowledge while preserving general capabilities and significantly reducing computational and parameter costs during fine-tuning 66. Domain-specific fine-tuning strategies exhibit distinct characteristics: medicine leverages PEFT to preserve general medical knowledge during adaptation to specialized subfields (e.g., respiratory diagnosis); finance prioritizes RLHF to ensure that model outputs conform to regulatory compliance; and education centers on instructional tuning to boost the capacity for pedagogical reasoning.

Fine-tuning is often combined with RAG to enhance robustness. For instance, as a flexible and learnable LLM tailored for high-energy physics (HEP), Xiwu 69 addresses the critical issue of severe hallucinations in general LLMs (e.g., GPT‑4) when handling professional terminology and experimental code generation. To support researchers’ demands for domain-specific question answering, experimental code generation, literature analysis, and physics knowledge interpretation, researchers constructed a dedicated domain data engine covering eight HEP subfields. Using seed fission, highly cited papers, paper abstracts, and real user queries, they built a high-quality dataset comprising 750 million tokens for pretraining and 26 thousand question-answer pairs for fine-tuning. The model adopts a switchable foundation model architecture compatible with LLaMA, LLaMA2, Vicuna, ChatGLM, and Grok‑1 and implements a dual-learning mechanism: just-in-time learning based on RAG vector storage for low-cost knowledge updates and on-the-fly fine-tuning optimized with LoRA, mixed-precision training, FlashAttention, and FSDP for efficient domain adaptation. Deployed via WebUI and API on the HepAI platform with open-source code, Xiwu‑13B achieves a 95% win/draw rate over the baseline Vicuna‑13B in HEP question answering and delivers 65% of the performance of ChatGPT‑175B with substantially fewer parameters. It accurately interprets professional terms, such as HEPS and BOSS, and generates valid BOSS decay card codes that general models fail to produce, thereby reducing repetitive workloads and significantly accelerating the HEP experimental design and data analysis. However, fine-tuning presents brings several challenges. The most significant issues include the scarcity of high-quality annotated domain data and the risk of catastrophic forgetting, in which LLMs lose their previously acquired general knowledge during task-specific training 70. Addressing these problems requires continuous learning strategies, domain-aware data management, and hybrid training processes that maintain generalization while focusing on domain-specific knowledge.

5.5. Emergent Ability

The emergent capabilities of large-scale LLMs are centered on the qualitative leap that occurs when the scale crosses a critical threshold 16. The significant gap and nonlinear emerging characteristics of these capabilities cannot be predicted using the performance trends of small-scale LLMs 71. This feature aligns highly with the core demands for specialized capabilities in vertical fields, such as health, finance, law, and education, where models need to possess precise understanding, deep reasoning, and professional adaptation beyond general foundational capabilities. Such capabilities are difficult to achieve through gradual optimization of small-scale LLMs. Only by relying on emergent capabilities can models break through performance bottlenecks in vertical scenarios and demonstrate task-processing levels that meet industry standards. Moreover, by studying the impact of model quantization on emergent phenomena, triggering emergent behaviors through in-context learning (ICL) 72, and actively stimulating the emergent capabilities of LLMs through strategies such as chain-of-thought (CoT) 73 and instruction fine-tuning, the application of large-scale LLMs in various vertical fields can be further promoted. This has become a key technical path for the construction and optimization of LLMs in vertical domains.

For instance, in the legal field, the model must possess the ability to accurately interpret legal provisions, deeply reason the logic of case judgments, and adapt appropriately to judicial documents. The realization of these capabilities depends on the emergent abilities of LLMs. AgentCourt 74 can intelligently simulate a court system driven by ERNIESpeed, and through ICL and CoT, it can stimulate emergent legal adversarial reasoning capabilities. It simulates the entire court process and provides a training platform for legal practitioners. Within a week, thousands of court cases could be simulated, significantly enhancing the efficiency and practical skills of legal training.

5.6. Knowledge Distillation

Application of knowledge distillation technology is necessary for the construction and deployment of LLMs in the vertical domain. Vertical domains often have unique data distributions and application scenarios that are sensitive to computing resources and deployment costs. By transferring the complex knowledge of PLMs to lightweight student models, it is possible to retain the adaptability of the LLM to vertical field data and achieve high performance while significantly reducing the computational cost and deployment threshold of the LLM, thus meeting the inference requirements of edge devices or low-computing-power environments 75. Moreover, by introducing strategies such as multi-teacher distillation or self-distillation, the generalization ability of the student model can be further enhanced to better adapt to the specific task requirements of vertical fields. The application of knowledge distillation in LLMs of vertical fields can follow the following approach: use a PLM adapted to the vertical domain as the teacher model and leverage its output soft label distribution, intermediate layer features, and attention weights, among other rich supervision signals, combined with labeled data in the vertical field to construct a combined loss function, guiding the more compact student model to learn the inference logic and knowledge representation of the teacher model. At the same time, it can also be combined with the task characteristics of the vertical field, such as KDRL 76, by considering the processing requirements of complex data, such as remote sensing images and medical images, integrating knowledge distillation with RL and other technologies, and designing a multi-stage training framework to enhance the adaptability of the distilled model to the complex data distribution in the vertical field, ultimately obtaining an LLM dedicated to the vertical field that combines performance and efficiency.

5.7. Agent Architecture

LLM agent systems are classified into single-agent and multi-agent systems (MAS) based on the agent count and collaboration mode. Single-agent systems enhance the intelligence, autonomy, and situational reasoning of a single foundational model through integrated mechanisms. For example, CoT enables step-by-step reasoning; ReAct 77 improves accuracy by combining reasoning with tool use, and Tree-of-Thought 78 explores multiple reasoning paths hierarchically to select the best solution, enabling explainable planning across domains. To address context window limits, memory mechanisms 79 are used: short-term memory tracks current task states, long-term memory retrieves domain knowledge, and RAG adds real-time information from external sources. Self-correction uses internal or external feedback to refine the outputs and improve the robustness, transparency, and fault tolerance. In contrast, MAS emphasizes collaborative intelligence for complex tasks and simulates team-based decision-making 80. Architectures are either centralized, where a coordinator manages task decomposition, assignment, and communication, or decentralized, supporting peer-to-peer interactions though iterative refinement and rule-based protocols. Agents communicate through structured messages or shared memory/state 81. MAS enables distributed cognition, specialization, and coordinated problem-solving and is effective in multi-expertise tasks such as rare disease diagnosis 82 and cross-modal reasoning 83.

6. Performance Evaluation and Benchmarking

Building upon the comprehensive review of core VDLLM methodologies, this study shifts the focus to performance evaluation and benchmarking in real-world task settings. Given the sustained advancement of vertical domain LLMs, establishing a unified evaluation framework that can quantitatively characterize the model capability, operational robustness, and application security has emerged as an increasingly critical research direction.

6.1. Key Benchmarks of VDLLMs

With the extensive penetration of LLMs into various domains, benchmark tests for VDLLMs have become core tools for evaluating the adaptability and performance of LLMs in specific fields. These benchmark tests focus on the task requirements, application scenarios, security, and data privacy of specific domains and quantify the capability boundaries of VDLLMs in professional scenarios using standardized datasets and evaluation metrics. Table 5 presents different benchmarks in the medical and scientific research fields as well as the VDLLM benchmarks classified from different perspectives. The core feature of the VDLLM benchmark is its close alignment with the practical requirements of specific domains, which transforms the capabilities of general LLMs into quantifiable professional evaluation metrics. From compliance requirements in clinical medicine to the need for explainability in scientific research and from data security to the control of hallucinations in multimodal scenarios, these benchmarks not only provide standardized measurement tools for the performance of VDLLMs but also effectively promote the safe, reliable, and fair application of VDLLMs.

Table 5. VDLLM benchmark examples and benchmarks from different directions.
Domain/Perspective Core benchmarks Benchmark descriptions
Clinical healthcare ClinicalBench 84 Encompassing three common clinical prediction tasks, two databases, 14 general LLMs, 8 medical LLMs, and 11 traditional machine learning models.
Scientific research MLCommons-Science Benchmarks 85 It addresses fragmented and non-standardized scientific machine learning benchmarks by expanding the MLCommons ecosystem to fields including physics, chemistry, materials science, biology, and climate science. It integrates existing benchmarks like XAI-BENCH and FastML Science Benchmarks to build a unified classification system across scientific, application, and system levels.
Logical reasoning LogicVista 86 Focuses on multimodal logical reasoning, covering five types of reasoning. Uses diverse visual inputs to evaluate models’ ability to apply logical reasoning across visual data.
Causal reasoning CausalBench 87 Offering three tasks of varying difficulties, including correlation, causal skeleton, and causality identification; and enabling comparative evaluations of LLMs against traditional causal learning algorithms, and revealing LLMs’ strengths.
Multimodal MultiTrust 88 Focuses on five key aspects of trustworthiness, evaluates multimodal risks and cross-modal impacts, and provides a scalable evaluation toolbox.
Language understanding SuperGLUE 89 Includes several challenging NLU tasks, provides human performance baselines and diagnostic tools, and combines multi-task performance into a single score.
Safety BackdoorLLM 90 The first comprehensive benchmark for LLM text-generation backdoor threats, covering 8 attack strategies, 7 scenarios, 6 models and 7 defense techniques.

6.2. Quantitative Analysis

This section systematically incorporates the benchmark metrics and quantitative evaluations across the diverse domain-specific LLMs covered in the survey, including finance, scientific research, education, agriculture, and healthcare. For the financial domain, BloombergGPT 40 was quantitatively validated on public financial benchmarks (e.g., FPB, FiQA SA, ConvFinQA) and internal financial tasks, trained as a base decoder-only causal language model adopting mixed-domain pretraining (combining curated financial data and general-purpose corpora) with few-shot prompting as the core inference method, achieving an average performance of 62.51% on public financial tasks and 62.47% on internal sentiment analysis tasks, significantly outperforming comparable LLMs such as GPT-NeoX and OPT-66B. In the chemical scientific research domain, ChemLLM 91 was built on the InternLM2‑Base‑7B base model and trained via a two‑stage SFT pipeline, enhanced by LoRA for efficient adaptation and template‑based structured knowledge conversion to integrate chemical data. It was evaluated on ChemBench, which comprises 4,100 multiple-choice questions across nine core chemical tasks; it delivered performance comparable to GPT‑4 and surpasses GPT‑3.5 and other 7B‑parameter models, with general‑domain scores of 65.6% on MMLU and 67.2% on C‑Eval. For mathematics education, MathBERT demonstrated quantitative improvements of 1.2%–22% over prior state-of-the-art methods and 2%–8% over vanilla BERT on three key tasks (knowledge component prediction, autograding, and knowledge tracing), with an MLM accuracy exceeding 99.8% after pretraining. In Parkinson’s disease management, the AutoHealth 92 system achieved 95.4% accuracy for tremor and freezing of gait detection via the LightGBM classifier and 96.9% accuracy for voice tremor recognition using SVM, outperforming conventional machine learning models in motor symptom identification. For agricultural LLM applications, AgriGPT 42 attained 16.52% in BLEU, 44.06% in METEOR, 31.60% in ROUGE-1, 9.63% in ROUGE-2, 30.32% in ROUGE-L, and 23.20% in LLM-Score on the AgriBench-13K benchmark, surpassing general-purpose LLMs across factual correctness, logical consistency, and domain relevance. Additionally, the system maintained 44.15 token/s inference speed on a single RTX 4090 GPU, ensuring efficient edge deployment. The numerical evidence presented not only confirms the advantages of domain-specific models in terms of reliability, accuracy, and robustness but also highlights the critical role of targeted optimization strategies. Specifically, domain-specific tuning, RAG, multi-agent systems, and MoE structures have proven to be effective approaches for bridging the gap between generic LLMs and the unique demands of real-world vertical domain deployments. These strategies enable domain-specific LLMs to capture domain-specific knowledge, terminology, and task characteristics better, thereby reducing errors, improving response relevance, and enhancing practical applicability in professional settings.

6.3. Performance Comparison

The performance of VDLLMs depends heavily on their base models, which vary significantly in terms of their knowledge breadth, reasoning accuracy, and task adaptability. These core capabilities set the upper limit for VDLLM performance. In high-stakes domains such as biomedicine, where reasoning precision and knowledge reliability are critical, models built on advanced base models such as GPT-4 and Claude 3 Opus achieve high accuracy in clinical diagnosis simulations and expert-level question answering, sometimes matching human experts. Their strong reasoning enables multi-step analysis, whereas their high factual consistency reduces the risk of hallucinations and meets medical safety standards. For mission-critical applications like clinical decision support or financial planning, developers typically choose leading closed-source models to ensure robust foundational capabilities. This approach delivers strong out-of-the-box performance but involves higher licensing costs and limited customization. In contrast, open-source models such as Mistral 8x7B offer a cost-effective alternative for less critical or budget-constrained use cases. Through domain-specific fine-tuning and RAG, their performance can be enhanced to approach that of their closed-source counterparts. Moreover, their openness supports deep customization, enabling local deployment and adaptation to low-resource environments.

7. Challenges and Solutions

Despite the promising performance of existing benchmarks, VDLLMs, which have achieved remarkable success across diverse domains, still face several critical and persistent challenges that impede their real-world reliability, deployment, practical effectiveness, and long-term sustainability. These challenges span multiple dimensions, ranging from fact illusion and explainability to tool integration, data bias, and regulatory governance; each category further aligns with targeted mitigation strategies proposed in recent literature, which underscores the imperative of principle-based, risk-aware design for the advancement and application of such models.

Current LLMs have achieved remarkable success across diverse domains. However, their practical effectiveness and long-term sustainability have been hindered by a range of persistent challenges. These challenges span multiple levels from fact illusion and explainability to tool integration, data bias, and regulatory governance. Each category further corresponds to the mitigation strategies proposed in recent literature, emphasizing the necessity of principle-based risk-aware design.

7.1. Data Quality and Quantity

Chief among these challenges are the quality and quantity of training data, which remain critical bottlenecks for further performance improvements. Currently, most VDLLMs rely primarily on publicly available datasets that frequently embed societal and linguistic biases and fail to adequately capture the full diversity and complexity of real-world contexts. Particularly in specialized domains or industry-specific applications, the scarcity of high-quality domain-relevant data significantly constrains the model’s ability to generalize. Furthermore, insufficiencies in both the volume and quality of training data directly undermine learning efficacy and real-world performance, leading to accuracy levels that fail to meet practical deployment standards.

This challenge can be mitigated by enhancing the data quality and diversity. Priority should be given to promoting the development of data-augmentation technology and generating representative and diverse data samples through multiple channels. For instance, in the field of financial risk control, fraudulent transaction samples are exceedingly rare, and fraud patterns evolve dynamically over time. Generative models, such as variational autoencoders 93, can be employed to synthesize minority-class fraudulent transaction data, whereas adversarial sample generation can be leveraged to simulate emerging fraud attack patterns, thereby strengthening the defensive robustness of the model. High-quality, varied, and thoughtfully assembled datasets are the cornerstones of bias reduction 94. To address bias effectively, mitigation strategies must be embedded at every stage of data curation, from data acquisition to model training and assessment, using methods such as adversarial debiasing, synthetic data generation, and optimization techniques that prioritize fairness 95. For example, regularly evaluating model predictions across different demographic segments is essential, and engaging stakeholders during dataset creation plays a vital role in promoting equitable representation and just outcomes.

7.2. Hallucinations and Factual Inaccuracies

The problem of model hallucinations persists, with LLMs frequently generating content that includes factual inaccuracies, logical inconsistencies, or semantically incoherent outputs during text generation. Despite their strong ability to produce fluent and naturalistic language, these models often fail to consistently differentiate between verifiable facts and fabricated information, resulting in conclusions that lack grounding in empirical reality. This fundamental limitation significantly compromises the reliability and trustworthiness of LLMs in generative applications.

A key strategy for alleviating hallucinations involves the effective integration of a model’s generative capabilities with external knowledge sources. By leveraging the RAG technology, dynamically updated knowledge bases can be seamlessly incorporated to provide models with reliable factual grounding. This approach substantially reduces the generation of fictitious content and improves both the accuracy and consistency of model outputs. For instance, LegalBench 96 can assess the generative capabilities of LLMs in the legal domain and is a benchmark specifically designed to evaluate the retrieval steps of RAG pipelines in the legal field. LegalBench-RAG emphasizes precise retrieval and focuses on extracting the smallest and most relevant text segments from legal documents.

7.3. Explainability and Transparency

Deep learning models are often characterized by their “black-box” nature, which complicates the efforts to understand the internal mechanisms that drive decisions in LLMs 97. In medical applications, this lack of transparency can undermine the confidence of clinicians, limit real-world deployment, and raise concerns about responsibility and oversight 98.

To address these issues, several approaches have been proposed to enhance the model interpretability. Techniques such as LLMSHAP 99, LIME 100, and attention mapping help reveal how specific inputs influence model outputs. Transparency in reasoning can also be strengthened using systematic prompting methods 101, including CoT, ICL, and argumentation-based frameworks such as those in Layered-CoT 102. The incorporation of structured knowledge graphs enables models to base their inferences on explicit and interpretable semantic relationships 103. In addition, model cards offer a standardized way to document key aspects, such as training data sources, performance metrics, and known limitations. Interactive human-in-the-loop systems further promote transparency by enabling users to actively examine, question, and provide feedback on model-generated results 104.

7.4. Ethics, Privacy, Security, and Regulation

Moreover, the challenges of value alignment remain largely unaddressed. Most existing LLMs are trained on vast amounts of textual data that have not undergone systematic curation or normalization from a moral, ethical, or societal standpoint. Consequently, model outputs may exhibit biases, generate inappropriate content, or propagate information that contradicts mainstream societal values, thereby producing behaviors that are misaligned with human ethical and value expectations.

To promote the alignment of model behavior with human ethical principles, it is essential to strengthen the integration of ethical and moral constraints at the design level. For instance, training data curation should adhere to rigorous standards of legality, de-identification, and bias mitigation, with the explicit exclusion of sensitive information and discriminatory samples. Furthermore, robust content reviews and feedback mechanisms that incorporate well-defined moderation policies and tiered response protocols must be implemented. The systematic reintegration of feedback data into iterative training processes ensures that models consistently align with widely accepted social values and legal regulations during content generation, thereby minimizing the risk of producing harmful or misleading output 105.

7.5. Resources and Costs

However, the high costs associated with training and deployment present significant practical barriers. The development and real-world application of LLMs require substantial computational resources and storage capacity, resulting in prohibitively high costs for many institutions and research teams. As model parameter scales continue to grow, the demand for computational power and infrastructure investment increases exponentially, severely impeding broad adoption and long-term sustainable advancement.

One solution is to develop compact models, and techniques such as knowledge distillation and quantization can be used to transfer the knowledge of large-scale LLMs to lightweight models 1. This approach maintains a high task performance while substantially reducing resource consumption. Furthermore, designing more optimized model architectures coupled with software-hardware co-optimization will facilitate the development of practical and sustainable models 106.

8. Conclusions and Future Research

This review focuses on the application value and developmental trajectory of LLMs in the vertical domain. This study identified a core set of enabling technologies for LLM verticalization, including domain-specific fine-tuning, RAG, knowledge distillation, and MAS. These technologies have evolved synergistically to satisfy the stringent demands for accuracy and reliability in specialized domains, thereby enhancing the robustness, transparency, and trustworthiness of VDLLMs.

Despite the growing promise of LLMs in vertical applications, significant challenges remain, such as limitations in domain data quality and availability, model hallucinations, poor explainability, value alignment risks, and resource constraints. To address these issues, targeted mitigation strategies can be developed in the future, including the design of homogeneous domain-specific architectures and the strategic integration of complementary technologies.

In conclusion, the vertical deployment of LLMs represents an ongoing multidimensional evolution driven by interconnected technical paradigms. Research and development within each framework is closely intertwined with advances in NLP, large-scale modeling, and deep learning. As LLM technology continues to mature, its domain-specific applications are poised for broader adoption, catalyzing increased intelligence and automation across specialized fields.

Acknowledgments

This work was supported by the Shandong Provincial Natural Science Foundation (grant number ZR2019LZH006).

References
  1. [1] M. A. K. Raiaan, M. S. H. Mukta, K. Fatema, N. M. Fahad, S. Sakib, M. M. J. Mim, J. Ahmad, M. E. Ali, and S. Azam, “A review on large language models: Architectures, applications, taxonomies, open issues and challenges,” IEEE Access, Vol.12, pp. 26839-26874, 2024. https://doi.org/10.1109/ACCESS.2024.3365742
  2. [2] T. Wu, S. He, J. Liu, S. Sun, K. Liu, Q.-L. Han, and Y. Tang, “A brief overview of chatGPT: The history, status quo and potential future development,” IEEE/CAA J. of Automatica Sinica, Vol.10, No.5, pp. 1122-1136, 2023. https://doi.org/10.1109/JAS.2023.123618
  3. [3] A. Bahrini, M. Khamoshifar, H. Abbasimehr, R. J. Riggs, M. Esmaeili, R. M. Majdabadkohne, and M. Pasehvar, “ChatGPT: Applications, opportunities, and threats,” arXiv:2304.09103, 2023. https://doi.org/10.48550/arXiv.2304.09103
  4. [4] L. Caruccio, S. Cirillo, G. Polese, G. Solimando, S. Sundaramurthy, and G. Tortora, “Claude 2.0 large language model: Tackling a real-world classification problem with a new iterative prompt engineering approach,” Intelligent Systems with Applications, Vol.21, Article No.200336, 2024. https://doi.org/10.1016/j.iswa.2024.200336
  5. [5] Gemini Team Google: R. Anil, S. Borgeaud, J.-B. Alayrac et al., “Gemini: A family of highly capable multimodal models,” arXiv:2312.11805, 2025. https://doi.org/10.48550/arXiv.2312.11805
  6. [6] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” arXiv:1810.04805, 2019. https://doi.org/10.48550/arXiv.1810.04805
  7. [7] M. Parović, Z. Li, and J. Du, “Generating domain-specific knowledge graphs from large language models,” Findings of the Association for Computational Linguistics (ACL 2025), 2025. https://doi.org/10.18653/v1/2025.findings-acl.602
  8. [8] R. Jegan and A. Henrich, “A structured literature review on traditional approaches in current natural language processing,” arXiv:2505.12970, 2025. https://doi.org/10.48550/arXiv.2505.12970
  9. [9] K. A. Jallad, N. Ghneim, and G. Rebdawi, “Survey of NLU benchmarks diagnosing linguistic phenomena: Why not standardize diagnostics benchmarks?,” arXiv:2507.20419, 2025. https://doi.org/10.48550/arXiv.2507.20419
  10. [10] C. Dong, Y. Li, H. Gong, M. Chen, J. Li, Y. Shen, and M. Yang, “A survey of natural language generation,” ACM Computing Surveys, Vol.55, No.8, Article No.173, 2023. https://doi.org/10.1145/3554727
  11. [11] L. Zangari, C. M. Greco, D. Picca, and A. Tagarelli, “A survey on moral foundation theory and pre-trained language models: Current advances and challenges,” AI & Society, Vol.40, No.6, pp. 4973-4998, 2025. https://doi.org/10.1007/s00146-025-02225-w
  12. [12] S. Minaee, T. Mikolov, N. Nikzad, M. Chenaghlu, R. Socher, X. Amatriain, and J. Gao, “Large language models: A survey,” arXiv:2402.06196, 2025. https://doi.org/10.48550/arXiv.2402.06196
  13. [13] A. Chowdhery, S. Narang, J. Devlin et al., “PaLM: Scaling language modeling with pathways,” arXiv:2204.02311, 2022. https://doi.org/10.48550/arXiv.2204.02311
  14. [14] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample, “Llama: Open and efficient foundation language models,” 2023. https://doi.org/10.48550/arXiv.2302.13971
  15. [15] OpenAI, J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, R. Avila, I. Babuschkin et al., “GPT-4 technical report,” arXiv:2303.08774, 2024. https://doi.org/10.48550/arXiv.2303.08774
  16. [16] L. Berti, F. Giorgi, and G. Kasneci, “Emergent abilities in large language models: A survey,” arXiv:2503.05788, 2025. https://doi.org/10.48550/arXiv.2503.05788
  17. [17] K. O’Shea and R. Nash, “An introduction to convolutional neural networks,” arXiv:1511.08458, 2015. https://doi.org/10.48550/arXiv.1511.08458
  18. [18] A. Sherstinsky, “Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network,” Physica D: Nonlinear Phenomena, Vol.404, Article No.132306, 2020. https://doi.org/10.1016/j.physd.2019.132306
  19. [19] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” arXiv:1706.03762, 2023. https://doi.org/10.48550/arXiv.1706.03762
  20. [20] F. Ahmadou, S. Ghaffarzadegan, B. Nour, M. Pourzandi, M. Debbabi, and C. Assi, “Automated attack testflow extraction from cyber threat report using BERT for contextual analysis,” arXiv:2507.07244, 2025. https://doi.org/10.48550/arXiv.2507.07244
  21. [21] Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “RoBERTa: A robustly optimized BERT pretraining approach,” arXiv:1907.11692, 2019. https://doi.org/10.48550/arXiv.1907.11692
  22. [22] Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut, “Albert: A lite bert for self-supervised learning of language representations,” arXiv:1909.11942, 2020. https://doi.org/10.48550/arXiv.1909.11942
  23. [23] B. Workshop, T. L. Scao, A. Fan, C. Akiki et al., “BLOOM: A 176B-parameter open-access multilingual language model,” arXiv:2211.05100, 2023. https://doi.org/10.48550/arXiv.2211.05100
  24. [24] A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey et al., “The llama 3 herd of models,” arXiv:2407.21783, 2024. https://doi.org/10.48550/arXiv.2407.21783
  25. [25] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” arXiv:1910.10683, 2023. https://doi.org/10.48550/arXiv.1910.10683
  26. [26] L. Xue, N. Constant, A. Roberts, M. Kale, R. Al-Rfou, A. Siddhant, A. Barua, and C. Raffel, “mT5: A massively multilingual pre-trained text-to-text transformer,” arXiv:2010.11934, 2021. https://doi.org/10.48550/arXiv.2010.11934
  27. [27] M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, “BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,” arXiv:1910.13461, 2019. https://doi.org/10.48550/arXiv.1910.13461
  28. [28] S. Lei, Y. Hua, and S. Zhihao, “Revisiting fine-tuning: A survey of parameter-efficient techniques for large AI models,” Preprint, 2025. https://doi.org/10.20944/preprints202504.0743.v1
  29. [29] Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, M. Wang, and H. Wang, “Retrieval-augmented generation for large language models: A survey,” arXiv:2312.10997, 2024. https://doi.org/10.48550/arXiv.2312.10997
  30. [30] C. de Vente, K. A. Vermeer, N. Jaccard, H. Wang, H. Sun et al., “AIROGS: Artificial intelligence for robust glaucoma screening challenge,” IEEE Trans. on Medical Imaging, Vol.43, No.1, pp. 542-557, 2024. https://doi.org/10.1109/TMI.2023.3313786
  31. [31] P. Kumar and S. Mishra, “Robustness in large language models: A survey of mitigation strategies and evaluation metrics,” arXiv:2505.18658, 2025. https://doi.org/10.48550/arXiv.2505.18658
  32. [32] Z. Zhang, Y. Zhang, H. Yao, J. Luo, R. Zhao, B. Huang, J. Zhao, Y. Liao, K. Li, L. Zhao, J. Cao, F. Qi, and C. Yuan, “Xiwu: A basis flexible and learnable LLM for high energy physics,” arXiv:2404.08001, 2024. https://doi.org/10.48550/arXiv.2404.08001
  33. [33] Z. Sun, Y.-S. Ting, Y. Liang, N. Duan, S. Huang, and Z. Cai, “Mephisto: Self-improving large language model-based agents for automated interpretation of multi-band galaxy observations,” arXiv:2510.08354, 2025. https://doi.org/10.48550/arXiv.2510.08354
  34. [34] Y. Zhang, Y. Han, S. Chen, R. Yu, X. Zhao, X. Liu, K. Zeng, M. Yu, J. Tian, F. Zhu, X. Yang, Y. Jin, and Y. Xu, “Large language models to accelerate organic chemistry synthesis,” Nature Machine Intelligence, Vol.7, No.7, pp. 1010-1022, 2025. https://doi.org/10.1038/s42256-025-01066-y
  35. [35] K. Singhal, T. Tu, J. Gottweis, R. Sayres, E. Wulczyn, L. Hou, K. Clark, S. Pfohl, H. Cole-Lewis, D. Neal, M. Schaekermann, A. Wang, M. Amin, S. Lachgar, P. Mansfield, S. Prakash, B. Green, E. Dominowska, B. A. y Arcas, N. Tomasev, Y. Liu, R. Wong, C. Semturs, S. S. Mahdavi, J. Barral, D. Webster, G. S. Corrado, Y. Matias, S. Azizi, A. Karthikesalingam, and V. Natarajan, “Towards expert-level medical question answering with large language models,” arXiv:2305.09617, 2023. https://doi.org/10.48550/arXiv.2305.09617
  36. [36] P. R. Liu, S. Bansal, J. Dinh, A. Pawar, R. Satishkumar, S. Desai, N. Gupta, X. Wang, and S. Hu, “Medchat: A multi-agent framework for multimodal diagnosis with large language models,” arXiv:2506.07400, 2025. https://doi.org/10.48550/arXiv.2506.07400
  37. [37] H. Liang, T. Yang, Z. Liu, W. Jian, Y. Chen, B. Li, Z. Yan, W. Xu, L. Chen, Y. Qi, Z. Wang, Y. Liao, P. Lin, J. Li, W. Wang, L. Li, M. Wang, Y. Zhang, L. Deng, T. Jiang, and J. He, “Lungdiag: Empowering artificial intelligence for respiratory diseases diagnosis based on electronic health records, a multicenter study,” MedComm, Vol.6, No.1, Article No.e70043, 2025. https://doi.org/10.1002/mco2.70043
  38. [38] Z. Zhang, D. Zhang-Li, J. Yu, L. Gong, J. Zhou, Z. Hao, J. Jiang, J. Cao, H. Liu, Z. Liu, L. Hou, and J. Li, “Simulating classroom education with LLM-empowered agents,” arXiv:2406.19226, 2024. https://doi.org/10.48550/arXiv.2406.19226
  39. [39] L. Chen, Z. Jiang, D. Xia, Z. Cai, L. Sun, P. Childs, and H. Zuo, “Bidtrainer: An LLMs-driven education tool for enhancing the understanding and reasoning in bio-inspired design,” Proc. of the 2024 CHI Conf. Factors in Computing Systems, Article No.676, 2024. https://doi.org/10.1145/3613904.3642887
  40. [40] S. Wu, O. Irsoy, S. Lu, V. Dabravolski, M. Dredze, S. Gehrmann, P. Kambadur, D. Rosenberg, and G. Mann, “BloombergGPT: A large language model for finance,” arXiv:2303.17564, 2023. https://doi.org/10.48550/arXiv.2303.17564
  41. [41] Y. Yang, Y. Tang, and K. Y. Tam, “InvestLM: A large language model for investment using financial domain instruction tuning,” arXiv:2309.13064, 2023. https://doi.org/10.48550/arXiv.2309.13064
  42. [42] B. Yang, Y. Zhang, L. Feng, Y. Chen, J. Zhang, X. Xu, N. Aierken, Y. Li, Y. Chen, G. Yang, Y. He, R. Huang, and S. Li, “AgriGPT: A large language model ecosystem for agriculture,” arXiv:2508.08632, 2025. https://doi.org/10.48550/arXiv.2508.08632
  43. [43] D. J. Samuel, I. Skarga-Bandurova, D. Sikolia, and M. Awais, “AgroLLM: Connecting farmers and agricultural practices through large language models for enhanced knowledge transfer and practical application,” arXiv:2503.04788, 2025. https://doi.org/10.48550/arXiv.2503.04788
  44. [44] K. Kumar, T. Ashraf, O. Thawakar, R. M. Anwer, H. Cholakkal, M. Shah, M.-H. Yang, P. H. S. Torr, F. S. Khan, and S. Khan, “LLM post-training: A deep dive into reasoning large language models,” arXiv:2502.21321, 2025. https://doi.org/10.48550/arXiv.2502.21321
  45. [45] J. Tack, J. Lanchantin, J. Yu, A. Cohen, I. Kulikov, J. Lan, S. Hao, Y. Tian, J. Weston, and X. Li, “LLM pretraining with continuous concepts,” arXiv:2502.08524, 2025. https://doi.org/10.48550/arXiv.2502.08524
  46. [46] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv:1412.6980, 2017. https://doi.org/10.48550/arXiv.1412.6980
  47. [47] I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” arXiv:1711.05101, 2019. https://doi.org/10.48550/arXiv.1711.05101
  48. [48] N. Shazeer and M. Stern, “Adafactor: Adaptive learning rates with sublinear memory cost,” arXiv:1804.04235, 2018. https://doi.org/10.48550/arXiv.1804.04235
  49. [49] S. Rajbhandari, J. Rasley, O. Ruwase, and Y. He, “ZeRo: Memory optimizations toward training trillion parameter models,” arXiv:1910.02054, 2020. https://doi.org/10.48550/arXiv.1910.02054
  50. [50] P. Micikevicius, S. Narang, J. Alben, G. Diamos, E. Elsen, D. Garcia, B. Ginsburg, M. Houston, O. Kuchaiev, G. Venkatesh, and H. Wu, “Mixed precision training,” arXiv:1710.03740, 2018. https://doi.org/10.48550/arXiv.1710.03740
  51. [51] A. Q. Jiang, A. Sablayrolles, A. Roux, A. Mensch, B. Savary, C. Bamford, D. S. Chaplot, D. d. l. Casas, E. B. Hanna, F. Bressand, G. Lengyel, G. Bour, G. Lample, L. R. Lavaud, L. Saulnier, M.-A. Lachaux, P. Stock, S. Subramanian, S. Yang, S. Antoniak, T. L. Scao, T. Gervet, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed, “Mixtral of experts,” arXiv:2401.04088, 2024. https://doi.org/10.48550/arXiv.2401.04088
  52. [52] N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean, “Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,” arXiv:1701.06538, 2017. https://doi.org/10.48550/arXiv.1701.06538
  53. [53] S. Jiang, T. Zheng, Y. Zhang, Y. Jin, L. Yuan, and Z. Liu, “Med-moe: Mixture of domain-specific experts for lightweight medical vision-language models,” arXiv:2404.10237, 2024. https://doi.org/10.48550/arXiv.2404.10237
  54. [54] A. Yang, B. Xiao, B. Wang et al., “Baichuan 2: Open large-scale language models,” arXiv:2309.10305, 2025. https://doi.org/10.48550/arXiv.2309.10305
  55. [55] A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu et al., “Qwen3 technical report,” arXiv:2505.09388, 2025. https://doi.org/10.48550/arXiv.2505.09388
  56. [56] T. GLM, :, A. Zeng, B. Xu et al., “ChatGLM: A family of large language models from GLM-130B to GLM-4 all tools,” arXiv:2406.12793, 2024. https://doi.org/10.48550/arXiv.2406.12793
  57. [57] Z. Cai, M. Cao, H. Chen, and other, “InternLM2 technical report,” arXiv:2403.17297, 2024. https://doi.org/10.48550/arXiv.2403.17297
  58. [58] E. Almazrouei, H. Alobeidli, A. Alshamsi, A. Cappelli, R. Cojocaru, M. Debbah, É. Goffinet, D. Hesslow, J. Launay, Q. Malartic, D. Mazzotta, B. Noune, B. Pannier, and G. Penedo, “The falcon series of open language models,” arXiv:2311.16867, 2023. https://doi.org/10.48550/arXiv.2311.16867
  59. [59] P. Zhao, H. Zhang, Q. Yu, Z. Wang, Y. Geng, F. Fu, L. Yang, W. Zhang, J. Jiang, and B. Cui, “Retrieval-augmented generation for AI-generated content: A survey,” arXiv:2402.19473, 2024. https://doi.org/10.48550/arXiv.2402.19473
  60. [60] K. Wu, E. Wu, K. Wei et al., “An automated framework for assessing how well LLMs cite relevant medical references,” Nature Communications, Vol.16, No.1, Article No.3615, 2025. https://doi.org/10.1038/s41467-025-58551-6
  61. [61] S. Zhang, L. Dong, X. Li, S. Zhang, X. Sun, S. Wang, J. Li, R. Hu, T. Zhang, F. Wu, and G. Wang, “Instruction tuning for large language models: A survey,” arXiv:2308.10792, 2025. https://doi.org/10.48550/arXiv.2308.10792
  62. [62] H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y. Tay, W. Fedus, Y. Li, X. Wang, M. Dehghani, S. Brahma, A. Webson, S. S. Gu, Z. Dai, M. Suzgun, X. Chen, A. Chowdhery, A. Castro-Ros, M. Pellat, K. Robinson, D. Valter, S. Narang, G. Mishra, A. Yu, V. Zhao, Y. Huang, A. Dai, H. Yu, S. Petrov, E. H. Chi, J. Dean, J. Devlin, A. Roberts, D. Zhou, Q. V. Le, and J. Wei, “Scaling instruction-finetuned language models,” arXiv:2210.11416, 2022. https://doi.org/10.48550/arXiv.2210.11416
  63. [63] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe, “Training language models to follow instructions with human feedback,” arXiv:2203.02155, 2022. https://doi.org/10.48550/arXiv.2203.02155
  64. [64] Z. Han, C. Gao, J. Liu, J. Zhang, and S. Q. Zhang, “Parameter-efficient fine-tuning for large models: A comprehensive survey,” arXiv:2403.14608, 2024. https://doi.org/10.48550/arXiv.2403.14608
  65. [65] B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter-efficient prompt tuning,” arXiv:2104.08691, 2021. https://doi.org/10.48550/arXiv.2104.08691
  66. [66] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” arXiv:2106.09685, 2021. https://doi.org/10.48550/arXiv.2106.09685
  67. [67] X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” arXiv:2101.00190, 2021. https://doi.org/10.48550/arXiv.2101.00190
  68. [68] X. Liu, K. Ji, Y. Fu, W. L. Tam, Z. Du, Z. Yang, and J. Tang, “P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks,” arXiv:2110.07602, 2022. https://doi.org/10.48550/arXiv.2110.07602
  69. [69] Z. Zhang, Y. Zhang, H. Yao, J. Luo, R. Zhao, B. Huang, J. Zhao, Y. Liao, K. Li, L. Zhao, J. Cao, F. Qi, and C. Yuan, “Xiwu: A basis flexible and learnable LLM for high energy physics,” arXiv:2404.08001, 2024. https://doi.org/10.48550/arXiv.2404.08001
  70. [70] S. Song, H. Xu, J. Ma, S. Li, L. Peng, Q. Wan, X. Liu, and J. Yu, “How to alleviate catastrophic forgetting in LLMs finetuning? Hierarchical layer-wise and element-wise regularization,” arXiv:2501.13669, 2025. https://doi.org/10.48550/arXiv.2501.13669
  71. [71] R. Schaeffer, B. Miranda, and S. Koyejo, “Are emergent abilities of large language models a mirage?,” arXiv:2304.15004, 2023. https://doi.org/10.48550/arXiv.2304.15004
  72. [72] J. Wei, J. Wei, Y. Tay, D. Tran, A. Webson, Y. Lu, X. Chen, H. Liu, D. Huang, D. Zhou, and T. Ma, “Larger language models do in-context learning differently,” arXiv:2303.03846, 2023. https://doi.org/10.48550/arXiv.2303.03846
  73. [73] J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou, “Chain-of-thought prompting elicits reasoning in large language models,” arXiv:2201.11903, 2023. https://doi.org/10.48550/arXiv.2201.11903
  74. [74] G. Chen, L. Fan, Z. Gong, N. Xie, Z. Li, Z. Liu, C. Li, Q. Qu, H. Alinejad-Rokny, S. Ni, and M. Yang, “AgentCourt: Simulating court with adversarial evolvable lawyer agents,” arXiv:2408.08089, 2025. https://doi.org/10.48550/arXiv.2408.08089
  75. [75] A. M. Mansourian, R. Ahmadi, M. Ghafouri, A. M. Babaei, E. B. Golezani, Z. Y. Ghamchi, V. Ramezanian, A. Taherian, K. Dinashi, A. Miri, and S. Kasaei, “A comprehensive survey on knowledge distillation,” arXiv:2503.12067, 2025. https://doi.org/10.48550/arXiv.2503.12067
  76. [76] T. Kitrungrotsakul, Y. Xu, and P. Srichola, “Knowledge distillation meets reinforcement learning: A cluster-driven approach to image processing,” Sensors, Vol.26, No.1, Article No.209, 2025. https://doi.org/10.3390/s26010209
  77. [77] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, “ReAct: Synergizing reasoning and acting in language models,” arXiv:2210.03629, 2022. https://doi.org/10.48550/arXiv.2210.03629
  78. [78] S. Yao, D. Yu, J. Zhao, I. Shafran, T. L. Griffiths, Y. Cao, and K. Narasimhan, “Tree of thoughts: Deliberate problem solving with large language models,” arXiv:2305.10601, 2023. https://doi.org/10.48550/arXiv.2305.10601
  79. [79] Z. Zhang, X. Bo, C. Ma, R. Li, X. Chen, Q. Dai, J. Zhu, Z. Dong, and J.-R. Wen, “A survey on the memory mechanism of large language model based agents,” arXiv:2404.13501, 2024. https://doi.org/10.48550/arXiv.2404.13501
  80. [80] S. Han, Q. Zhang, Y. Yao, W. Jin, and Z. Xu, “LLM multi-agent systems: Challenges and open problems,” arXiv:2402.03578, 2025. https://doi.org/10.48550/arXiv.2402.03578
  81. [81] A. Ehtesham, A. Singh, G. K. Gupta, and S. Kumar, “A survey of agent interoperability protocols: Model context protocol (MCP), agent communication protocol (ACP), agent-to-agent protocol (A2A), and agent network protocol (ANP),” arXiv:2505.02279, 2025. https://doi.org/10.48550/arXiv.2505.02279
  82. [82] Z. Jia, M. Jia, J. Duan, and J. Wang, “DDO: Dual-decision optimization for LLM-based medical consultation via multi-agent collaboration,” arXiv:2505.18630, 2025. https://doi.org/10.48550/arXiv.2505.18630
  83. [83] L. Long, Y. He, W. Ye, Y. Pan, Y. Lin, H. Li, J. Zhao, and W. Li, “Seeing, listening, remembering, and reasoning: A multimodal agent with long-term memory,” arXiv:2505.18630, 2025. https://doi.org/10.48550/arXiv.2508.09736
  84. [84] C. Chen, J. Yu, S. Chen, C. Liu, Z. Wan, D. Bitterman, F. Wang, and K. Shu, “Clinicalbench: Can LLMs beat traditional ML models in clinical prediction?,” arXiv:2411.06469, 2024. https://doi.org/10.48550/arXiv.2411.06469
  85. [85] B. Hawks, G. v. Laszewski, M. D. Sinclair, M. Colombo, S. Venkataraman, R. Jain, Y. Jiang, N. Tran, and G. Fox, “An MLcommons scientific benchmarks ontology,” arXiv:2511.05614, 2025. https://doi.org/10.48550/arXiv.2511.05614
  86. [86] Y. Xiao, E. Sun, T. Liu, and W. Wang, “Logicvista: Multimodal LLM logical reasoning benchmark in visual contexts,” arXiv:2407.04973, 2024. https://doi.org/10.48550/arXiv.2407.04973
  87. [87] Y. Zhou, X. Wu, B. Huang, J. Wu, L. Feng, and K. C. Tan, “CausalBN-Bench: A comprehensive benchmark for causal learning capability of LLMs,” arXiv:2404.06349, 2024. https://doi.org/10.48550/arXiv.2404.06349
  88. [88] Y. Zhang, Y. Huang, Y. Sun, C. Liu, Z. Zhao, Z. Fang, Y. Wang, H. Chen, X. Yang, X. Wei, H. Su, Y. Dong, and J. Zhu, “Multitrust: A comprehensive benchmark towards trustworthy multimodal large language models,” arXiv:2406.07057, 2024. https://doi.org/10.48550/arXiv.2406.07057
  89. [89] A. Wang, Y. Pruksachatkun, N. Nangia, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman, “SuperGLUE: A stickier benchmark for general-purpose language understanding systems,” arXiv:1905.00537, 2020. https://doi.org/10.48550/arXiv.1905.00537
  90. [90] Y. Li, H. Huang, Y. Zhao, X. Ma, and J. Sun, “BackdoorLLM: A comprehensive benchmark for backdoor attacks and defenses on large language models,” arXiv:2408.12798, 2025. https://doi.org/10.48550/arXiv.2408.12798
  91. [91] D. Zhang, W. Liu, Q. Tan, J. Chen, H. Yan, Y. Yan, J. Li, W. Huang, X. Yue, W. Ouyang, D. Zhou, S. Zhang, M. Su, H.-S. Zhong, and Y. Li, “ChemLLM: A chemical large language model,” arXiv:2402.06852, 2024. https://doi.org/10.48550/arXiv.2402.06852
  92. [92] L. Cardenas, K. Parajes, M. Zhu, and S. Zhai, “AutoHealth: Advanced LLM-empowered wearable personalized medical butler for parkinson’s disease management,” 2024 IEEE 14th Annual Computing and Communication Workshop and Conf. (CCWC), pp. 0375-0379, 2024. https://doi.org/10.1109/CCWC60891.2024.10427622
  93. [93] D. P. Kingma and M. Welling, “An introduction to variational autoencoders,” Foundations and Trends® in Machine Learning, Vol.12, No.4, pp. 307-392, 2019. https://doi.org/10.1561/2200000056
  94. [94] M. Li, H. Chen, Y. Wang, T. Zhu, W. Zhang, K. Zhu, K.-F. Wong, and J. Wang, “Understanding and mitigating the bias inheritance in LLM-based data augmentation on downstream tasks,” arXiv:2502.04419, 2025. https://doi.org/10.48550/arXiv.2502.04419
  95. [95] I. X. Weissburg, S. Anand, S. Levy, and H. Jeong, “LLMs are biased teachers: Evaluating llm bias in personalized education,” arXiv:2410.14012, 2024. https://doi.org/10.48550/arXiv.2410.14012
  96. [96] N. Pipitone and G. H. Alami, “LegalBench-RAG: A benchmark for retrieval-augmented generation in the legal domain,” arXiv:2408.10343, 2024. https://doi.org/10.48550/arXiv.2408.10343
  97. [97] H. Zhao, H. Chen, F. Yang, N. Liu, H. Deng, H. Cai, S. Wang, D. Yin, and M. Du, “Explainability for large language models: A survey,” ACM Trans. on Intelligent Systems and Technology, Vol.15, No.2, Article No.20, 2024. https://doi.org/10.1145/3639372
  98. [98] M. A. Aljohani, J. Hou, S. Kommu, and X. Wang, “A comprehensive survey on the trustworthiness of large language models in healthcare,” arXiv:2502.15871, 2025. https://doi.org/10.48550/arXiv.2502.15871
  99. [99] F. Naudot, T. Sundqvist, and T. Kampik, “llmSHAP: A principled approach to LLM explainability,” arXiv:2511.01311, 2025. https://doi.org/10.48550/arXiv.2511.01311
  100. [100] S. Sztwiertnia, F. Friedrich, K. Kersting, P. Schramowski, and B. Deiseroth, “LIME: Making LLM data more efficient with linguistic metadata embeddings,” arXiv:2512.07522, 2025. https://doi.org/10.48550/arXiv.2512.07522
  101. [101] H. Luo and L. Specia, “From understanding to utilization: A survey on explainability for large language models,” arXiv:2401.12874, 2024. https://doi.org/10.48550/arXiv.2401.12874
  102. [102] M. Sanwal, “Layered chain-of-thought prompting for multi-agent LLM systems: A comprehensive approach to explainable large language models,” arXiv:2501.18645, 2025. https://doi.org/10.48550/arXiv.2501.18645
  103. [103] J. S. Erickson, H. Santos, V. Pinheiro, J. P. McCusker, and D. L. McGuinness, “LLM experimentation through knowledge graphs: Towards improved management, repeatability, and verification,” J. of Web Semantics, Vol.85, Article No.100853, 2025. https://doi.org/10.1016/j.websem.2024.100853
  104. [104] J. Arabneydi, S. Islam, S. Das, S. K. Gottipati, W. Duguay, C. Mars, M. E. Taylor, M. J. Guzdial, A. Fagette, and Y. Zerouali, “A systematic approach to design real-world human-in-the-loop deep reinforcement learning: Salient features, challenges and trade-offs,” arXiv:2504.17006, 2025. https://doi.org/10.48550/arXiv.2504.17006
  105. [105] R. Patil and V. Gudivada, “A review of current trends, techniques, and challenges in large language models (LLMs),” Applied Sciences, Vol.14, No.5, Article No.2074, 2024. https://doi.org/10.3390/app14052074
  106. [106] K. Hau, S. Hassan, and S. Zhou, “LLMs in mobile apps: Practices, challenges, and opportunities,” 2025 IEEE/ACM 12th Int. Conf. on Mobile Software Engineering and Systems (MOBILESoft), pp. 3-14, 2025. https://doi.org/10.1109/mobilesoft66462.2025.00008

*This site is desgined based on HTML5 and CSS3 for modern browsers, e.g. Chrome, Firefox, Safari, Edge, Opera.

Last updated on Nov. 20, 2026