Review:
Vertical Domain Large Language Models: Applications, Challenges, Future Prospects
Yanling Chen*, Wujun Wang*, Jiyang Peng*, Jifa Xu*, Kang Jia**, Mengyao Liu*,, and Zhongyun Tian*
*Inspur Electronic Information Industry Co., Ltd.
1036 Caoshanling East Road, Jinan, Shandong 250000, China
Corresponding author
**Institute of Frontier and Interdisciplinary Science, Shandong University
72 Binhai Road, Qingdao, Shandong 260000, China
Large language models (LLMs) have become a cornerstone of modern generative artificial intelligence, demonstrating remarkable performance across a wide range of tasks, including machine translation, question answering, and dialogue generation. However, most existing reviews primarily emphasize theoretical foundations and training methodologies, and offer limited insights into real-world application performance and domain-specific deployment patterns. To address this gap, we conducted a systematic literature review following PRISMA guidelines, synthesizing the application of LLMs across diverse vertical domains. The review examines system architectures, technical approaches, and performance benchmarks reported in the literature. In addition, it analyzes core challenges associated with LLM deployment, such as hallucinations, explainability, value alignment, and data quality, along with targeted solutions. Finally, it identifies emerging research opportunities and future directions, contributing to a more coherent, application-oriented understanding of LLM development and impact.
Outline structure of the article
- [1] M. A. K. Raiaan, M. S. H. Mukta, K. Fatema, N. M. Fahad, S. Sakib, M. M. J. Mim, J. Ahmad, M. E. Ali, and S. Azam, “A review on large language models: Architectures, applications, taxonomies, open issues and challenges,” IEEE Access, Vol.12, pp. 26839-26874, 2024. https://doi.org/10.1109/ACCESS.2024.3365742
- [2] T. Wu, S. He, J. Liu, S. Sun, K. Liu, Q.-L. Han, and Y. Tang, “A brief overview of chatGPT: The history, status quo and potential future development,” IEEE/CAA J. of Automatica Sinica, Vol.10, No.5, pp. 1122-1136, 2023. https://doi.org/10.1109/JAS.2023.123618
- [3] A. Bahrini, M. Khamoshifar, H. Abbasimehr, R. J. Riggs, M. Esmaeili, R. M. Majdabadkohne, and M. Pasehvar, “ChatGPT: Applications, opportunities, and threats,” arXiv:2304.09103, 2023. https://doi.org/10.48550/arXiv.2304.09103
- [4] L. Caruccio, S. Cirillo, G. Polese, G. Solimando, S. Sundaramurthy, and G. Tortora, “Claude 2.0 large language model: Tackling a real-world classification problem with a new iterative prompt engineering approach,” Intelligent Systems with Applications, Vol.21, Article No.200336, 2024. https://doi.org/10.1016/j.iswa.2024.200336
- [5] Gemini Team Google: R. Anil, S. Borgeaud, J.-B. Alayrac et al., “Gemini: A family of highly capable multimodal models,” arXiv:2312.11805, 2025. https://doi.org/10.48550/arXiv.2312.11805
- [6] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” arXiv:1810.04805, 2019. https://doi.org/10.48550/arXiv.1810.04805
- [7] M. Parović, Z. Li, and J. Du, “Generating domain-specific knowledge graphs from large language models,” Findings of the Association for Computational Linguistics (ACL 2025), 2025. https://doi.org/10.18653/v1/2025.findings-acl.602
- [8] R. Jegan and A. Henrich, “A structured literature review on traditional approaches in current natural language processing,” arXiv:2505.12970, 2025. https://doi.org/10.48550/arXiv.2505.12970
- [9] K. A. Jallad, N. Ghneim, and G. Rebdawi, “Survey of NLU benchmarks diagnosing linguistic phenomena: Why not standardize diagnostics benchmarks?,” arXiv:2507.20419, 2025. https://doi.org/10.48550/arXiv.2507.20419
- [10] C. Dong, Y. Li, H. Gong, M. Chen, J. Li, Y. Shen, and M. Yang, “A survey of natural language generation,” ACM Computing Surveys, Vol.55, No.8, Article No.173, 2023. https://doi.org/10.1145/3554727
- [11] L. Zangari, C. M. Greco, D. Picca, and A. Tagarelli, “A survey on moral foundation theory and pre-trained language models: Current advances and challenges,” AI & Society, Vol.40, No.6, pp. 4973-4998, 2025. https://doi.org/10.1007/s00146-025-02225-w
- [12] S. Minaee, T. Mikolov, N. Nikzad, M. Chenaghlu, R. Socher, X. Amatriain, and J. Gao, “Large language models: A survey,” arXiv:2402.06196, 2025. https://doi.org/10.48550/arXiv.2402.06196
- [13] A. Chowdhery, S. Narang, J. Devlin et al., “PaLM: Scaling language modeling with pathways,” arXiv:2204.02311, 2022. https://doi.org/10.48550/arXiv.2204.02311
- [14] H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample, “Llama: Open and efficient foundation language models,” 2023. https://doi.org/10.48550/arXiv.2302.13971
- [15] OpenAI, J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, R. Avila, I. Babuschkin et al., “GPT-4 technical report,” arXiv:2303.08774, 2024. https://doi.org/10.48550/arXiv.2303.08774
- [16] L. Berti, F. Giorgi, and G. Kasneci, “Emergent abilities in large language models: A survey,” arXiv:2503.05788, 2025. https://doi.org/10.48550/arXiv.2503.05788
- [17] K. O’Shea and R. Nash, “An introduction to convolutional neural networks,” arXiv:1511.08458, 2015. https://doi.org/10.48550/arXiv.1511.08458
- [18] A. Sherstinsky, “Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network,” Physica D: Nonlinear Phenomena, Vol.404, Article No.132306, 2020. https://doi.org/10.1016/j.physd.2019.132306
- [19] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” arXiv:1706.03762, 2023. https://doi.org/10.48550/arXiv.1706.03762
- [20] F. Ahmadou, S. Ghaffarzadegan, B. Nour, M. Pourzandi, M. Debbabi, and C. Assi, “Automated attack testflow extraction from cyber threat report using BERT for contextual analysis,” arXiv:2507.07244, 2025. https://doi.org/10.48550/arXiv.2507.07244
- [21] Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “RoBERTa: A robustly optimized BERT pretraining approach,” arXiv:1907.11692, 2019. https://doi.org/10.48550/arXiv.1907.11692
- [22] Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut, “Albert: A lite bert for self-supervised learning of language representations,” arXiv:1909.11942, 2020. https://doi.org/10.48550/arXiv.1909.11942
- [23] B. Workshop, T. L. Scao, A. Fan, C. Akiki et al., “BLOOM: A 176B-parameter open-access multilingual language model,” arXiv:2211.05100, 2023. https://doi.org/10.48550/arXiv.2211.05100
- [24] A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey et al., “The llama 3 herd of models,” arXiv:2407.21783, 2024. https://doi.org/10.48550/arXiv.2407.21783
- [25] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” arXiv:1910.10683, 2023. https://doi.org/10.48550/arXiv.1910.10683
- [26] L. Xue, N. Constant, A. Roberts, M. Kale, R. Al-Rfou, A. Siddhant, A. Barua, and C. Raffel, “mT5: A massively multilingual pre-trained text-to-text transformer,” arXiv:2010.11934, 2021. https://doi.org/10.48550/arXiv.2010.11934
- [27] M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, “BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,” arXiv:1910.13461, 2019. https://doi.org/10.48550/arXiv.1910.13461
- [28] S. Lei, Y. Hua, and S. Zhihao, “Revisiting fine-tuning: A survey of parameter-efficient techniques for large AI models,” Preprint, 2025. https://doi.org/10.20944/preprints202504.0743.v1
- [29] Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, M. Wang, and H. Wang, “Retrieval-augmented generation for large language models: A survey,” arXiv:2312.10997, 2024. https://doi.org/10.48550/arXiv.2312.10997
- [30] C. de Vente, K. A. Vermeer, N. Jaccard, H. Wang, H. Sun et al., “AIROGS: Artificial intelligence for robust glaucoma screening challenge,” IEEE Trans. on Medical Imaging, Vol.43, No.1, pp. 542-557, 2024. https://doi.org/10.1109/TMI.2023.3313786
- [31] P. Kumar and S. Mishra, “Robustness in large language models: A survey of mitigation strategies and evaluation metrics,” arXiv:2505.18658, 2025. https://doi.org/10.48550/arXiv.2505.18658
- [32] Z. Zhang, Y. Zhang, H. Yao, J. Luo, R. Zhao, B. Huang, J. Zhao, Y. Liao, K. Li, L. Zhao, J. Cao, F. Qi, and C. Yuan, “Xiwu: A basis flexible and learnable LLM for high energy physics,” arXiv:2404.08001, 2024. https://doi.org/10.48550/arXiv.2404.08001
- [33] Z. Sun, Y.-S. Ting, Y. Liang, N. Duan, S. Huang, and Z. Cai, “Mephisto: Self-improving large language model-based agents for automated interpretation of multi-band galaxy observations,” arXiv:2510.08354, 2025. https://doi.org/10.48550/arXiv.2510.08354
- [34] Y. Zhang, Y. Han, S. Chen, R. Yu, X. Zhao, X. Liu, K. Zeng, M. Yu, J. Tian, F. Zhu, X. Yang, Y. Jin, and Y. Xu, “Large language models to accelerate organic chemistry synthesis,” Nature Machine Intelligence, Vol.7, No.7, pp. 1010-1022, 2025. https://doi.org/10.1038/s42256-025-01066-y
- [35] K. Singhal, T. Tu, J. Gottweis, R. Sayres, E. Wulczyn, L. Hou, K. Clark, S. Pfohl, H. Cole-Lewis, D. Neal, M. Schaekermann, A. Wang, M. Amin, S. Lachgar, P. Mansfield, S. Prakash, B. Green, E. Dominowska, B. A. y Arcas, N. Tomasev, Y. Liu, R. Wong, C. Semturs, S. S. Mahdavi, J. Barral, D. Webster, G. S. Corrado, Y. Matias, S. Azizi, A. Karthikesalingam, and V. Natarajan, “Towards expert-level medical question answering with large language models,” arXiv:2305.09617, 2023. https://doi.org/10.48550/arXiv.2305.09617
- [36] P. R. Liu, S. Bansal, J. Dinh, A. Pawar, R. Satishkumar, S. Desai, N. Gupta, X. Wang, and S. Hu, “Medchat: A multi-agent framework for multimodal diagnosis with large language models,” arXiv:2506.07400, 2025. https://doi.org/10.48550/arXiv.2506.07400
- [37] H. Liang, T. Yang, Z. Liu, W. Jian, Y. Chen, B. Li, Z. Yan, W. Xu, L. Chen, Y. Qi, Z. Wang, Y. Liao, P. Lin, J. Li, W. Wang, L. Li, M. Wang, Y. Zhang, L. Deng, T. Jiang, and J. He, “Lungdiag: Empowering artificial intelligence for respiratory diseases diagnosis based on electronic health records, a multicenter study,” MedComm, Vol.6, No.1, Article No.e70043, 2025. https://doi.org/10.1002/mco2.70043
- [38] Z. Zhang, D. Zhang-Li, J. Yu, L. Gong, J. Zhou, Z. Hao, J. Jiang, J. Cao, H. Liu, Z. Liu, L. Hou, and J. Li, “Simulating classroom education with LLM-empowered agents,” arXiv:2406.19226, 2024. https://doi.org/10.48550/arXiv.2406.19226
- [39] L. Chen, Z. Jiang, D. Xia, Z. Cai, L. Sun, P. Childs, and H. Zuo, “Bidtrainer: An LLMs-driven education tool for enhancing the understanding and reasoning in bio-inspired design,” Proc. of the 2024 CHI Conf. Factors in Computing Systems, Article No.676, 2024. https://doi.org/10.1145/3613904.3642887
- [40] S. Wu, O. Irsoy, S. Lu, V. Dabravolski, M. Dredze, S. Gehrmann, P. Kambadur, D. Rosenberg, and G. Mann, “BloombergGPT: A large language model for finance,” arXiv:2303.17564, 2023. https://doi.org/10.48550/arXiv.2303.17564
- [41] Y. Yang, Y. Tang, and K. Y. Tam, “InvestLM: A large language model for investment using financial domain instruction tuning,” arXiv:2309.13064, 2023. https://doi.org/10.48550/arXiv.2309.13064
- [42] B. Yang, Y. Zhang, L. Feng, Y. Chen, J. Zhang, X. Xu, N. Aierken, Y. Li, Y. Chen, G. Yang, Y. He, R. Huang, and S. Li, “AgriGPT: A large language model ecosystem for agriculture,” arXiv:2508.08632, 2025. https://doi.org/10.48550/arXiv.2508.08632
- [43] D. J. Samuel, I. Skarga-Bandurova, D. Sikolia, and M. Awais, “AgroLLM: Connecting farmers and agricultural practices through large language models for enhanced knowledge transfer and practical application,” arXiv:2503.04788, 2025. https://doi.org/10.48550/arXiv.2503.04788
- [44] K. Kumar, T. Ashraf, O. Thawakar, R. M. Anwer, H. Cholakkal, M. Shah, M.-H. Yang, P. H. S. Torr, F. S. Khan, and S. Khan, “LLM post-training: A deep dive into reasoning large language models,” arXiv:2502.21321, 2025. https://doi.org/10.48550/arXiv.2502.21321
- [45] J. Tack, J. Lanchantin, J. Yu, A. Cohen, I. Kulikov, J. Lan, S. Hao, Y. Tian, J. Weston, and X. Li, “LLM pretraining with continuous concepts,” arXiv:2502.08524, 2025. https://doi.org/10.48550/arXiv.2502.08524
- [46] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv:1412.6980, 2017. https://doi.org/10.48550/arXiv.1412.6980
- [47] I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” arXiv:1711.05101, 2019. https://doi.org/10.48550/arXiv.1711.05101
- [48] N. Shazeer and M. Stern, “Adafactor: Adaptive learning rates with sublinear memory cost,” arXiv:1804.04235, 2018. https://doi.org/10.48550/arXiv.1804.04235
- [49] S. Rajbhandari, J. Rasley, O. Ruwase, and Y. He, “ZeRo: Memory optimizations toward training trillion parameter models,” arXiv:1910.02054, 2020. https://doi.org/10.48550/arXiv.1910.02054
- [50] P. Micikevicius, S. Narang, J. Alben, G. Diamos, E. Elsen, D. Garcia, B. Ginsburg, M. Houston, O. Kuchaiev, G. Venkatesh, and H. Wu, “Mixed precision training,” arXiv:1710.03740, 2018. https://doi.org/10.48550/arXiv.1710.03740
- [51] A. Q. Jiang, A. Sablayrolles, A. Roux, A. Mensch, B. Savary, C. Bamford, D. S. Chaplot, D. d. l. Casas, E. B. Hanna, F. Bressand, G. Lengyel, G. Bour, G. Lample, L. R. Lavaud, L. Saulnier, M.-A. Lachaux, P. Stock, S. Subramanian, S. Yang, S. Antoniak, T. L. Scao, T. Gervet, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed, “Mixtral of experts,” arXiv:2401.04088, 2024. https://doi.org/10.48550/arXiv.2401.04088
- [52] N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean, “Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,” arXiv:1701.06538, 2017. https://doi.org/10.48550/arXiv.1701.06538
- [53] S. Jiang, T. Zheng, Y. Zhang, Y. Jin, L. Yuan, and Z. Liu, “Med-moe: Mixture of domain-specific experts for lightweight medical vision-language models,” arXiv:2404.10237, 2024. https://doi.org/10.48550/arXiv.2404.10237
- [54] A. Yang, B. Xiao, B. Wang et al., “Baichuan 2: Open large-scale language models,” arXiv:2309.10305, 2025. https://doi.org/10.48550/arXiv.2309.10305
- [55] A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu et al., “Qwen3 technical report,” arXiv:2505.09388, 2025. https://doi.org/10.48550/arXiv.2505.09388
- [56] T. GLM, :, A. Zeng, B. Xu et al., “ChatGLM: A family of large language models from GLM-130B to GLM-4 all tools,” arXiv:2406.12793, 2024. https://doi.org/10.48550/arXiv.2406.12793
- [57] Z. Cai, M. Cao, H. Chen, and other, “InternLM2 technical report,” arXiv:2403.17297, 2024. https://doi.org/10.48550/arXiv.2403.17297
- [58] E. Almazrouei, H. Alobeidli, A. Alshamsi, A. Cappelli, R. Cojocaru, M. Debbah, É. Goffinet, D. Hesslow, J. Launay, Q. Malartic, D. Mazzotta, B. Noune, B. Pannier, and G. Penedo, “The falcon series of open language models,” arXiv:2311.16867, 2023. https://doi.org/10.48550/arXiv.2311.16867
- [59] P. Zhao, H. Zhang, Q. Yu, Z. Wang, Y. Geng, F. Fu, L. Yang, W. Zhang, J. Jiang, and B. Cui, “Retrieval-augmented generation for AI-generated content: A survey,” arXiv:2402.19473, 2024. https://doi.org/10.48550/arXiv.2402.19473
- [60] K. Wu, E. Wu, K. Wei et al., “An automated framework for assessing how well LLMs cite relevant medical references,” Nature Communications, Vol.16, No.1, Article No.3615, 2025. https://doi.org/10.1038/s41467-025-58551-6
- [61] S. Zhang, L. Dong, X. Li, S. Zhang, X. Sun, S. Wang, J. Li, R. Hu, T. Zhang, F. Wu, and G. Wang, “Instruction tuning for large language models: A survey,” arXiv:2308.10792, 2025. https://doi.org/10.48550/arXiv.2308.10792
- [62] H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y. Tay, W. Fedus, Y. Li, X. Wang, M. Dehghani, S. Brahma, A. Webson, S. S. Gu, Z. Dai, M. Suzgun, X. Chen, A. Chowdhery, A. Castro-Ros, M. Pellat, K. Robinson, D. Valter, S. Narang, G. Mishra, A. Yu, V. Zhao, Y. Huang, A. Dai, H. Yu, S. Petrov, E. H. Chi, J. Dean, J. Devlin, A. Roberts, D. Zhou, Q. V. Le, and J. Wei, “Scaling instruction-finetuned language models,” arXiv:2210.11416, 2022. https://doi.org/10.48550/arXiv.2210.11416
- [63] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe, “Training language models to follow instructions with human feedback,” arXiv:2203.02155, 2022. https://doi.org/10.48550/arXiv.2203.02155
- [64] Z. Han, C. Gao, J. Liu, J. Zhang, and S. Q. Zhang, “Parameter-efficient fine-tuning for large models: A comprehensive survey,” arXiv:2403.14608, 2024. https://doi.org/10.48550/arXiv.2403.14608
- [65] B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter-efficient prompt tuning,” arXiv:2104.08691, 2021. https://doi.org/10.48550/arXiv.2104.08691
- [66] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” arXiv:2106.09685, 2021. https://doi.org/10.48550/arXiv.2106.09685
- [67] X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” arXiv:2101.00190, 2021. https://doi.org/10.48550/arXiv.2101.00190
- [68] X. Liu, K. Ji, Y. Fu, W. L. Tam, Z. Du, Z. Yang, and J. Tang, “P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks,” arXiv:2110.07602, 2022. https://doi.org/10.48550/arXiv.2110.07602
- [69] Z. Zhang, Y. Zhang, H. Yao, J. Luo, R. Zhao, B. Huang, J. Zhao, Y. Liao, K. Li, L. Zhao, J. Cao, F. Qi, and C. Yuan, “Xiwu: A basis flexible and learnable LLM for high energy physics,” arXiv:2404.08001, 2024. https://doi.org/10.48550/arXiv.2404.08001
- [70] S. Song, H. Xu, J. Ma, S. Li, L. Peng, Q. Wan, X. Liu, and J. Yu, “How to alleviate catastrophic forgetting in LLMs finetuning? Hierarchical layer-wise and element-wise regularization,” arXiv:2501.13669, 2025. https://doi.org/10.48550/arXiv.2501.13669
- [71] R. Schaeffer, B. Miranda, and S. Koyejo, “Are emergent abilities of large language models a mirage?,” arXiv:2304.15004, 2023. https://doi.org/10.48550/arXiv.2304.15004
- [72] J. Wei, J. Wei, Y. Tay, D. Tran, A. Webson, Y. Lu, X. Chen, H. Liu, D. Huang, D. Zhou, and T. Ma, “Larger language models do in-context learning differently,” arXiv:2303.03846, 2023. https://doi.org/10.48550/arXiv.2303.03846
- [73] J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou, “Chain-of-thought prompting elicits reasoning in large language models,” arXiv:2201.11903, 2023. https://doi.org/10.48550/arXiv.2201.11903
- [74] G. Chen, L. Fan, Z. Gong, N. Xie, Z. Li, Z. Liu, C. Li, Q. Qu, H. Alinejad-Rokny, S. Ni, and M. Yang, “AgentCourt: Simulating court with adversarial evolvable lawyer agents,” arXiv:2408.08089, 2025. https://doi.org/10.48550/arXiv.2408.08089
- [75] A. M. Mansourian, R. Ahmadi, M. Ghafouri, A. M. Babaei, E. B. Golezani, Z. Y. Ghamchi, V. Ramezanian, A. Taherian, K. Dinashi, A. Miri, and S. Kasaei, “A comprehensive survey on knowledge distillation,” arXiv:2503.12067, 2025. https://doi.org/10.48550/arXiv.2503.12067
- [76] T. Kitrungrotsakul, Y. Xu, and P. Srichola, “Knowledge distillation meets reinforcement learning: A cluster-driven approach to image processing,” Sensors, Vol.26, No.1, Article No.209, 2025. https://doi.org/10.3390/s26010209
- [77] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, “ReAct: Synergizing reasoning and acting in language models,” arXiv:2210.03629, 2022. https://doi.org/10.48550/arXiv.2210.03629
- [78] S. Yao, D. Yu, J. Zhao, I. Shafran, T. L. Griffiths, Y. Cao, and K. Narasimhan, “Tree of thoughts: Deliberate problem solving with large language models,” arXiv:2305.10601, 2023. https://doi.org/10.48550/arXiv.2305.10601
- [79] Z. Zhang, X. Bo, C. Ma, R. Li, X. Chen, Q. Dai, J. Zhu, Z. Dong, and J.-R. Wen, “A survey on the memory mechanism of large language model based agents,” arXiv:2404.13501, 2024. https://doi.org/10.48550/arXiv.2404.13501
- [80] S. Han, Q. Zhang, Y. Yao, W. Jin, and Z. Xu, “LLM multi-agent systems: Challenges and open problems,” arXiv:2402.03578, 2025. https://doi.org/10.48550/arXiv.2402.03578
- [81] A. Ehtesham, A. Singh, G. K. Gupta, and S. Kumar, “A survey of agent interoperability protocols: Model context protocol (MCP), agent communication protocol (ACP), agent-to-agent protocol (A2A), and agent network protocol (ANP),” arXiv:2505.02279, 2025. https://doi.org/10.48550/arXiv.2505.02279
- [82] Z. Jia, M. Jia, J. Duan, and J. Wang, “DDO: Dual-decision optimization for LLM-based medical consultation via multi-agent collaboration,” arXiv:2505.18630, 2025. https://doi.org/10.48550/arXiv.2505.18630
- [83] L. Long, Y. He, W. Ye, Y. Pan, Y. Lin, H. Li, J. Zhao, and W. Li, “Seeing, listening, remembering, and reasoning: A multimodal agent with long-term memory,” arXiv:2505.18630, 2025. https://doi.org/10.48550/arXiv.2508.09736
- [84] C. Chen, J. Yu, S. Chen, C. Liu, Z. Wan, D. Bitterman, F. Wang, and K. Shu, “Clinicalbench: Can LLMs beat traditional ML models in clinical prediction?,” arXiv:2411.06469, 2024. https://doi.org/10.48550/arXiv.2411.06469
- [85] B. Hawks, G. v. Laszewski, M. D. Sinclair, M. Colombo, S. Venkataraman, R. Jain, Y. Jiang, N. Tran, and G. Fox, “An MLcommons scientific benchmarks ontology,” arXiv:2511.05614, 2025. https://doi.org/10.48550/arXiv.2511.05614
- [86] Y. Xiao, E. Sun, T. Liu, and W. Wang, “Logicvista: Multimodal LLM logical reasoning benchmark in visual contexts,” arXiv:2407.04973, 2024. https://doi.org/10.48550/arXiv.2407.04973
- [87] Y. Zhou, X. Wu, B. Huang, J. Wu, L. Feng, and K. C. Tan, “CausalBN-Bench: A comprehensive benchmark for causal learning capability of LLMs,” arXiv:2404.06349, 2024. https://doi.org/10.48550/arXiv.2404.06349
- [88] Y. Zhang, Y. Huang, Y. Sun, C. Liu, Z. Zhao, Z. Fang, Y. Wang, H. Chen, X. Yang, X. Wei, H. Su, Y. Dong, and J. Zhu, “Multitrust: A comprehensive benchmark towards trustworthy multimodal large language models,” arXiv:2406.07057, 2024. https://doi.org/10.48550/arXiv.2406.07057
- [89] A. Wang, Y. Pruksachatkun, N. Nangia, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman, “SuperGLUE: A stickier benchmark for general-purpose language understanding systems,” arXiv:1905.00537, 2020. https://doi.org/10.48550/arXiv.1905.00537
- [90] Y. Li, H. Huang, Y. Zhao, X. Ma, and J. Sun, “BackdoorLLM: A comprehensive benchmark for backdoor attacks and defenses on large language models,” arXiv:2408.12798, 2025. https://doi.org/10.48550/arXiv.2408.12798
- [91] D. Zhang, W. Liu, Q. Tan, J. Chen, H. Yan, Y. Yan, J. Li, W. Huang, X. Yue, W. Ouyang, D. Zhou, S. Zhang, M. Su, H.-S. Zhong, and Y. Li, “ChemLLM: A chemical large language model,” arXiv:2402.06852, 2024. https://doi.org/10.48550/arXiv.2402.06852
- [92] L. Cardenas, K. Parajes, M. Zhu, and S. Zhai, “AutoHealth: Advanced LLM-empowered wearable personalized medical butler for parkinson’s disease management,” 2024 IEEE 14th Annual Computing and Communication Workshop and Conf. (CCWC), pp. 0375-0379, 2024. https://doi.org/10.1109/CCWC60891.2024.10427622
- [93] D. P. Kingma and M. Welling, “An introduction to variational autoencoders,” Foundations and Trends® in Machine Learning, Vol.12, No.4, pp. 307-392, 2019. https://doi.org/10.1561/2200000056
- [94] M. Li, H. Chen, Y. Wang, T. Zhu, W. Zhang, K. Zhu, K.-F. Wong, and J. Wang, “Understanding and mitigating the bias inheritance in LLM-based data augmentation on downstream tasks,” arXiv:2502.04419, 2025. https://doi.org/10.48550/arXiv.2502.04419
- [95] I. X. Weissburg, S. Anand, S. Levy, and H. Jeong, “LLMs are biased teachers: Evaluating llm bias in personalized education,” arXiv:2410.14012, 2024. https://doi.org/10.48550/arXiv.2410.14012
- [96] N. Pipitone and G. H. Alami, “LegalBench-RAG: A benchmark for retrieval-augmented generation in the legal domain,” arXiv:2408.10343, 2024. https://doi.org/10.48550/arXiv.2408.10343
- [97] H. Zhao, H. Chen, F. Yang, N. Liu, H. Deng, H. Cai, S. Wang, D. Yin, and M. Du, “Explainability for large language models: A survey,” ACM Trans. on Intelligent Systems and Technology, Vol.15, No.2, Article No.20, 2024. https://doi.org/10.1145/3639372
- [98] M. A. Aljohani, J. Hou, S. Kommu, and X. Wang, “A comprehensive survey on the trustworthiness of large language models in healthcare,” arXiv:2502.15871, 2025. https://doi.org/10.48550/arXiv.2502.15871
- [99] F. Naudot, T. Sundqvist, and T. Kampik, “llmSHAP: A principled approach to LLM explainability,” arXiv:2511.01311, 2025. https://doi.org/10.48550/arXiv.2511.01311
- [100] S. Sztwiertnia, F. Friedrich, K. Kersting, P. Schramowski, and B. Deiseroth, “LIME: Making LLM data more efficient with linguistic metadata embeddings,” arXiv:2512.07522, 2025. https://doi.org/10.48550/arXiv.2512.07522
- [101] H. Luo and L. Specia, “From understanding to utilization: A survey on explainability for large language models,” arXiv:2401.12874, 2024. https://doi.org/10.48550/arXiv.2401.12874
- [102] M. Sanwal, “Layered chain-of-thought prompting for multi-agent LLM systems: A comprehensive approach to explainable large language models,” arXiv:2501.18645, 2025. https://doi.org/10.48550/arXiv.2501.18645
- [103] J. S. Erickson, H. Santos, V. Pinheiro, J. P. McCusker, and D. L. McGuinness, “LLM experimentation through knowledge graphs: Towards improved management, repeatability, and verification,” J. of Web Semantics, Vol.85, Article No.100853, 2025. https://doi.org/10.1016/j.websem.2024.100853
- [104] J. Arabneydi, S. Islam, S. Das, S. K. Gottipati, W. Duguay, C. Mars, M. E. Taylor, M. J. Guzdial, A. Fagette, and Y. Zerouali, “A systematic approach to design real-world human-in-the-loop deep reinforcement learning: Salient features, challenges and trade-offs,” arXiv:2504.17006, 2025. https://doi.org/10.48550/arXiv.2504.17006
- [105] R. Patil and V. Gudivada, “A review of current trends, techniques, and challenges in large language models (LLMs),” Applied Sciences, Vol.14, No.5, Article No.2074, 2024. https://doi.org/10.3390/app14052074
- [106] K. Hau, S. Hassan, and S. Zhou, “LLMs in mobile apps: Practices, challenges, and opportunities,” 2025 IEEE/ACM 12th Int. Conf. on Mobile Software Engineering and Systems (MOBILESoft), pp. 3-14, 2025. https://doi.org/10.1109/mobilesoft66462.2025.00008
This article is published under a Creative Commons Attribution-NoDerivatives 4.0 Internationa License.