CoChat
Sign Up
Cited by 46Topic ModelingNatural Language Processing TechniquesMultimodal Machine Learning Applications

Large language models: an overview of foundational architectures, recent trends, and a new taxonomy

Open access
Mienye, Ibomoiye Domor, Jere, Nobert, Obaido, George, Ogunruku, Oyindamola Omolara, Esenogho, Ebenezer, Modisane, Cameron·2025·Discover Applied Sciences
DOI 10.1007/s42452-025-07668-w
Open this paper in CoChat — free accountOpen PDF

Abstract

Since the introduction of foundational models such as Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-trained Transformers (GPT), there has been rapid evolution in both the scale and application of large language models (LLMs). This review provides a concise overview of LLMs, their architecture, training methodologies, and recent innovative applications, focusing on notable models such as the GPT series, BERT, Pathways Language Model (PaLM), and Large Language Model Meta AI (LLaMA), and recently the DeepSeek-R1 model. Additionally, this paper presents a taxonomy for categorizing LLMs based on three critical dimensions: scalability, application domains, and ethical considerations. This taxonomy aims to enable researchers and practitioners to better understand these models in terms of their potential and limitations. Lastly, by reviewing contributions from numerous publications, this study identifies emerging trends, gaps, and opportunities in LLM research, providing a structured guide for future investigations.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s42452-025-07668-w

Cites

20 of 20 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Recent advances in GenAI have substantially expanded the capabilities of AI systems, enabling increasingly autonomous forms of human-AI interaction.

    “Recent advances in GenAI have substantially expanded the capabilities of AI systems, enabling increasingly autonomous forms of human-AI interaction (Mienye et al. 2025).”

    From Machine marketing: rethinking the customer in the age of generative AI (2026) · cites this paper as Mienye et al. 2025

    Section 2 (Related works; page unavailable) describes expanded LLM capabilities, including content generation and automated customer service. Section 8.3 (Ethical risk tiers; page unavailable) also discusses models used in autonomous systems. Neither passage establishes the claimed progression toward increasingly autonomous human–AI interaction. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 90

  1. Zhang et al. [107] 2023 Examined potential over-reliance on LLM-generated code and its implications on software quality. Legal and con- tractual analysis Katz et al. [108] 2024 Streamlined legal document review, contract analysis, and clause extraction, improving efficiency and reducing errors.
  2. Ye Q, Axmed M, Pryzant R, Khani F Prompt engineering a prompt engineer. arXiv preprint arXiv:2311.05661. 2023;1-39 · arXiv
  3. Li J, Tang T, Zhao WX, Nie J-Y, Wen J-R. Pre-trained language models for text generation: a survey. ACM Comput Surv. 2024;56(9):1-39.
  4. Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, et al. Language models are few-shot learners. Adv Neural Inf Process Syst. 2020;33:1877-901.
  5. Marvin G, Hellen N, Jjingo D, Nakatumba-Nabende J. Prompt engineering in large language models. In: Falkowski-Gilski P, Jacob IJ, Shanmugam SK, Piramuthu S, editors. Data intelligence and cognitive informatics: proceedings of the inter- national conference on data intelligence and cognitive informatics (ICDICI) algorithms for intelligent systems. Singapore: Springer; 2023. p. 387-402.
  6. Hagos DH, Battle R, Rawat DB. Recent advances in generative AI and large language models: Current status, challenges, and perspectives. IEEE Trans Artif Intell. 2024.
  7. Mienye ID, Sun Y, Ileberi E. Artificial intelligence and sustainable development in Africa: a comprehensive review. Mach Learn Appl. 2024;18:1-14.
  8. Chen B, Zhang Z, Langrené N, Zhu S. Unleashing the potential of prompt engineering in large language models: a com- prehensive review. arXiv preprint arXiv:2310.14735. 2023. · arXiv
  9. Arvidsson S, Axell J. Prompt engineering guidelines for LLMs in requirements engineering. PhD thesis, University of Gothenburg, Chalmers University of Technology. 2023.
  10. Abd-Alrazaq A, AlSaad R, Alhuwail D, Ahmed A, Healy PM, Latifi S, et al. Large language models in medical education: opportunities, challenges, and future directions. JMIR Med Edu. 2023;9(1):48291.
  11. Thirunavukarasu AJ, Ting DSJ, Elangovan K, Gutierrez L, Tan TF, Ting DSW. Large language models in medicine. Nat Med. 2023;29(8):1930-40.
  12. Williams CV. Bracing for impact: Revising legal writing assessments ahead of the collision of generative AI and the nextgen bar exam. Legal Writing J Legal Writing Inst. 2024;28:1.
  13. Papasavva A, Johnson S, Lowther E, Lundrigan S, Mariconti E, Markovska A, Tuptuk N Application of AI-based models for online fraud detection and analysis. arXiv preprint arXiv:2409.19022. 2024. · arXiv
  14. Nasseri M, Brandtner P, Zimmermann R, Falatouri T, Darbanian F, Obinwanne T. Applications of large language models (LLMs) in business analytics-exemplary use cases in data preparation tasks. In: International conference on human-com- puter interaction. Cham: Springer; 2023. p. 182-98.
  15. Bhat A, Shrivastava D, Guo JL. Do LLMs meet the needs of software tutorial writers? Opportunities and design implica- tions. In: Proceedings of the 2024 ACM designing interactive systems conference. 2024. p. 1760-73.
  16. Rahali A, Akhloufi MA. End-to-end transformer-based models in textual-based NLP. AI. 2023;4(1):54-110.
  17. Pandey R, Waghela H, Rakshit S, Rangari A, Singh A, Kumar R, Ghosal R, Sen J. Generative AI-based text generation meth- ods using pre-trained GPT-2 model. arXiv preprint arXiv:2404.01786. 2024;1-57. · arXiv
  18. Mienye ID, Swart TG. ChatGPT in education: a review of ethical challenges and approaches to enhancing transparency and privacy. Procedia Comput Sci. 2025;254:181-90.
  19. Achiam J, Adler S, Agarwal S, Ahmad L, Akkaya I, Aleman FL, Almeida D, Altenschmidt J, Altman S, Anadkat S, et al. GPT-4 technical report. arXiv preprint arXiv:2303.08774. 2023. · arXiv
  20. Devlin J, Chang M-W, Lee K, Toutanova K. BERT: Pre-training of deep bidirectional transformers for language understand- ing. 2019. https://arxiv.org/abs/1810.04805 · arXiv
  21. Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V. RoBERTa: a robustly optimized BERT Pretraining Approach. 2019. https://arxiv.org/abs/1907.11692. · arXiv
  22. Lan Z. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942. 2019. · arXiv
  23. Zaheer M, Guruganesh G, Dubey KA, Ainslie J, Alberti C, Ontanon S, et al. Big bird: transformers for longer sequences. Adv Neural Inf Process Syst. 2020;33:17283-97.
  24. Chowdhery A, Narang S, Devlin J, Bosma M, Mishra G, Roberts A, Barham P, Chung HW, Sutton C, Gehrmann S, Schuh P, Shi K, Tsvyashchenko S, Maynez J, Rao A, Barnes P, Tay Y, Shazeer N, Prabhakaran V, Reif E, Du N, Hutchinson B, Pope R, Bradbury J, Austin J, Isard M, Gur-Ari G, Yin P, Duke T, Levskaya A, Ghemawat S, Dev S, Michalewski H, Garcia X, Misra, V., Robinson, K., Fedus, L., Zhou, D., Ippolito, D., Luan, D., Lim, H., Zoph, B., Spiridonov, A., Sepassi, R., Dohan D, Agrawal S, Omernick M, Dai AM, Pillai TS, Pellat M, Lewkowycz A, Moreira E, Child R, Polozov O, Lee K, Zhou Z, Wang X, Saeta B, Diaz M, Firat O, Catasta M, Wei J, Meier-Hellstern K, Eck D, Dean J, Petrov S, Fiedel N 2022 PaLM: Scaling Language Modeling with Pathways. https://arxiv.org/abs/2204.02311. · arXiv
  25. Minaee S, Mikolov T, Nikzad N, Chenaghlu M, Socher R, Amatriain X, Gao J Large language models: a survey. arXiv preprint arXiv:2402.06196. 2024. · arXiv
  26. Zhang X, Rajabi N, Duh K, Koehn P Machine translation with large language models: Prompting, few-shot learning, and fine-tuning with qlora. In: Proceedings of the eighth conference on machine translation, 2023; p. 468-81
  27. Anil R, Dai AM, Firat O, Johnson M, Lepikhin D, Passos A, Shakeri S, Taropa E, Bailey P, Chen Z, et al. Palm 2 technical report. arXiv preprint arXiv:2305.10403. 2023. · arXiv
  28. Touvron H, Lavril T, Izacard G, Martinet X, Lachaux M-A, Lacroix T, Rozi re B, Goyal N, Hambro E, Azhar F, Rodriguez A, Joulin A, Grave E, Lample G. LLaMA: open and efficient foundation language models. 2023. https://arxiv.org/abs/2302.13971. · arXiv
  29. Joshi S, Khan MS, Dafe A, Singh K, Zope V, Jhamtani T. Fine tuning LLMs for low resource languages. In: 2024 5th Interna- tional Conference on Image Processing and Capsule Networks (ICIPCN). IEEE; 2024. p. 511-9
  30. Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, et al. Exploring the limits of transfer learning with a unified text- to-text transformer. J Mach Learn Res. 2020;21(140):1-67.
  31. Xue L. mT5: A massively multilingual pre-trained text-to-text transformer. arXiv preprint arXiv:2010.11934. 2020;1-17. · arXiv
  32. Team G, Anil R, Borgeaud S, Alayrac J-B, Yu J, Soricut R, Schalkwyk J, Dai AM, Hauth A, Millican K, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805. 2023;1-90. · arXiv
  33. DeepSeek-AI GD, Yang D, Zhang H, Song J, Zhang R, Xu R, Zhu Q, Ma S, Wang P, Bi X, Zhang X, Yu X, Wu Y, Wu ZF, Gou Z, Shao Z, Li Z, Gao Z, Liu A, Xue B, Wang B, Wu B, Feng B, Lu C, Zhao C, Deng C, Zhang C, Ruan C, Dai D, Chen D, Ji D, Li E, Lin F, Dai F, Luo F, Hao G, Chen G, Li G, Zhang H, Bao H, Xu H, Wang H, Ding H, Xin H, Gao H, Qu H, Li H, Guo J, Li J, Wang J, Chen J, Yuan J, Qiu J, Li J, Cai JL, Ni J, Liang J, Chen J, Dong K, Hu K, Gao K, Guan K, Huang K, Yu K, Wang L, Zhang L, Zhao L, Wang L, Zhang L, Xu L, Xia L, Zhang M, Zhang M, Tang M, Li M, Wang M, Li M, Tian N, Huang P, Zhang P, Wang Q, Chen Q, Du Q, Ge R, Zhang R, Pan R Wang R, Chen RJ, Jin RL, Chen R, Lu S, Zhou S, Chen S, Ye S, Wang S, Yu S, Zhou S, Pan S, Li SS, Zhou S, Wu S, Ye S, Yun T, Pei T, Sun T, Wang T, Zeng W, Zhao W, Liu W, Liang W, Gao W, Yu W, Zhang W, Xiao WL, An W, Liu X, Wang X, Chen X, Nie X, Cheng X, Liu X, Xie X, Liu X, Yang X, Li X, Su X, Lin X, Li XQ, Jin X, Shen X, Chen X, Sun X, Wang X, Song X, Zhou X, Wang X, Shan, X, Li YK, Wang YQ, Wei YX, Zhang Y, Xu Y, Li Y, Zhao Y, Sun Y, Wang Y, Yu Y, Zhang Y, Shi Y, Xiong Y, He Y, Piao Y, Wang Y, Tan Y, Ma Y, Liu Y, Guo Y, Ou Y, Wang Y, Gong Y, Zou Y, He Y, Xiong Y, Luo Y, You Y, Liu Y, Zhou Y, Zhu YX, Xu Y, Huang Y, Li Y, Zheng Y, Zhu Y, Ma Y, Tang Y, Zha Y Yan Y, Ren ZZ, Ren Z, Sha Z, Fu Z, Xu Z, Xie Z, Zhang Z, Hao Z, Ma Z, Yan Z, Wu Z, Gu Z, Zhu Z, Liu Z Li Z, Xie Z, Song Z, Pan Z, Huang Z, Xu Z, Zhang Z Zhang Z. DeepSeek-R1: incentiv- izing reasoning capability in llms via reinforcement learning. 2025. https://arxiv.org/abs/2501.12948. · arXiv
  34. Shao Z, Wang P, Zhu Q, Xu R, Song J, Bi X, Zhang H, Zhang M, Li Y, Wu Y, et al.:Deepseekmath: Pushing the limits of math- ematical reasoning in open language models. arXiv preprint arXiv:2402.03300. 2024. · arXiv
  35. Guo D, Yang D, Zhang H, Song J, Zhang R, Xu R, Zhu Q, Ma S, Wang P, Bi X, et al. Deepseek-r1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501.12948. 2025. · arXiv
  36. Patil R, Gudivada V. A review of current trends, techniques, and challenges in large language models (LLMs). Appl Sci. 2024;14(5):2074.
  37. Wang H, Li J, Wu H, Hovy E, Sun Y. Pre-trained language models and their applications. Engineering. 2023;25:51-65.
  38. Chen S, Hou Y, Cui Y, Che W. Liu T, Yu X. Recall and learn: Fine-tuning deep pretrained language models with less forget- ting. arXiv preprint arXiv:2004.12651. 2020. · arXiv
  39. Han X, Zhang Z, Ding N, Gu Y, Liu X, Huo Y, et al. Pre-trained models: past, present and future. AI Open. 2021;2:225-50.
  40. Yang J, Jin H, Tang R, Han X, Feng Q, Jiang H, et al. Harnessing the power of LLMs in practice: a survey on ChatGPT and beyond. ACM Trans Knowl Discov Data. 2024;18(6):1-32.
  41. Kucharavy A. Adapting LLMS to downstream applications. In: Large language models in cybersecurity: threats, exposure and mitigation technology monitoring series, cyber-defence campus. Switzerland: Springer, Cham; 2024. p. 19-29.
  42. Zhu B, Jordan M, Jiao J. Principled reinforcement learning with human feedback from pairwise or k-wise comparisons. In: International conference on machine learning. PMLR; 2023:p. 43037-67.
  43. Wang X, Zhu W, Saxon M, Steyvers M, Wang WY. Large language models are latent variable models: Explaining and finding good demonstrations for in-context learning. Adv Neurel Inf Process Syst. 2024;36:15614-38.
  44. Lewis P, Perez E, Piktus A, Petroni F, Karpukhin V, Goyal N, et al. Retrieval-augmented generation for knowledge-intensive NLP tasks. Adv Neural Inf Process Syst. 2020;33:9459-74.
  45. Kamath U, Keenan K, Somers G, Sorenson S. Retrieval-augmented generation. In: Large language models: a deep dive: bridging theory and practice. Cham, Switzerland: Springer; 2024. p. 275-313.
  46. Geroimenko V. Key concepts in prompt engineering. In: The essential guide to prompt engineering: key principles. Tech- niques, challenges, and security risks. Cham, Switzerland: Springer; 2025. p. 1-16.
  47. Kojima T, Gu SS, Reid M, Matsuo Y, Iwasawa Y. Large language models are zero-shot reasoners. Adv Neural Inf Process Syst. 2022;35:22199-213.
  48. Wei J, Wang X, Schuurmans D, Bosma M, Xia F, Chi E, et al. Chain-of-thought prompting elicits reasoning in large language models. Adv Neural Inf Process Syst. 2022;35:24824-37.
  49. Wang L, Chen X, Deng X, Wen H, You M, Liu W, et al. Prompt engineering in consistency and reliability with the evidence- based guideline for LLMs. NPJ digital medicine. 2024;7(1):41.
  50. Li M, Mishra A, Mujumdar U. Bridging the language gap: enhancing multilingual prompt-based code generation in LLMs via zero-shot cross-lingual transfer. arXiv preprint arXiv:2408.09701. 2024:1-9. · arXiv
  51. Nagano T, Kurata G, Thomas S, Kuo H-KJ, Bolanos D, Jung H, Saon G. LLM-based text generation for improved low-resource speech recognition models. In: ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Process- ing (ICASSP), Hyderabad, India. IEEE; 2025. p. 1-5.
  52. Guang J, Chao J, Tianqi Z, Siya C, Chaoyuan C, Jun F. Data redundancy elimination and noise processing via large language model prompt engineering. In: 2024 10th International Conference on Big Data and Information Analytics (BigDIA), Chiang Mai, Thailand. IEEE; 2024. 818-825.
  53. Ijebu FF, Liu Y, Sun C, Jere N, Mienye ID, Inyang UG. Cross-encoder-based semantic evaluation of extractive and generative question answering in low-resourced African languages. Technologies. 2025;13(3):119.
  54. Colla D, Delsanto M, Agosto M, Vitiello B, Radicioni DP. Semantic coherence markers: the contribution of perplexity met- rics. Artif Intell Med. 2022;134:102393.
  55. Chen Y, Zulkernine F. Bird-qa: a bert-based information retrieval approach to domain specific question answering. In: 2021 IEEE International Conference On Big Data (Big Data). IEEE; 2021. p. 3503-3510
  56. Engelbach M, Klau D, Scheerer F, Drawehn J, Kintz M. Fine-tuning and aligning question answering models for complex information extraction tasks. arXiv preprint arXiv:2309.14805. 2023. · arXiv
  57. Obaido G, Ogbuokiri B, Mienye ID, Kasongo SMA. voting classifier for mortality prediction post-thoracic surgery. In: Inter- national conference on intelligent systems design and applications. Cham: Springer; 2022. p. 263-72.
  58. Han C, Lu X. Can automated machine translation evaluation metrics be used to assess students interpretation in the language learning classroom? Comput Assist Lang Learn. 2023;36(5-6):1064-87.
  59. Lin C-Y. Rouge: A package for automatic evaluation of summaries. In: Text summarization branches out, 2004;pp. 74-81.
  60. Zhang M, Li C, Wan M, Zhang X, Zhao Q. Rouge-sem: Better evaluation of summarization using rouge combined with semantics. Expert Syst Appl. 2024;237:121364.
  61. Srivastava R, Singh P, Rana K, Kumar V. A topic modeled unsupervised approach to single document extractive text sum- marization. Knowl-Based Syst. 2022;246:108636.
  62. Adams G, Shing H-C, Sun Q, Winestock C, McKeown K, Elhadad N. Learning to revise references for faithful summarization. arXiv preprint arXiv:2204.10290. 2022. · arXiv
  63. Mienye ID, Swart TG. Deep autoencoder neural networks: a comprehensive review and new perspectives. Arch Comput Methods Eng. 2025;1-20.
  64. Yuan J, Zhang J, Wen A, Hu X. The science of evaluating foundation models. arXiv preprint arXiv:2502.09670. 2025. · arXiv
  65. Wang A, Pruksachatkun Y, Nangia N, Singh A, Michael J, Hill F, Levy O, Bowman SR. SuperGLUE: A stickier benchmark for general-purpose language understanding systems. 2020. https://arxiv.org/abs/1905.00537 · arXiv
  66. Hendrycks D, Burns C, Basart S, Zou A, Mazeika M, Song D, Steinhardt J. Measuring massive multitask language under- standing. 2021. https://arxiv.org/abs/2009.03300. · arXiv
  67. Solaiman I, Talat Z, Agnew W, Ahmad L, Baker D, Blodgett SL, Chen C, Daumé III H, Dodge J, Duan I, et al. Evaluating the social impact of generative AI systems in systems and society. arXiv preprint arXiv:2306.05949. 2023. · arXiv
  68. Mienye ID, Swart TG, Obaido G. Fairness metrics in AI healthcare applications: a review. In: 2024 IEEE International confer- ence on information reuse and integration for data science (IRI). IEEE; 2024. p.284-289.
  69. Berengueres J. How to regulate large language models for responsible AI. IEEE Trans Technol Soc. 2024:1-7.
  70. Mienye ID, Obaido, G., Emmanuel, I.D., Ajani, A.A.: A survey of bias and fairness in healthcare AI. In: 2024 IEEE 12th Interna- tional conference on healthcare informatics (ICHI). IEEE; 2024. p.642-650.
  71. Canton H. Organisation for economic co-operation and development OECD. In: Publications E, editor. The Europa direc- tory of international organizations 2021. 23rd ed. London: Routledge; 2021. p. 677-87.
  72. Akhtar ZB. Unveiling the evolution of generative AI (gai): a comprehensive and investigative analysis toward LLM models (2021-2024) and beyond. J Electr Syst Inf Technol. 2024;11(1):22.
  73. Chua J, Li Y, Yang S, Wang C, Yao L. AI safety in generative AI large language models: a survey. arXiv preprint arXiv:2407.18369. 2024. 100. · arXiv
  74. Aejas B, Belhi A, Zhang H, Bouras A. Deep learning-based automatic analysis of legal contracts: a named entity recognition benchmark. Neural Comput Appl. 2024;36:1-17. 110.
  75. Padiu B, Iacob R, Rebedea T, Dascalu M. To what extent have LLMs reshaped the legal domain so far? A scoping literature review. Information. 2024;15(11):662. 111. Nay JJ, Karamardian D, Lawsky SB, Tao W, Bhat M, Jain R, et al. Large language models as tax attorneys: a case study in legal capabilities emergence. Phil Trans R Soc A. 2024;382(2270):20230159. 112. Yang X, Wang Z, Wang Q, Wei K, Zhang K, Shi J. Large language models for automated q &a involving legal documents: a survey on algorithms, frameworks and applications. Int J Web Inf Syst. 2024;20(4):413-35. 113. Harasta J, Novotná T, Savelka J. It cannot be right if it was written by AI: On lawyers' preferences of documents perceived as authored by an LLM vs a human. arXiv preprint arXiv:2407.06798. 2024. 114. Kim Y, Park C, Jeong H, Chan YS, Xu X, McDuff D, Breazeal C, Park HW Adaptive collaboration strategy for LLMs in medical decision making. arXiv preprint arXiv:2404.15155. 2024. 115. · arXiv
  76. Nazi, Z.A., Peng, W.: Large language models in healthcare and medical domain: a review. In: Informatics. (vol. 11). MDPI; 2024. p.57. 116.
  77. Singhal K, Azizi S, Tu T, Mahdavi SS, Wei J, Chung HW, et al. Large language models encode clinical knowledge. Nature. 2023;620(7972):172-80. 117.
  78. Gardner J, Durand S. Stoller D, Bittner RM. Llark: A multimodal foundation model for music. arXiv preprint arXiv:2310.07160. 2023;1-46. 130. Liu H-H, Liu Y-W. Agent-driven large language models for Mandarin lyric generation. arXiv preprint arXiv:2410.01450. 2024;1-6. 131. Sarker IH. Generative AI and large language modeling in cybersecurity. In: AI-driven cybersecurity and threat intelligence: Cyber automation. Intelligent decision-making and explainability. Cham: Springer; 2024. p. 79-99. 132. · arXiv
  79. Bethany M, Galiopoulos A, Bethany E, Karkevandi MB, Vishwamitra N, Najafirad P. Large language model lateral spear phishing: A comparative study in large-scale organizational settings. arXiv preprint arXiv:2401.09727. 2024. 133. Elouardi S, Motii A, Jouhari M, Amadou ANH, Hedabou M. A survey on hybrid-CNN and LLMs for intrusion detection systems: Recent iot datasets. IEEE Access. 2024. 1-25. 134. · arXiv
  80. Lefort B, Benhamou E, Ohana J-J, Saltiel D, Guez B, Challet D. Can chatGPT compute trustworthy sentiment scores from bloomberg market wraps? arXiv preprint arXiv:2401.05447. 2024;1-32. 141. Bhatia G, Nagoudi EMB, Cavusoglu H, Abdul-Mageed M. Fintral: A family of GPT-4 level multimodal financial large lan- guage models. arXiv preprint arXiv:2402.10986. 2024;1-24. 142. Zhao H, Liu Z, Wu Z, Li Y, Yang T, Shu P, Xu S, Dai H, Zhao L, Mai G, et al. Revolutionizing finance with LLMs: An overview of applications and insights. arXiv preprint arXiv:2401.11641. 2024. 143. · arXiv
  81. Sanh V, Debut L, Chaumond J, Wolf T. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. 2020. https://arxiv.org/abs/1910.01108. 144. · arXiv
  82. Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I. Language models are unsupervised multitask learners. OpenAI blog. 2019;1(8):9. 145.
  83. Lee J, Yoon W, Kim S, Kim D, Kim S, So CH, et al. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics. 2019;36(4):1234-40. h t t p s : / / d o i . o r g / 1 0 . 1 0 9 3 / b i o i n f o · DOI · Paper page
  84. Chalkidis I, Fergadiotis M, Malakasiotis P, Aletras N, Androutsopoulos I. LEGAL-BERT: The muppets straight out of law school. 2020. https://arxiv.org/abs/2010.02559. 147. Ramesh A, Pavlov M, Goh G, Gray S, Voss C, Radford A, Chen M, Sutskever I. Zero-shot text-to-image generation. 2021. https://arxiv.org/abs/2102.12092. 148. Plank B, AgiC Z. Bert syntactic transfer: a computational experiment on Italian, French and English languages. Comput Linguist. 2020;71:101261. 149. Conneau A, Lample G. Cross-lingual language model pretraining. NeurIPS. 2019;1-11. 150. · arXiv
  85. Conneau A, Khandelwal K, Goyal N, Chaudhary V, Wenzek G, Guzmán F, Grave E, Ott M, Zettlemoyer L, Stoyanov V. Unsu- pervised cross-lingual representation learning at scale. ACL. 2020. 151. Xue L, Constant N, Roberts A, Kale M, Al-Rfou R, Siddhant A, Barua A, Raffel C. mt5: A massively multilingual pre-trained text-to-text transformer. In: Proceedings of the 2021 Conference of the North American chapter of the association for computational linguistics (NAACL), p. 483-498. Association for Computational Linguistics, ??? 2021. 152. Scao TL, Rush A, et al. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint. 2022;1-73. arXiv:2211.05100. 153. Wang Y, Liu T, Zhang Q. Multilingual LLMs are better cross-lingual in-context learners with alignment. Transactions of the ACL. 2024. p. 1-14. 154. Zhao L, Hassan S, Chen W. Don't trust ChatGPT when your question is not in english: a study of multilingual abilities and types of llms. Findings of EMNLP. 2024. 155. · arXiv
  86. Shah SB, Thapa S, Acharya A, Rauniyar K, Poudel S, Jain S, Masood A, Naseem U. Navigating the web of disinformation and misinformation: Large language models as double-edged swords. IEEE Access, 2024. p. 1-1. h t t p s : / / d o i . o r g / 1 0 . 1 1 0 9 / A C C E S S . 2 0 2 4 . 3 4 0 6 6 4 4 . 165. · DOI
  87. Hendrycks D, Burns C, Kadavath S, Arora A, Basart S, Tang E, Song D, Steinhardt J. Measuring Mathematical Problem Solv- ing With the MATH Dataset. 2021. arXiv:org/abs/2103.03874. 166.
  88. Radford A, Kim JW, Hallacy C, Ramesh A, Goh, G, Agarwal S, Sastry G, Askell A, Mishkin P, Clark J, Krueger G, Sutskever I. Learning transferable visual models from natural language supervision. In: Proceedings of the 38th International Confer- ence on Machine Learning. Proceedings of Machine Learning Research, vol. 139. PMLR, Virtual; 2021. p. 8748-8763. 167. Mienye ID, Obaido GR, Obi T. Assessing large language models in children's education in low-resource settings: Opportu- nities and challenges. In: Proceedings of the Workshop on AI for Children: Healthcare, Psychology, Education, Vancouver, Canada 2025. ICLR; 2025. 168. Kairouz P, McMahan HB, Avent B, Bellet A, Bennis M, Bhagoji AN, et al. Advances and open problems in federated learning. Found Trends® Mach Learn. 2021;14(1-76):1-210. 169.
  89. Patterson D, Gonzalez J, Le Q, Liang C, Munguia L-M, Rothchild D, So D, Texier M, Dean J. Carbon emissions and large neural network training. 2021. arxiv:org/abs/2104.10350 170.
  90. Siino M, Tinnirello I. GPT prompt engineering for scheduling appliances usage for energy cost optimization. In: 2024 IEEE international symposium on measurements & networking (M &N). IEEE; 2024. p.1-6.

Cited by

Sorted by citations

Authors
Mienye, I. D., Jere, N., Obaido, G., Ogunruku, O. O., Esenogho, E., & Modisane, C.
Affiliations
  • University of South Africa (ZA)
  • University of Fort Hare (ZA)
  • Western Illinois University (US)
Corresponding author
Obaido, George
ORCID
Venue
Discover Applied Sciences
Publisher
Springer Science and Business Media LLC
Published
2025-09-02
Volume
7
Issue
9
ISSN
3004-9261 (ISSN-L)
DOI
10.1007/s42452-025-07668-w
OpenAlex
W4413933064
Cited by
46
References
20
Citations by year
2026: 40 · 2025: 6
FWCI
18.45
Type
journal article
Language
en
Open access
Licence
Locations
  • Discover Applied Sciences (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
University of California
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
large language models, LLM architecture, transformer models, BERT, GPT, Arecaceae, llamas, DeepSeek-R1, scalability, ethical considerations, LLM applications

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review