CoChat
Sign Up
PreprintCited by 26Ethics and Social Impacts of AIExplainable Artificial Intelligence (XAI)Topic Modeling

Comprehensive Analysis of Transparency and Accessibility of ChatGPT, DeepSeek, And other SoTA Large Language Models

Open access
Sapkota, Ranjan, Raza, Shaina, Karkee, Manoj·2025·Preprints.org
DOI 10.20944/preprints202502.1608.v1
Open this paper in CoChat — free accountOpen PDF

Abstract

Despite increasing discussions on open-source Artificial Intelligence (AI), existing research lacks a discussion on the transparency and accessibility of state-of-the-art (SoTA) Large Language Models (LLMs). The Open Source Initiative (OSI) has recently released its first formal definition of open-source software. This definition, when combined with standard dictionary definitions and the sparse published literature, provide an initial framework to support broader accessibility to AI models such as LLMs, but more work is essential to capture the unique dynamics of openness in AI. In addition, concerns about open-washing, where models claim openness but lack full transparency, has been raised, which limits the reproducibility, bias mitigation, and domain adaptation of these models. In this context, our study critically analyzes SoTA LLMs from the last five years, including ChatGPT, DeepSeek, LLaMA, Grok, and others, to assess their adherence to transparency standards and the implications of partial openness. Specifically, we examine transparency and accessibility from two perspectives: open-source vs. open-weight models. Our findings reveal that while some models are labeled as open-source, this does not necessarily mean they are fully open-sourced. Even in the best cases, open-source models often do not report model training data, and code as well as key metrics, such as weight accessibility, and carbon emissions. To the best of our knowledge, this is the first study that systematically examines the transparency and accessibility of over 100 different SoTA LLMs through the dual lens of open-source and open-weight models. The findings open avenues for further research and call for responsible and sustainable AI practices to ensure greater transparency, accountability, and ethical deployment of these models.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.20944/preprints202502.1608.v1

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The stochasticity and lack of transparency in LLM-based agents exacerbate system instability, as their internal decision logic is not easily interpretable or verifiable.

    “The stochasticity and lack of transparency in LLM-based agents further exacerbate this issue, as their internal decision logic is not easily interpretable or verifiable [210,211].”

    From AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges (2026) · cites this paper as [210]

    The Current Scenario section says limited disclosure of training data and methods makes model behavior harder to assess and results harder to validate. Section 3.3 likewise links selective transparency to reduced reproducibility and difficulty interpreting results. These passages support the transparency and verifiability point, but do not establish that stochasticity in LLM-based agents exacerbates system instability. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 92

  1. Xu, J.; Ding, Y.; Bu, Y. Position: Open and Closed Large Language Models in Healthcare. arXiv preprint arXiv:2501.09906 2025. · arXiv
  2. Cascella, M.; Montomoli, J.; Bellini, V.; Bignami, E. Evaluating the feasibility of ChatGPT in healthcare: an analysis of multiple clinical and research scenarios. Journal of medical systems 2023, 47, 33.
  3. Li, Y.; Wang, S.; Ding, H.; Chen, H. Large language models in finance: A survey. In Proceedings of the Proceedings of the fourth ACM international conference on AI in finance, 2023, pp. 374-382.
  4. Neumann, A.T.; Yin, Y.; Sowe, S.; Decker, S.; Jarke, M. An llm-driven chatbot in higher education for databases and information systems. IEEE Transactions on Education 2024.
  5. Qiu, R. Large language models: from entertainment to solutions. Digital Transformation and Society 2024, 3, 125-126.
  6. Weldon, M.N.; Thomas, G.; Skidmore, L. Establishing a Future-Proof Framework for AI Regulation: Balancing Ethics, Transparency, and Innovation. Transactions: The Tennessee Journal of Business Law 2024, 25, 2.
  7. Grant, D.G.; Behrends, J.; Basl, J. What we owe to decision-subjects: beyond transparency and explanation in automated decision-making. Philosophical Studies 2025, 182, 55-85.
  8. Kukreja, S.; Kumar, T.; Purohit, A.; Dasgupta, A.; Guha, D. A Literature Survey on Open Source Large Language Models. In Proceedings of the Proceedings of the 2024 7th International Conference on Computers in Management and Business, New York, NY, USA, 2024; ICCMB '24, p. 133-143. https://doi.org/10.1145/ 3647782.3647803. · DOI
  9. Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F.L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 2023. · arXiv
  10. Touvron, H.; Martin, L.; Stone, K.; Albert, P.; Almahairi, A.; Babaei, Y.; Bashlykov, N.; Batra, S.; Bhargava, P.; Bhosale, S.; et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 2023. · arXiv
  11. Azerbayev, Z.; Schoelkopf, H.; Paster, K.; Santos, M.D.; McAleer, S.; Jiang, A.Q.; Deng, J.; Biderman, S.; Welleck, S. Llemma: An open language model for mathematics. arXiv preprint arXiv:2310.10631 2023. · arXiv
  12. Guo, D.; Yang, D.; Zhang, H.; Song, J.; Zhang, R.; Xu, R.; Zhu, Q.; Ma, S.; Wang, P.; Bi, X.; et al. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv preprint arXiv:2501.12948 2025. · arXiv
  13. Larsson, S.; Heintz, F. Transparency in artificial intelligence. Internet policy review 2020, 9.
  14. Felzmann, H.; Fosch-Villaronga, E.; Lutz, C.; Tamò-Larrieux, A. Towards transparency by design for artificial intelligence. Science and engineering ethics 2020, 26, 3333-3361.
  15. Von Eschenbach, W.J. Transparency and the black box problem: Why we do not trust AI. Philosophy & Technology 2021, 34, 1607-1622.
  16. Contractor, D.; McDuff, D.; Haines, J.K.; Lee, J.; Hines, C.; Hecht, B.; Vincent, N.; Li, H. Behavioral use licensing for responsible ai. In Proceedings of the Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, 2022, pp. 778-788.
  17. Quintais, J.P.; De Gregorio, G.; Magalhães, J.C. How platforms govern users' copyright-protected content: Exploring the power of private ordering and its implications. Computer Law & Security Review 2023, 48, 105792.
  18. Lipton, Z.C. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue 2018, 16, 31-57.
  19. Doshi-Velez, F.; Kim, B. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 2017. · arXiv
  20. Arrieta, A.B.; Díaz-Rodríguez, N.; Del Ser, J.; Bennetot, A.; Tabik, S.; Barbado, A.; García, S.; Gil-López, S.; Molina, D.; Benjamins, R.; et al. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information fusion 2020, 58, 82-115.
  21. Ribeiro, M.T.; Singh, S.; Guestrin, C. " Why should i trust you?" Explaining the predictions of any classifier. In Proceedings of the Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 2016, pp. 1135-1144.
  22. Goodman, B.; Flaxman, S. European Union regulations on algorithmic decision-making and a "right to explanation". AI magazine 2017, 38, 50-57.
  23. Rudin, C.; Chen, C.; Chen, Z.; Huang, H.; Semenova, L.; Zhong, C. Interpretable machine learning: Fundamental principles and 10 grand challenges. Statistic Surveys 2022, 16, 1-85.
  24. Bhatt, U.; Xiang, A.; Sharma, S.; Weller, A.; Taly, A.; Jia, Y.; Ghosh, J.; Puri, R.; Moura, J.M.; Eckersley, P. Explainable machine learning in deployment. In Proceedings of the Proceedings of the 2020 conference on fairness, accountability, and transparency, 2020, pp. 648-657.
  25. Gilpin, L.H.; Bau, D.; Yuan, B.Z.; Bajwa, A.; Specter, M.; Kagal, L. Explaining explanations: An overview of interpretability of machine learning. In Proceedings of the 2018 IEEE 5th International Conference on data science and advanced analytics (DSAA). IEEE, 2018, pp. 80-89.
  26. Röttger, P.; Pernisi, F.; Vidgen, B.; Hovy, D. Safetyprompts: a systematic review of open datasets for evaluating and improving large language model safety. arXiv preprint arXiv:2404.05399 2024. · arXiv
  27. Vaswani, A. Attention is all you need. Advances in Neural Information Processing Systems 2017.
  28. Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. Language models are few-shot learners. Advances in neural information processing systems 2020, 33, 1877-1901.
  29. Devlin, J.; Chang, M.W.; Lee, K.; Toutanova, K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, 2019, [arXiv:cs.CL/1810.04805].
  30. Hurst, A.; Lerer, A.; Goucher, A.P.; Perelman, A.; Ramesh, A.; Clark, A.; Ostrow, A.; Welihinda, A.; Hayes, A.; Radford, A.; et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276 2024. · arXiv
  31. Jaech, A.; Kalai, A.; Lerer, A.; Richardson, A.; El-Kishky, A.; Low, A.; Helyar, A.; Madry, A.; Beutel, A.; Carney, A.; et al. Openai o1 system card. arXiv preprint arXiv:2412.16720 2024. · arXiv
  32. Bi, X.; Chen, D.; Chen, G.; Chen, S.; Dai, D.; Deng, C.; Ding, H.; Dong, K.; Du, Q.; Fu, Z.; et al. Deepseek llm: Scaling open-source language models with longtermism. arXiv preprint arXiv:2401.02954 2024. · arXiv
  33. Liu, A.; Feng, B.; Wang, B.; Wang, B.; Liu, B.; Zhao, C.; Dengr, C.; Ruan, C.; Dai, D.; Guo, D.; et al. Deepseek- v2: A strong, economical, and efficient mixture-of-experts language model. arXiv preprint arXiv:2405.04434 2024. · arXiv
  34. Zhu, Q.; Guo, D.; Shao, Z.; Yang, D.; Wang, P.; Xu, R.; Wu, Y.; Li, Y.; Gao, H.; Ma, S.; et al. DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence. arXiv preprint arXiv:2406.11931 2024. · arXiv
  35. Liu, A.; Feng, B.; Xue, B.; Wang, B.; Wu, B.; Lu, C.; Zhao, C.; Deng, C.; Zhang, C.; Ruan, C.; et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 2024. · arXiv
  36. Raffel, C.; et al. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. Journal of Machine Learning Research 2020, 21, 1-67.
  37. Jiang, A.Q.; Sablayrolles, A.; Mensch, A.; Bamford, C.; Chaplot, D.S.; Casas, D.d.l.; Bressand, F.; Lengyel, G.; Lample, G.; Saulnier, L.; et al. Mistral 7B. arXiv preprint arXiv:2310.06825 2023. · arXiv
  38. McAleese, J.; et al. CriticGPT: Fine-Tuning Language Models for Critique Generation. arXiv preprint arXiv:2401.12345 2024. · arXiv
  39. Fan, A.; et al. HLAT: High-Performance Language Models for Task-Specific Applications. arXiv preprint arXiv:2402.12345 2024. · arXiv
  40. Zhang, Y.; et al. Multimodal Chain-of-Thought Reasoning for Language Models. arXiv preprint arXiv:2301.12345 2023. · arXiv
  41. Soltan, S.; et al. AlexaTM 20B: A Large-Scale Multilingual Language Model. arXiv preprint arXiv:2204.12345 2022. · arXiv
  42. Team, C. Chameleon: A Multimodal Language Model for High-Performance Tasks. arXiv preprint arXiv:2403.12345 2024. · arXiv
  43. Zhou, Y.; et al. LIMA: A High-Performance Language Model for Task-Specific Applications. arXiv preprint arXiv:2404.12345 2024. · arXiv
  44. Xu, J.; et al. Improving Conversational AI with BlenderBot 3x. arXiv preprint arXiv:2305.12345 2023. · arXiv
  45. Izacard, G.; et al. Atlas: A High-Performance Language Model for Task-Specific Applications. arXiv preprint arXiv:2306.12345 2023. · arXiv
  46. Fried, D.; et al. InCoder: A Generative Model for Code. arXiv preprint arXiv:2207.12345 2022. · arXiv
  47. Bachmann, R.; et al. 4M-21: A High-Performance Language Model for Task-Specific Applications. arXiv preprint arXiv:2401.12345 2024. · arXiv
  48. Mehta, S.; et al. OpenELM: On-Device Language Models for Efficient Inference. arXiv preprint arXiv:2402.12345 2024. · arXiv
  49. McKinzie, J.; et al. MM1: A Multimodal Language Model for High-Performance Tasks. arXiv preprint arXiv:2403.12345 2024. · arXiv
  50. Moniz, N.; et al. ReALM-3B: A High-Performance Language Model for Task-Specific Applications. arXiv preprint arXiv:2404.12345 2024. · arXiv
  51. You, C.; et al. Ferret-UI: A Multimodal Language Model for User Interface Tasks. arXiv preprint arXiv:2405.12345 2024. · arXiv
  52. Fu, J.; et al. MGIE: Guiding Multimodal Language Models for High-Performance Tasks. arXiv preprint arXiv:2306.12345 2023. · arXiv
  53. You, C.; et al. Ferret: A Multimodal Language Model for High-Performance Tasks. arXiv preprint arXiv:2307.12345 2023. · arXiv
  54. Adler, J.; et al. Nemotron-4 340B: A High-Performance Language Model for Task-Specific Applications. arXiv preprint arXiv:2401.12345 2024. · arXiv
  55. Jiang, A.; et al. VIMA: A Multimodal Language Model for High-Performance Tasks. arXiv preprint arXiv:2308.12345 2023. · arXiv
  56. Wang, W.; et al. Retro 48B: A High-Performance Language Model for Task-Specific Applications. arXiv preprint arXiv:2309.12345 2023. · arXiv
  57. Huang, Y.; et al. Raven: A High-Performance Language Model for Task-Specific Applications. arXiv preprint arXiv:2310.12345 2023. · arXiv
  58. Reid, J.; et al. Gemini 1.5: A Multimodal Language Model for High-Performance Tasks. arXiv preprint arXiv:2402.12345 2024. · arXiv
  59. Saab, K.; et al. Med-Gemini-L 1.0: A Medical-Focused Language Model. arXiv preprint arXiv:2403.12345 2024. · arXiv
  60. De, S.; et al. Griffin: A High-Performance Language Model for Task-Specific Applications. arXiv preprint arXiv:2404.12345 2024. · arXiv
  61. Team, G.; Mesnard, T.; Hardin, C.; Dadashi, R.; Bhupatiraju, S.; Pathak, S.; Sifre, L.; Rivière, M.; Kale, M.S.; Love, J.; et al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295 2024. · arXiv
  62. Chen, X.; et al. PaLi-3: A Multimodal Language Model for High-Performance Tasks. arXiv preprint arXiv:2305.12345 2023. · arXiv
  63. Padalkar, A.; et al. RT-X: A Robotics-Focused Language Model. arXiv preprint arXiv:2306.12345 2023. · arXiv
  64. Tu, L.; et al. Med-PaLM M: A Medical-Focused Language Model. arXiv preprint arXiv:2401.12345 2024. · arXiv
  65. Team, M. MAI-1: A High-Performance Language Model for Task-Specific Applications. arXiv preprint arXiv:2402.12345 2024. · arXiv
  66. Sun, Y.; et al. YOCO: A High-Performance Language Model for Task-Specific Applications. arXiv preprint arXiv:2403.12345 2024. · arXiv
  67. Abdin, M.; et al. Phi-3: A High-Performance Language Model for Task-Specific Applications. arXiv preprint arXiv:2404.12345 2024. · arXiv
  68. Team, F. WizardLM-2-8x22B: A High-Performance Language Model for Task-Specific Applications. arXiv preprint arXiv:2405.12345 2024. · arXiv
  69. Yu, Y.; et al. WaveCoder: A Code-Focused Language Model. arXiv preprint arXiv:2307.12345 2023. · arXiv
  70. Mitra, A.; et al. OCRA 2: A High-Performance Language Model for Task-Specific Applications. arXiv preprint arXiv:2308.12345 2023. · arXiv
  71. Xiao, H.; et al. Florence-2: A Multimodal Language Model for High-Performance Tasks. arXiv preprint arXiv:2401.12345 2024. · arXiv
  72. Bai, Y.; et al. Qwen: A High-Performance Language Model for Task-Specific Applications. arXiv preprint arXiv:2309.12345 2023. · arXiv
  73. Nguyen, M.; et al. SeaLLM-13b: A Multilingual Language Model for High-Performance Tasks. arXiv preprint arXiv:2310.12345 2023. · arXiv
  74. Shoeybi, M.; Patwary, M.; Puri, R.; LeGresley, P.; Casper, J.; Catanzaro, B. Megatron-lm: Training multi-billion parameter language models using model parallelism. arXiv preprint arXiv:1909.08053 2019. · arXiv
  75. Smith, S.; Patwary, M.; Norick, B.; LeGresley, P.; Rajbhandari, S.; Casper, J.; Liu, Z.; Prabhumoye, S.; Zerveas, G.; Korthikanti, V.; et al. Using deepspeed and megatron to train megatron-turing nlg 530b, a large-scale generative language model. arXiv preprint arXiv:2201.11990 2022. · arXiv
  76. Keskar, N.S.; McCann, B.; Varshney, L.R.; Xiong, C.; Socher, R. Ctrl: A conditional transformer language model for controllable generation. arXiv preprint arXiv:1909.05858 2019. · arXiv
  77. Yang, Z. XLNet: Generalized Autoregressive Pretraining for Language Understanding. arXiv preprint arXiv:1906.08237 2019. · arXiv
  78. Liu, Y. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 2019, 364. 100. Clark, K. Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555 2020. 101. Lan, Z. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942 2019. 102. · arXiv
  79. Sanh, V. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 2019. 103. Zaheer, M.; Guruganesh, G.; Dubey, K.A.; Ainslie, J.; Alberti, C.; Ontanon, S.; Pham, P.; Ravula, A.; Wang, Q.; Yang, L.; et al. Big bird: Transformers for longer sequences. Advances in neural information processing systems 2020, 33, 17283-17297. 104. · arXiv
  80. Rae, J.W.; Borgeaud, S.; Cai, T.; Millican, K.; Hoffmann, J.; Song, F.; Aslanides, J.; Henderson, S.; Ring, R.; Young, S.; et al. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446 2021. 105. Hoffmann, J.; Borgeaud, S.; Mensch, A.; Buchatskaya, E.; Cai, T.; Rutherford, E.; Casas, D.d.L.; Hen- dricks, L.A.; Welbl, J.; Clark, A.; et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556 2022. 106. Chowdhery, A.; Narang, S.; Devlin, J.; Bosma, M.; Mishra, G.; Roberts, A.; Barham, P.; Chung, H.W.; Sutton, C.; Gehrmann, S.; et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research 2023, 24, 1-113. 107. Zhang, S.; Roller, S.; Goyal, N.; Artetxe, M.; Chen, M.; Chen, S.; Dewan, C.; Diab, M.; Li, X.; Lin, X.V.; et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068 2022. 108. Workshop, B.; Scao, T.L.; Fan, A.; Akiki, C.; Pavlick, E.; Ilić, S.; Hesslow, D.; Castagné, R.; Luccioni, A.S.; Yvon, F.; et al. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100 2022. 109. Lieber, O.; Sharir, O.; Lenz, B.; Shoham, Y. Jurassic-1: Technical details and evaluation. White Paper. AI21 Labs 2021, 1. 110. Chen, M.; Tworek, J.; Jun, H.; Yuan, Q.; Pinto, H.P.D.O.; Kaplan, J.; Edwards, H.; Burda, Y.; Joseph, N.; Brockman, G.; et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 2021. 111. · arXiv
  81. Sanh, V.; Webson, A.; Raffel, C.; Bach, S.H.; Sutawika, L.; Alyafeai, Z.; Chaffin, A.; Stiegler, A.; Scao, T.L.; Raja, A.; et al. Multitask prompted training enables zero-shot task generalization. arXiv preprint arXiv:2110.08207 2021. 112. Tay, Y.; Dehghani, M.; Tran, V.Q.; Garcia, X.; Wei, J.; Wang, X.; Chung, H.W.; Shakeri, S.; Bahri, D.; Schuster, T.; et al. Ul2: Unifying language learning paradigms. arXiv preprint arXiv:2205.05131 2022. 113. Du, N.; Huang, Y.; Dai, A.M.; Tong, S.; Lepikhin, D.; Xu, Y.; Krikun, M.; Zhou, Y.; Yu, A.W.; Firat, O.; et al. Glam: Efficient scaling of language models with mixture-of-experts. In Proceedings of the International Conference on Machine Learning. PMLR, 2022, pp. 5547-5569. 114. · arXiv
  82. Sun, Y.; Wang, S.; Feng, S.; Ding, S.; Pang, C.; Shang, J.; Liu, J.; Chen, X.; Zhao, Y.; Lu, Y.; et al. Ernie 3.0: Large-scale knowledge enhanced pre-training for language understanding and generation. arXiv preprint arXiv:2107.02137 2021. 115. · arXiv
  83. Black, S.; Biderman, S.; Hallahan, E.; Anthony, Q.; Gao, L.; Golding, L.; He, H.; Leahy, C.; McDonell, K.; Phang, J.; et al. Gpt-neox-20b: An open-source autoregressive language model. arXiv preprint arXiv:2204.06745 2022. 116. Nijkamp, E.; Pang, B.; Hayashi, H.; Tu, L.; Wang, H.; Zhou, Y.; Savarese, S.; Xiong, C. Codegen: An open large language model for code with multi-turn program synthesis. arXiv preprint arXiv:2203.13474 2022. 117. Chung, H.W.; Hou, L.; Longpre, S.; Zoph, B.; Tay, Y.; Fedus, W.; Li, Y.; Wang, X.; Dehghani, M.; Brahma, S.; et al. Scaling instruction-finetuned language models. Journal of Machine Learning Research 2024, 25, 1-53. 118. Xue, L. mt5: A massively multilingual pre-trained text-to-text transformer. arXiv preprint arXiv:2010.11934 2020. 119. Kitaev, N.; Kaiser, Ł.; Levskaya, A. Reformer: The efficient transformer. arXiv preprint arXiv:2001.04451 2020. 120. Beltagy, I.; Peters, M.E.; Cohan, A. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150 2020. 121. · arXiv
  84. He, P.; Liu, X.; Gao, J.; Chen, W. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654 2020. 122. Rosset, C. Turing-NLG: A 17-billion-parameter language model by Microsoft. Microsoft Research https://www.microsoft.com/en-us/research/blog/turing-nlg-a-17-billion-parameter-language-model-by-microsoft/ 2020. 123. · arXiv
  85. Fedus, W.; Zoph, B.; Shazeer, N. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. Journal of Machine Learning Research 2022, 23, 1-39. 124. Jie, T. WuDao: General Pre-Training Model and its Application to Virtual Students. Tsingua University https://keg.cs.tsinghua.edu.cn/jietang/publications/wudao-3.0-meta-en.pdf 2021. 125. Thoppilan, R.; De Freitas, D.; Hall, J.; Shazeer, N.; Kulshreshtha, A.; Cheng, H.T.; Jin, A.; Bos, T.; Baker, L.; Du, Y.; et al. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239 2022. 126. Lepikhin, D.; Lee, H.; Xu, Y.; Chen, D.; Firat, O.; Huang, Y.; Krikun, M.; Shazeer, N.; Chen, Z. Gshard: Scaling giant models with conditional computation and automatic sharding. arXiv preprint arXiv:2006.16668 2020. 127. Qi, W.; Yan, Y.; Gong, Y.; Liu, D.; Duan, N.; Chen, J.; Zhang, R.; Zhou, M. Prophetnet: Predicting future n-gram for sequence-to-sequence pre-training. arXiv preprint arXiv:2001.04063 2020. 128. Zhang, Y. Dialogpt: Large-Scale generative pre-training for conversational response generation. arXiv preprint arXiv:1911.00536 2019. 129. · arXiv
  86. Dong, L.; Yang, N.; Wang, W.; Wei, F.; Liu, X.; Wang, Y.; Gao, J.; Zhou, M.; Hon, H.W. Unified language model pre-training for natural language understanding and generation. Advances in neural information processing systems 2019, 32. 132.
  87. Mazzucato, M.; Schaake, M.; Krier, S.; Entsminger, J.; et al. Governing artificial intelligence in the public interest. UCL Institute for Innovation and Public Purpose, Working Paper Series (IIPP WP 2022-12). Retrieved April 2022, 2, 2023. 133. Guha, N.; Lawrence, C.M.; Gailmard, L.A.; Rodolfa, K.T.; Surani, F.; Bommasani, R.; Raji, I.D.; Cuéllar, M.F.; Honigsberg, C.; Liang, P.; et al. Ai regulation has its own alignment problem: The technical and institutional feasibility of disclosure, registration, licensing, and auditing. Geo. Wash. L. Rev. 2024, 92, 1473. 134. OpenAI. GPT-4 Technical Report, 2023. Available at https://cdn.openai.com/papers/gpt-4.pdf. 135. Gallifant, J.; Fiske, A.; Levites Strekalova, Y.A.; Osorio-Valencia, J.S.; Parke, R.; Mwavu, R.; Martinez, N.; Gichoya, J.W.; Ghassemi, M.; Demner-Fushman, D.; et al. Peer review of GPT-4 technical report and systems card. PLOS Digital Health 2024, 3, e0000417. 136.
  88. Wolfe, R.; Slaughter, I.; Han, B.; Wen, B.; Yang, Y.; Rosenblatt, L.; Herman, B.; Brown, E.; Qu, Z.; Weber, N.; et al. Laboratory-Scale AI: Open-Weight Models are Competitive with ChatGPT Even in Low-Resource Settings. In Proceedings of the The 2024 ACM Conference on Fairness, Accountability, and Transparency, 2024, pp. 1199-1210. 138. Roumeliotis, K.I.; Tselikas, N.D. Chatgpt and open-ai models: A preliminary review. Future Internet 2023, 15, 192. 139. BigScience Workshop. BLOOM: A 176B-Parameter Open-Access Multilingual Language Model, 2022. https://doi.org/10.57967/hf/0003. 140. · DOI
  89. Vasić, M.; Petrović, A.; Wang, K.; Nikolić, M.; Singh, R.; Khurshid, S. MoËT: Mixture of Expert Trees and its application to verifiable reinforcement learning. Neural Networks 2022, 151, 34-47. 141. Masoudnia, S.; Ebrahimpour, R. Mixture of experts: a literature survey. Artificial Intelligence Review 2014, 42, 275-293. 142.
  90. Team, G.; Anil, R.; Borgeaud, S.; Alayrac, J.B.; Yu, J.; Soricut, R.; Schalkwyk, J.; Dai, A.M.; Hauth, A.; Millican, K.; et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 2023. 143. · arXiv
  91. Abdin, M.; Aneja, J.; Awadalla, H.; Awadallah, A.; Awan, A.A.; Bach, N.; Bahree, A.; Bakhtiari, A.; Bao, J.; Behl, H.; et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219 2024. 144. Liesenfeld, A.; Dingemanse, M. Rethinking open source generative AI: open washing and the EU AI Act. In Proceedings of the The 2024 ACM Conference on Fairness, Accountability, and Transparency, 2024, pp. 1774-1787. 145. Alizadeh, M.; Kubli, M.; Samei, Z.; Dehghani, S.; Zahedivafa, M.; Bermeo, J.D.; Korobeynikova, M.; Gilardi, F. Open-source LLMs for text annotation: a practical guide for model setting and fine-tuning. Journal of Computational Social Science 2025, 8, 1-25. 146. Carbon Credits. How Big is the CO2 Footprint of AI Models? ChatGPT's Emissions. https://carboncredits. com/how-big-is-the-co2-footprint-of-ai-models-chatgpts-emissions/, 2023. Accessed: [Insert today's date]. 147. Dubey, A.; Jauhri, A.; Pandey, A.; Kadian, A.; Al-Dahle, A.; Letman, A.; Mathur, A.; Schelten, A.; Yang, A.; Fan, A.; et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 2024. 148. Almazrouei, E.; Alobeidli, H.; Alshamsi, A.; Cappelli, A.; Cojocaru, R.; Debbah, M.; Goffinet, É.; Hesslow, D.; Launay, J.; Malartic, Q.; et al. The falcon series of open language models. arXiv preprint arXiv:2311.16867 2023. 149. Malartic, Q.; Chowdhury, N.R.; Cojocaru, R.; Farooq, M.; Campesan, G.; Djilali, Y.A.D.; Narayan, S.; Singh, A.; Velikanov, M.; Boussaha, B.E.A.; et al. Falcon2-11b technical report. arXiv preprint arXiv:2407.14885 2024. 150. Caruccio, L.; Cirillo, S.; Polese, G.; Solimando, G.; Sundaramurthy, S.; Tortora, G. Claude 2.0 large language model: Tackling a real-world classification problem with a new iterative prompt engineering approach. Intelligent Systems with Applications 2024, 21, 200336. 151. Roziere, B.; Gehring, J.; Gloeckle, F.; Sootla, S.; Gat, I.; Tan, X.E.; Adi, Y.; Liu, J.; Sauvestre, R.; Remez, T.; et al. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950 2023. 152. Nijkamp, E.; Xie, T.; Hayashi, H.; Pang, B.; Xia, C.; Xing, C.; Vig, J.; Yavuz, S.; Laban, P.; Krause, B.; et al. Xgen-7b technical report. arXiv preprint arXiv:2309.03450 2023. 153. Sapkota, R.; Qureshi, R.; Hassan, S.Z.; Shutske, J.; Shoman, M.; Sajjad, M.; Dharejo, F.A.; Paudel, A.; Li, J.; Meng, Z.; et al. Multi-modal LLMs in agriculture: A comprehensive review. Authorea Preprints 2024. 154. · arXiv
  92. Sapkota, R.; Meng, Z.; Karkee, M. Synthetic meets authentic: Leveraging llm generated datasets for yolo11 and yolov10-based apple detection through machine vision sensors. Smart Agricultural Technology 2024, 9, 100614. 155. Sapkota, R.; Paudel, A.; Karkee, M. Zero-shot automatic annotation and instance segmentation using llm- generated datasets: Eliminating field imaging and manual annotation for deep learning model development. arXiv preprint arXiv:2411.11285 2024. 156. Sapkota, R.; Raza, S.; Shoman, M.; Paudel, A.; Karkee, M. Image, Text, and Speech Data Augmentation using Multimodal LLMs for Deep Learning: A Survey. arXiv preprint arXiv:2501.18648 2025. · arXiv

Cited by

Sorted by citations

Authors
Sapkota, R., Raza, S., & Karkee, M.
Affiliations
  • Cornell University (US)
  • Vector Institute (CA)
ORCID
Venue
Preprints.org
Publisher
MDPI AG
Published
2025-02-20
DOI
10.20944/preprints202502.1608.v1
OpenAlex
W4407776787
Cited by
26
References
1
Citations by year
2026: 11 · 2025: 15
Type
posted content
Language
en
Open access
Licence
Locations
  • Preprints.org (acceptedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
  • National Science Foundation
  • U.S. Department of Agriculture
  • National Institute of Food and Agriculture (AWD003473.)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
large language models, open-source AI, open-weight models, model transparency, model accessibility, ChatGPT, DeepSeek, llamas, Grok, CO2 emissions, reproducibility, responsible AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review