The DOI matches its Crossref record; 114 of the 176 references it cites were found in a registry or an index.
Regulating ChatGPT and other Large Generative AI Models
Open accessAbstract
Large generative AI models (LGAIMs), such as ChatGPT, GPT-4 or Stable Diffusion, are rapidly transforming the way we communicate, illustrate, and create. However, AI regulation, in the EU and beyond, has primarily focused on conventional AI models, not LGAIMs. This paper will situate these new generative models in the current debate on trustworthy AI regulation, and ask how the law can be tailored to their capabilities. After laying technical foundations, the legal part of the paper proceeds in four steps, covering (1) direct regulation, (2) data protection, (3) content moderation, and (4) policy proposals. It suggests a novel terminology to capture the AI value chain in LGAIM settings by differentiating between LGAIM developers, deployers, professional and non-professional users, as well as recipients of LGAIM output. We tailor regulatory duties to these different actors along the value chain and suggest strategies to ensure that LGAIMs are trustworthy and deployed for the benefit of society at large. Rules in the AI Act and other direct regulation must match the specificities of pre-trained models. The paper argues for three layers of obligations concerning LGAIMs (minimum standards for all LGAIMs; high-risk obligations for high-risk use cases; collaborations along the AI value chain). In general, regulation should focus on concrete high-risk applications, and not the pre-trained model itself, and should include (i) obligations regarding transparency and (ii) risk management. Non-discrimination provisions (iii) may, however, apply to LGAIM developers. Lastly, (iv) the core of the DSA's content moderation rules should be expanded to cover LGAIMs. This includes notice and action mechanisms, and trusted flaggers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For example, improperly secured LLMs could enable bad actors to generate fake patient data or insurance claims [62].”
From Generative AI in medical practice: in-depth exploration of privacy and security challenges (2024) · cites this paper as [62]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 2×
“Transparent LGAIMs connected to (good-old) knowledge bases, such as AtMan [43], present a promising way forward in this direction.”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 4×
“LGAIMs are usually trained with several billion, if not hundreds of billions, parameters [43, 44], requiring large amounts of training data and computing power [45].”
BackgroundInfluentialLearning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · cited 1×
“OpenAI’s “CLIP” image classifier, for example, was built using a set of 400 million image-text pairs [46].”
MethodLewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., & Zettlemoyer, L. (2019) · arXiv · doi:10.48550/arxiv.1910.13461 · Cited by 8,816 · cited 1×
“Although the emergence of these models in recent years constitutes a significant technical advance (for foundations, see [39-43]), they harness, to a reasonable extent, existing technologies in a vastly increased scale and scope.”
BackgroundLiu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208
Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR
Wachter, S., Mittelstadt, B., & Russell, C. (2017) · arXiv.org · doi:10.48550/arxiv.1711.00399 · Cited by 3,582
Training Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117 · cited 1×
“2 LGAIMs can often digest human text input [57, 58] and produce an output (text; image; audio; video) based on it.”
Background
Research works printed but not linked: 44
- Marcus, G. A Skeptical Take on the A.I. Revolution. The Ezra Klein Show, The New York Times, 2023.
- Beuth, P. Wie sich ChatGPT mit Worten hacken lässt. Der Spiegel, 2023.
- Satariano, A. and Mozur, P. The People Onscreen Are Fake. The Disinformation Is Real., 2023.
- Edwards, L. Regulating AI in Europe: four problems and four solutions (2022), 2022.
- Hacker, P., Engel, A. and List, T. Understanding and regulating ChatGPT, and other large generative AI models. 2023.
- Heikkilä, M. The EU wants to regulate your favorite AI tools. 2023.
- Chee, F. Y. and Mukherjee, S. Exclusive: ChatGPT in spotlight as EU's Breton bats for tougher AI rules. Reuters, 2023.
- Smith, B. Meeting the AI moment: advancing the future through responsible AI. 2023.
- Lieu, T. I'm a Congressman Who Codes. A.I. Freaks Me Out., 2023.
- Douek, E. Content Moderation as Systems Thinking. Harv. L. Rev., 136 (2022), 526.
- 10.1007/978-3-030-95220-4_4 · DOI
- Meyer, P. ChatGPT: How Does It Work Internally? , 2022.
- Sassoli, M. Autonomous weapons and international humanitarian law: Advantages, open technical questions and legal issues to be clarified. International Law Studies, 90, 1 (2014), 1.
- Radford, A., Narasimhan, K., Salimans, T. and Sutskever, I. Improving language understanding by generative pre-training (2018).
- Bienert, J. and Klös, H.-P. Große KI-Modelle als Basis für Forschung und wirtschaftliche Entwicklung. IW-Kurzbericht, 2022.
- OECD Measuring the Environmental Impacts of AI Compute and Applications: The AI Footprint. 2022.
- ACM, T. P. C. ACM TechBrief: Computing and Climate Change. 2021.
- 10.1016/j.oneear.2021.05.018 · DOI
- Ananthaswamy, A. The Physics Principle That Inspired Modern AI Art. 2023.
- Luccioni, A. S. and Viviano, J. D. What's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpus. arXiv preprint arXiv:2105.02732 (2021). · arXiv
- Perrigo, B. OpenAI Used Kenyan Workers on Less Than $2 Per Hour to Make ChatGPT Less Toxic. 2023.
- Bertuzzi, L. Leading MEPs exclude general-purpose AI from high-risk categories – for now. 2022.
- Bertuzzi, L. AI Act: EU Parliament's crunch time on high-risk categorisation, prohibited practices. 2023.
- Bertuzzi, L. AI Act: MEPs close in on rules for general purpose AI, foundation models. 2023.
- Bertuzzi, L. MEPs seal the deal on Artificial Intelligence Act. 2023.
- Liang, P., Bommasani, R., Creel, K. and Reich, R. The time is now to develop community norms for the release of foundation models. 2022.
- Bornstein, M., Appenzeller, G. and Casado, M. Who Owns the Generative AI Platform? , 2023.
- Micklitz, H.-W., Stuyck, J., Terryn, E. and School, I. C. Consumer law. Hart London, 2010.
- Meyers, J. M. Artificial intelligence and trade secrets. Landslide, 11 (2018), 17.
- Bertuzzi, L. Leading EU lawmakers propose obligations for General Purpose AI. 2023.
- Calvin, N. and Leung, J. Who owns artificial intelligence? A preliminary analysis of corporate intellectual property strategies and why they matter. Future of Humanity Institute, February (2020).
- Deeks, A. The judicial demand for explainable artificial intelligence. Columbia Law Review, 119, 7 (2019), 1829-1850.
- McKown, J. R. Discovery of Trade Secrets. Santa Clara Computer & High Tech. LJ, 10 (1994), 35.
- Subrin, S. N. Discovery in Global Perspective: Are We Nuts. DePaul L. Rev., 52 (2002), 299.
- Daniel, P. F. Protecting Trade Secrets from Discovery. Tort & Ins. LJ, 30 (1994), 1033.
- Wachter, S. Affinity profiling and discrimination by association in online behavioral advertising. Berkeley Tech. LJ, 35 (2020), 367.
- Kiela, D., Firooz, H., Mohan, A., Goswami, V., Singh, A., Fitzpatrick, C. A., Bull, P., Lipstein, G., Nelli, T. and Zhu, R. The hateful memes challenge: Competition report. PMLR, 2021.
- Seeha, S. Prompt Engineering and Zero-Shot/Few-Shot Learning [Guide]. 2022.
- European, C., Directorate-General for Communications Networks, C., Technology, Hoboken, J., Quintais, J., Poort, J. and Eijk, N. Hosting intermediary services and illegal content online : an analysis of the scope of article 14 ECD in light of developments in the online service landscape : final report. Publications Office, 2019.
- 10.1017/CBO9780511676161 · DOI
- Gerdemann, S. and Spindler, G. Das Gesetz über digitale Dienste (Digital Services Act) (Part 2). Gewerblicher Rechtsschutz und Urheberrecht (2023), 115-125.
- Bertuzzi, L. AI Act: MEPs want fundamental rights assessments, obligations for high-risk users. 2023.
- Crootof, R. Artificial intelligence research needs responsible publication norms. Lawfare Blog (2019).
- Hoffmann-Riem, W. Innovation und Recht-Recht und Innovation: Recht im Ensemble seiner Kontexte. Mohr Siebeck, 2016.
Cited by
Sorted by citations
The imperative for regulatory oversight of large language models (or generative AI) in healthcare
Meskó, B., & Topol, E. J. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00873-0 · Cited by 1,047
“In details, regulation should focus on concrete high-risk applications, and not the pre-trained model itself, and should include (i) obligations regarding transparency, (ii) risk management, (iii) non-discrimination provisions, and (iv) content moderation rules 15 .”
Summary of ChatGPT-Related research and perspective towards the future of large language models
Liu, Y., Han, T., Ma, S., Zhang, J., Yang, Y., Tian, J., He, H., Li, A., He, M., Liu, Z., et al. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100017 · Cited by 855
“Regarding relevant policies and regulations, Hacker et al. [25] discussed the nature and rules of large generative AI models, including ChatGPT, which are rapidly changing the way we communicate, explain, and create.”
BackgroundLarge language models in healthcare and medical domain: A reviewPreprint
Nazi, Z. A., & Peng, W. (2023) · Informatics · doi:10.48550/arxiv.2401.06775 · Cited by 551
“However, they also promote reliability, legal accountability, and ethical standards in AI development, fostering trust among users and stakeholders and encouraging the responsible and wider adoption of AI technologies in healthcare [135].”
BackgroundUnlocking the opportunities through ChatGPT Tool towards ameliorating the education system
Javaid, M., Haleem, A., Singh, R. P., Khan, S., & Khan, I. H. (2023) · BenchCouncil Transactions on Benchmarks, Standards and Evaluations · doi:10.1016/j.tbench.2023.100115 · Cited by 535
A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4Preprint
Kalyan, K. S. (2023) · Natural Language Processing Journal · doi:10.2139/ssrn.4593895 · Cited by 438
Generative artificial intelligence
Banh, L., & Strobel, G. (2023) · Electronic Markets · doi:10.1007/s12525-023-00680-1 · Cited by 879
“…the European Union AI Act or US National AI Initiative) serve as promising initial steps toward opening pathways for future research to effectively address these issues, ensuring that AI-generated content is morally, ethically, and legally appropriate and cannot be mis-used (Hacker et al., 2023).”
BackgroundJackson, I., Ivanov, D., Dolgui, A. B., & Namdar, J. (2024) · International Journal of Production Research · doi:10.1080/00207543.2024.2309309 · Cited by 567
“These generated datasets can reflect real-world complexities while preserving the privacy and anonymity necessary in many use cases (Hacker, Engel, and Mauer 2023).”
Generative AI in medical practice: in-depth exploration of privacy and security challenges
Chen, Y., & Esmaeilzadeh, P. (2024) · Journal of Medical Internet Research · doi:10.2196/53008 · Cited by 353
“For example, improperly secured LLMs could enable bad actors to generate fake patient data or insurance claims [62].”
A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4
Kalyan, K. S. (2024) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100048 · Cited by 335
Generative AI Literacy: Twelve Defining Competencies
Annapureddy, R., Fornaroli, A., & Gatica-Perez, D. (2025) · Digital Government: Research and Practice · doi:10.1145/3685680 · Cited by 333
“…article, only a few countries have begun a legislative process regulating AI and its applications, it is likely that several others will follow [37, 65] Parliament approved the European AI Act [32], determining a set of norms to be followed with respect to several aspects of AI at large [41,…”
Ethical challenges and solutions of generative AI: an interdisciplinary perspective
Al-kfairy, M., Mustafa, D., Kshetri, N., Insiew, M., & Alfandi, O. (2024) · Informatics · doi:10.3390/informatics11030058 · Cited by 328
“[37] Generic Conf.”
BackgroundInfluentialThe potential of generative AI for personalized persuasion at scale
Matz, S. C., Teeny, J. D., Vaid, S. S., Peters, H., Harari, G. M., & Cerf, M. (2024) · Scientific Reports · doi:10.1038/s41598-024-53755-0 · Cited by 315
“For example, academic proposals to regulate generative AI have collectively argued that regulation should occur at the stage of deployment (e.g., at the advertiser and platform level), rather than at the level of message reception (e.g., at the time point consumers are seeing the specific ad) 79 .”
BackgroundThe Potential of Generative AI for Personalized Persuasion at Scale
Matz, S., Teeny, J., Vaid, S. S., Peters, H., Harari, G. M., & Cerf, M. (2023) · doi:10.31234/osf.io/rn97c · Cited by 312
Can LLM-Generated Misinformation Be Detected?Preprint
Chen, C., & Shu, K. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.13788 · Cited by 235
“, 2021), we call for more research on the regulation aspect of LLMs (Anderljung et al., 2023; Ho et al., 2023; Trager et al., 2023; Hacker et al., 2023; Barrett et al., 2022) and countermeasures against LLM-”
BackgroundAuditing large language models: a three-layered approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2024) · AI and Ethics · doi:10.1007/s43681-023-00289-2 · Cited by 289
“Therefore, the question of how LLMs should be governed has attracted much attention, with proposals ranging from structured access protocols designed to prevent malicious use [76] to hard regulation prohibiting the deployment of LLMs for specific purposes [86].”
BackgroundSummary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models
Liu, Y., Han, T., Ma, S., Zhang, J., Yang, Y., Tian, J., He, H., Li, A., He, M., Liu, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.01852 · Cited by 1,172
“Regarding relevant policies and regulations, Hacker et al. [25] discussed the nature and rules of large generative AI models, including ChatGPT, which are rapidly changing the way we communicate, explain, and create.”
BackgroundA Survey of Large Language Models in Medicine: Progress, Application, and Challenge
Zhou, H., Liu, F., Gu, B., Zou, X., Huang, J., Wu, J., Li, Y., Chen, S. S., Zhou, P., Liu, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.05112 · Cited by 240
Transforming Cardiovascular Care With Artificial Intelligence: From Discovery to Practice
Khera, R., Oikonomou, E. K., Nadkarni, G. N., Morley, J., Wiens, J., Butte, A. J., & Topol, E. J. (2024) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2024.05.003 · Cited by 324
A Systematic Survey of Prompt Engineering on Vision-Language Foundation ModelsPreprint
Gu, J., Han, Z. Y., Chen, S., Beirami, A., He, B., Zhang, G., Liao, R., Qin, Y., Volker Tresp, C., & Torr, P. H. S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.12980 · Cited by 266
“Last but not least, managing the accompanying risks of prompt engineering and large models requires the close collaboration of society, research institutions, and government [214, 215].”
BackgroundExploring the Implications of ChatGPT for Language Learning in Higher Education
Baskara, R., & Mukarto, M. (2023) · IJELTAL (Indonesian Journal of English Language Teaching and Applied Linguistics) · doi:10.21093/ijeltal.v7i2.1387 · Cited by 275
“This issue arises due to the model's training data, which can contain biased or discriminatory language (Hacker et al., 2023).”
Combating Misinformation in the Age of LLMs: Opportunities and ChallengesPreprint
Chen, C., & Shu, K. (2023) · The AI Magazine · doi:10.48550/arxiv.2311.05656 · Cited by 245
Ning, Y., Teixayavong, S., Shang, Y., Savulescu, J., Nagaraj, V., Miao, D., Mertens, M., Ting, D. S. W., Ong, J. C. L., Liu, M., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00143-2 · Cited by 212
“12 Similarly, general users of GenAI are unlikely to understand the technology well enough to prescribe a reasonable amount of trust on its output or be able to identify potential misinformation.”
BackgroundThe Advent of Generative Language Models in Medical Education
Karabacak, M., Ozkara, B. B., Margetis, K., Wintermark, M., & Bisdas, S. (2023) · JMIR Medical Education · doi:10.2196/48163 · Cited by 208
“Issues related to data privacy, transparency, and intellectual property must be addressed to ensure that these tools are used responsibly [18].”
Schneider, J., Meske, C., & Kuss, P. (2024) · Business & Information Systems Engineering · doi:10.1007/s12599-024-00851-0 · Cited by 145
“In the context of foundation models, prompting is highly relevant because the ability of the respective system to generate diverse and possibly objectionable output demands the possibility to specify and constrain desirable generations (Hacker et al. 2023).”
Marketing with ChatGPT: Navigating the Ethical Terrain of GPT-Based Chatbot Technology
Rivas, P., & Zhao, L. (2023) · AI · doi:10.3390/ai4020019 · Cited by 191
“Furthermore, training ChatGPT has caused severe mental stress on its content curators [21,22].”
Wiratunga, N., Abeyratne, R., Jayawardena, L., Martin, K., Massie, S., Nkisi-Orji, I., Weerasinghe, R., Liret, A., & Fleisch, B. (2024) · International Conference on Case-Based Reasoning · doi:10.48550/arxiv.2404.04302 · Cited by 170
“However, such a system would require output that could be validated, and moderation of content generated by LLMs has been highlighted as a critical concern [11].”
BackgroundPromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts
Zhu, K., Wang, J., Zhou, J., Wang, Z., Chen, H., Wang, Y., Yang, L., Ye, W., Zhang, Y., Gong, N. Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.04528 · Cited by 153
“Recent studies evaluated LLMs from various aspects including natural language processing [2, 41, 26], ethics [17, 45], robustness [56, 69], and education [21, 16].”
BackgroundThe paradoxes of generative AI-enabled customer service: A guide for managers
Ferraro, C., Demsar, V., Sands, S., Restrepo, M., & Campbell, C. (2024) · Business Horizons · doi:10.1016/j.bushor.2024.04.013 · Cited by 143
ChatGPT Performs on the Chinese National Medical Licensing Examination
Wang, X., Gong, Z., Wang, G., Jia, J., Xu, Y., Zhao, J., Fan, Q., Wu, S., Hu, W., & Li, X. (2023) · Journal of Medical Systems · doi:10.1007/s10916-023-01961-0 · Cited by 166
“Similarly, in the 2021 NLME exam, ChatGPT correctly answered 275 out of 600 items (45.8%).”
MethodInfluentialHacker, P. (2022) · Computer Law and Security Review · doi:10.48550/arxiv.2211.13960 · Cited by 141
Self-citationA Design Space for Intelligent and Interactive Writing Assistants
Lee, M., Gero, K. I., Chung, J. J. Y., Shum, S. B., Raheja, V., Shen, H., Venugopalan, S., Wambsganss, T., Zhou, D., Alghamdi, E. A., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642697 · Cited by 136
Jailbreaking LLM-Controlled Robots
Robey, A., Ravichandran, Z., Kumar, V., Hassani, H., & Pappas, G. J. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11128119 · Cited by 38
“One prominent aspect of AI safety is the process of model alignment , which is an umbrella term referring to algorithms which ensure that LLM outputs are consistent with human values [27]–[29].”
Generative AI in EU Law: Liability, Privacy, Intellectual Property, and Cybersecurity
(2024) · doi:10.2139/ssrn.4694565 · Cited by 124
“These articles pose a unique challenge for Generative AI due to the nature and scope of the data they process [2].”
InfluentialSelf-citationTypology of Risks of Generative Text-to-Image Models
Bird, C., Ungless, E., & Kasirzadeh, A. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604722 · Cited by 121
“technologies, including data privacy, intellectual property infringement, and security vulnerabilities [70, 138, 161].”
BackgroundLarge Language Model‐Based Chatbots in Higher Education
Yigci, D., Eryilmaz, M., Yetisen, A. K., Tasoglu, S., & Ozcan, A. (2025) · Advanced Intelligent Systems · doi:10.1002/aisy.202400429 · Cited by 113
Andersen, J. P., Degn, L., Fishberg, R., Graversen, E., Horbach, S., Schmidt, E. K., Schneider, J. W., & Sørensen, M. P. (2025) · Technology and Society · doi:10.1016/j.techsoc.2025.102813 · Cited by 113
A Survey on Detection of LLMs-Generated ContentPreprint
Yang, X., Pan, L., Zhao, X., Chen, H.-F., Petzold, L., Wang, W. Y., & Cheng, W. (2023) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2310.15654 · Cited by 108
“Specifically, this survey concentrates on the detection of text and code generated by LLMs, as well as the attacks.”
Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Survey
Huang, T., Hu, S., İlhan, F., Tekin, S. F., & Liu, L. (2026) · ACM Computing Surveys · doi:10.1145/3817114 · Cited by 129
Large language models can infer psychological dispositions of social media users
Peters, H., & Matz, S. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae231 · Cited by 131
Walter, Y. (2024) · Discover Artificial Intelligence · doi:10.1007/s44163-024-00109-4 · Cited by 108
Beyond the Cloud: Edge Inference for Generative Large Language Models in Wireless Networks
Zhang, X., Nie, J., Huang, Y., Xie, G., Xiong, Z., Liu, J., Niyato, D. T., & Shen, X. (2024) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2024.3497923 · Cited by 86
“A significant milestone in this evolution is Generative AI (GAI), which exceeds mere analytics to enable machine creativity [2], [3], [4], [5].”
Do large language models have a legal duty to tell the truth?
Wachter, S., Mittelstadt, B., & Russell, C. (2024) · Royal Society Open Science · doi:10.1098/rsos.240197 · Cited by 104
Wang, P. Q. (2024) · Current Issues in Tourism · doi:10.1080/13683500.2023.2300030 · Cited by 130
“…in creative applications such as those associated with art and music, but it can also be used for more practical applications by learning patterns and structures in a large dataset during training, and then generating new data that follows the same patterns and structures (Hacker et al., 2023).”
Nie, T., Sun, J., & Ma, W. (2025) · Artificial Intelligence for Transportation · doi:10.1016/j.ait.2025.100003 · Cited by 70
The Security of Using Large Language Models: A Survey with Emphasis on ChatGPT
Zhou, W., Zhu, X., Han, Q., Li, L., Chen, X., Wen, S., & Xiang, Y. (2024) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2024.124983 · Cited by 107
Beyond the Safeguards: Exploring the Security Risks of ChatGPTPreprint
Derner, E., & Batistič, K. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.08005 · Cited by 135
“Regulation of large generative models is also proposed to ensure trans-parency, risk management, non-discrimination, and content moderation obligations [19].”
BackgroundXiao, P., Chen, Y., & Bao, W. (2023) · doi:10.2139/ssrn.4458269 · Cited by 93
“Hacker, Engel, and Mauer (2023) situate the emerging generative models in the current debate on AI regulation and suggest that more differentiated regulation would better fit the realities of the evolving AI value chain. de Rassenfosse, Jaffe, and Wasserman (2023) propose that current policy…”
BackgroundTriguero, I., Molina, D., Poyatos, J., Del Ser, J., & Herrera, F. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102135 · Cited by 90
Large Language Models for Text Classification: From Zero-Shot Learning to Instruction-Tuning
Chae, Y., & Davidson, T. (2025) · Sociological Methods & Research · doi:10.1177/00491241251325243 · Cited by 72
Exploring ChatGPT Capabilities and Limitations: A Survey
Koubâa, A., Boulila, W., Ghouti, L., Alzahem, A., & Latif, S. (2023) · IEEE Access · doi:10.1109/access.2023.3326474 · Cited by 188
“Hacker et al. [60] presented a brief discussion on various issues regulating large generative AI models, such as ChatGPT.”
BackgroundOn the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective
Wang, J., Hu, X., Hou, W., Chen, H., Zheng, R., Wang, Y., Yang, L., Huang, H., Ye, W., Geng, X., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.12095 · Cited by 86
“[18, 41] expressed concerns that ChatGPT and other large models should be regulated since they are double-edged swords.”
BackgroundAI for crop production – Where can large language models (LLMs) provide substantial value?
Kuśka, M. T., Wahabzada, M., & Paulus, S. (2024) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2024.108924 · Cited by 95
Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Models
Henderson, P., Mitchell, E., Manning, C. D., Jurafsky, D., & Finn, C. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604690 · Cited by 42
“Others argue that such a move would stifle innovation and make it more difficult to develop safer overall models [6, 17, 25].”
BackgroundTransparent AI Disclosure Obligations: Who, What, When, Where, Why, How
Ali, A. E., Venkatraj, K. P., Morosoli, S., Naudts, L., Helberger, N., & Cesar, P. S. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650750 · Cited by 72
“Themes and sub-themes, with examples selected mainly based on participant votes, are presented below:”
BackgroundFriend or foe? Exploring the implications of large language models on the science system
Fecher, B., Hebing, M., Laufer, M., Pohle, J., & Sofsky, F. (2025) · AI & SOCIETY · doi:10.1007/s00146-023-01791-1 · Cited by 78
“Furthermore, Hacker et al. (2023) call for specific regulation of LLM-based tools, “large generative AI models”, under the EU Digital Services Act and provide four concrete, workable suggestions that include transparency obligations, mandatory yet limited risk management, non-discrimination data…”
Navigating the Power of Artificial Intelligence in Risk Management: A Comparative Analysis
Yazdi, M., Zarei, E., Adumene, S., & Beheshti, A. (2024) · Safety · doi:10.3390/safety10020042 · Cited by 117
“Hacker, Engel, and Mauer (2023) [22] discuss the necessity for a comprehensive risk management framework for large generative AI models like ChatGPT, emphasising the need to address potential high-risk applications and innate vulnerabilities of these technologies.”
BackgroundShen, H., Knearem, T., Ghosh, R., Alkiek, K., Krishna, K., Liu, Y.-C., Ma, Z., Petridis, S., Peng, Y.-H., Qiwei, L., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2406.09264 · Cited by 75
“[104, 105, 130].”
A Robust Governance for the AI Act: AI Office, AI Board, Scientific Panel, and National Authorities
Novelli, C., Hacker, P., Morley, J., Trondal, J., & Floridi, L. (2024) · European Journal of Risk Regulation · doi:10.1017/err.2024.57 · Cited by 74
Self-citationChatGPT as an important tool in organizational management: A review of the literature
Ayinde, L. A., Wibowo, M. P., Ravuri, B., & Emdad, F. B. (2023) · Business Information Review · doi:10.1177/02663821231187991 · Cited by 95
Performance of ChatGPT and Google Translate for Pediatric Discharge Instruction Translation
Brewster, R., Gonzalez, P., Khazanchi, R., Butler, A., Selcer, R., Chu, D., Aires, B. P., Luercio, M., & Hron, J. D. (2024) · PEDIATRICS · doi:10.1542/peds.2023-065573 · Cited by 77
An automated framework for assessing how well LLMs cite relevant medical references
Wu, K., Wu, E., Wei, K., Zhang, A., Casasola, A., Nguyen, T., Riantawan, S., Shi, P., Ho, D., & Zou, J. (2025) · Nature Communications · doi:10.1038/s41467-025-58551-6 · Cited by 72
“The need to cite the sources for medical statements goes beyond gaining clinician and patient trust -- there is also an urgent regulatory case as well 19 .”
Why the European AI Act transparency obligation is insufficient
Li, Z. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00672-y · Cited by 71
“Therefore, it is necessary to urgently reconsider the regulation of general-purpose LLMs. 9 The parroting and hallucination issues show that minimal transparency obligations are insufficient, since LLMs often lull users into misplaced trust.”
Hou, C., Zhu, G., Sudarshan, V., Lim, F. S., & Ong, Y. S. (2025) · Computers & Education · doi:10.1016/j.compedu.2025.105329 · Cited by 70
General-purpose AI regulation and the European Union AI Act
Gstrein, O. J., Haleem, N., & Zwitter, A. (2024) · Internet Policy Review · doi:10.14763/2024.3.1790 · Cited by 101
“…Borge-sius, 2021; Gstrein, 2022, pp. 756–760), the unprecedented popularity of general-purpose AI sparked a debate on whether it requires specific laws (Hacker et al., 2023; Helberger & Diakopoulos, 2023), or even legally binding international treaties (Harris, 2023; Milmo & Stacey, 2023).”
Mess, S. A., Mackey, A., & Yarowsky, D. (2025) · Plastic & Reconstructive Surgery Global Open · doi:10.1097/gox.0000000000006450 · Cited by 84
Copiloting the future: How generative AI transforms Software Engineering
Banh, L., Holldack, F., & Strobel, G. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107751 · Cited by 78
Symeou, L., Louca, L. Τ., Kavadella, A., Mackay, J., Danidou, Y., & Raffay, V. I. (2025) · European Journal Of Dental Education · doi:10.1111/eje.13069 · Cited by 87
“These principles are aligned with concerns raised in the literature regarding the risks associated with data privacy and ethical data handling in AI applications [66, 67].”
Generative AI in the Wild: Prospects, Challenges, and Strategies
Sun, Y., Jang, E., Ma, F., & Wang, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642160 · Cited by 64
“Our study revealed the intricate ways in which practitioners individually confronted and navigated complex ethical challenges, especially when GenAI was operated in a ‘grey area’ lacking clear policies or regulations [1, 34].”
BackgroundCreative Use of OpenAI in Education: Case Studies from Game Development
French, F., Levi, D., Maczo, C., Simonaityte, A., Triantafyllidis, S., & Varda, G. (2023) · Multimodal Technologies and Interaction · doi:10.3390/mti7080081 · Cited by 64
Generative Ai in EU Law: Liability, Privacy, Intellectual Property, and Cybersecurity
Novelli, C., Casolari, F., Hacker, P., Spedicato, G., & Floridi, L. (2024) · doi:10.2139/ssrn.4821952 · Cited by 57
Self-citationBeyond Discrimination: Generative AI Applications and Ethical Challenges in Forensic Psychiatry
Tortora, L. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1346059 · Cited by 78
“For this reason, throughout this paper, the broader term ‘ large generative AI models (LGAIMs) ’ will be adopted to encompass all the different types of generative AI models, of which large language models (LLMs) are only a subset (10).”
BackgroundGöktaş, P. (2024) · Journal of Decision System · doi:10.1080/12460125.2024.2410042 · Cited by 69
Towards a Definition of Generative Artificial Intelligence
Ronge, R., Maier, M., & Rathgeber, B. (2025) · Philosophy & Technology · doi:10.1007/s13347-025-00863-y · Cited by 89
“In a semi-technical setting – e.g., technically informed researchers writing for a broader or interdisciplinary audience – GenAI is usually understood to somehow refer to generative models (Hacker et al., 2023), but this relationship is often not specified further.”
Huang, R., Adarkwah, M. A., Liu, M., Hu, Y., Zhuang, R., & Chang, T. (2024) · Frontiers of digital education. · doi:10.1007/s44366-024-0031-x · Cited by 109
“Utilising GenAI tools comes with risks and errors (Hacker et al., 2023).”
Liu, G., Lin, W., Mu, Q., Huang, T., Mo, R., Tao, Y., & Shen, L. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3615412 · Cited by 5
“To address harmful fine-tuning attacks, recent mainstream defenses can be divided into three categories: alignment-stage defense [17], [18], [19], [20], fine-tuning-stage defense [21], [22], and post-fine-tuning stage defense [23], [24], [25].”
A Security Risk Taxonomy for Prompt-Based Interaction With Large Language Models
Derner, E., Batistič, K., Zahálka, J., & Babuška, R. (2024) · IEEE Access · doi:10.1109/access.2024.3450388 · Cited by 74
“Regulation of large generative models is also proposed to ensure transparency, risk management, and non-discrimination [21].”
BackgroundCan generative AI motivate management students? The role of perceived value and information literacy
Jose, E. M. K., Prasanna, A., Kushwaha, B. P., & Das, M. (2024) · The International Journal of Management Education · doi:10.1016/j.ijme.2024.101082 · Cited by 55
AI and Computing Horizons: Cloud and Edge in the Modern Era
Prangon, N. F., & Wu, J. (2024) · Journal of Sensor and Actuator Networks · doi:10.3390/jsan13040044 · Cited by 65
“The use cases of these computing paradigms will be tempered by stiffer data privacy laws across these computing paradigms, as emphasized in [63,64].”
MethodDeng, C., Duan, Y., Jin, X., Chang, H., Tian, Y., Liu, H., Wang, Y., Gao, K., Zou, H. P., Jin, Y., et al. (2025) · AI and Ethics · doi:10.1007/s43681-025-00797-3 · Cited by 73
“…discussions are also proposed around its practical compliance with LLMs. Hacker et al. argue that this direct regulation is unsatisfactory and could be further enhanced from 1) the definition of GPAIS, 2) the risk management of GPAIS, and 3) the adverse consequences for competi - tion [101].”
Adversarial Attacks on Robotic Vision Language Action ModelsPreprint
Jones, E. K., Robey, A., Zou, A., Ravichandran, Z., Pappas, G. J., Hassani, H., Fredrikson, M., & Kolter, J. Z. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.03350 · Cited by 49
Explaining AI through mechanistic interpretability
Kästner, L., & Crook, B. (2024) · European Journal for Philosophy of Science · doi:10.1007/s13194-024-00614-4 · Cited by 59
“It is even listed as the first objective in the EU AI Act (Artificial Intelligence Act, 2021, sec.”
InfluentialXu, R., & Wang, Z. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e32364 · Cited by 55
“ChatGPT has led the revolutionary development of generative artificial intelligence/large language models (hereafter, GenAI LLMs) [1 – 3] and has gradually been applied to important fields such as healthcare [4,5].”
BackgroundTrustworthiness Assurance Assessment for High-Risk AI-Based Systems
Stettinger, G., Weissensteiner, P., & Khastgir, S. (2024) · IEEE Access · doi:10.1109/access.2024.3364387 · Cited by 71
“Participants in the sandbox were generally cautiously optimistic about the need for AI regulation [23].”
Cooking With Agents: Designing Context-aware Voice Interaction
Jaber, R. N., Zhong, S., Kuoppamäki, S., Hosseini, A., Gessinger, I., Brumby, D., Cowan, B. R., & Mcmillan, D. (2024) · International Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642183 · Cited by 48
“…integrating LLMs in conversational interaction include combinations of rule-based and heuristic methods for determining the correct next utterance [33], where the results of this paper can be directly applied to guide the VA in both informing the user of the current state and in how they can…”
On opportunities and challenges of large multimodal foundation models in education
Küchemann, S., Avila, K. E., Dinc, Y., Hortmann, C., Revenga, N., Ruf, V., Stausberg, N., Steinert, S., Fischer, F., Fischer, M., et al. (2025) · npj Science of Learning · doi:10.1038/s41539-025-00301-w · Cited by 47
“Forexample,undertheEUAIAct,theuseofgenerative AIineducationwouldbesubjecttorigorousethicalandregulatory standards,focusingonriskassessment,transparency,dataprotection, andfairness 105 .”
Actions Speak Louder than Words: Agent Decisions Reveal Implicit Biases in Language Models
Li, Y., Shirado, H., & Das, S. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732212 · Cited by 25
“The advent of LLMs has led to further concerns about their potential to encode harmful biases due to their reliance on large-scale, uncu-rated datasets [8, 34].”
The Foundation Model Transparency Index
Bommasani, R., Klyman, K., Longpre, S., Kapoor, S., Maslej, N., Xiong, B., Zhang, D., & Liang, P. (2023) · arXiv · doi:10.48550/arxiv.2310.12941 · Cited by 46
Understanding the Impact of AI-Generated Content on Social Media: The Pixiv Case
Wei, Y., & Gareth Tyson, A. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680631 · Cited by 50
Balancing innovation and regulation in the age of generative artificial intelligence
Wang, X., & Wu, Y. C. (2024) · Journal of Information Policy · doi:10.5325/jinfopoli.14.2024.0012 · Cited by 42
Monzon, N., & Hays, F. A. (2025) · JMIR Medical Education · doi:10.2196/59210 · Cited by 61
How well do LLMs cite relevant medical references? An evaluation framework and analyses
Wu, K., Wu, E., Cassasola, A., Zhang, A., Wei, K., Nguyen, T., Riantawan, S., Riantawan, P. S., Ho, D. E., & Zou, J. (2024) · arXiv · doi:10.48550/arxiv.2402.02008 · Cited by 41
“The need to cite the sources for medical statements goes beyond gaining clinician and patient trust – there is also an urgent regulatory case as well (Hacker et al., 2023).”
BackgroundWatME: Towards Lossless Watermarking Through Lexical Redundancy
Chen, L., Bian, Y., Deng, Y., Cai, D., Li, S., Zhao, P., & Wong, K. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.496 · Cited by 25
“Regulatory bodies can later extract this information to trace and regulate AI-generated content (Atal-lah et al., 2001; Wilson et al., 2014; Hacker et al., 2023).”
BackgroundHacker, P. (2023) · Social Science Research Network · doi:10.48550/arxiv.2306.00292 · Cited by 42
Self-citationManaging the risks of artificial general intelligence: A human factors and ergonomics perspective
Salmon, P. M., Baber, C., Burns, C. M., Carden, T., Cooke, N. J., Cummings, M. L., Hancock, P. A., McLean, S. G., Read, G. J. M., & Stanton, N. A. (2023) · Human Factors and Ergonomics in Manufacturing & Service Industries · doi:10.1002/hfm.20996 · Cited by 52
“Recent evolutions in generative AI tools (Hacker et al., 2023) have led to their increased sophistication, utility, and accessibility”
BackgroundTo ChatGPT, or not to ChatGPT: That is the question!
Pegoraro, A., Kumari, K., Fereidooni, H., & Sadeghi, A.-R. (2023) · arXiv · doi:10.48550/arxiv.2304.01487 · Cited by 39
“Malicious actors use it to propagate fake news on social media platforms [19], [9], and educational institutions are employing it to provide mental health education to students without their consent [2], among other uses.”
BackgroundWe Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent SystemsPreprint
Fang, J., Yao, Z., Wang, R., Ma, H., Xiang, W., & Chua, T. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.13666 · Cited by 31
“To enhance safety during fine-tuning, numerous defense mechanisms employ model-level regularization [64, 56] and detection architectures [24, 38, 39, 11], complemented by data manipulation techniques to mitigate harmful data influences [5, 107, 19, 77, 51].”
The Role of Explainability in Collaborative Human-AI Disinformation Detection
Schmitt, V., Villa-Arenas, L. F., Feldhus, N., Meyer, J., Spang, R. P., & Möller, S. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659031 · Cited by 35
SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks
Robey, A., Wong, E., Hassani, H., & Pappas, G. J. (2023) · arXiv · doi:10.48550/arxiv.2310.03684 · Cited by 37
“Indeed, the propensity of LLMs to reproduce this objectionable content has invigorated the field of AI alignment [2–4], wherein various mechanisms are used to “align” the output text generated by LLMs with ethical and legal standards [5–7].”
Assessing trustworthy AI: Technical and legal perspectives of fairness in AI
Kattnig, M., Angerschmid, A., Reichel, T., & Kern, R. (2024) · Computer law & security review · doi:10.1016/j.clsr.2024.106053 · Cited by 63
transformative potential of Generative Artificial Intelligence (GenAI) in business
Caño-Marín, E. (2024) · ESIC MARKET Economic and Business Journal · doi:10.7200/esicm.55.333 · Cited by 53
Metadata
- Authors
- Hacker, P., Engel, A., & Mauer, M.
- Affiliations
- European University Viadrina (DE)
- Heidelberg University (DE)
- Venue
- 2023 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2023-06-12
- Pages
- 1112-1123
- OpenAlex
- W4380319827
- Cited by
- 466
- References
- 176
- Citations by year
- 2026: 59 · 2025: 151 · 2024: 191 · 2023: 65
- FWCI
- 1289.86
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by-nc-sa)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Funders
- Universität Hamburg
- Topics
- Social Sciences › Law › Digitalization, Law, and Regulation
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Law, AI, and Intellectual Property
- Keywords
- ChatGPT, AI regulation, trustworthy AI, EU AI Act, AI value chain, high-risk AI systems, data protection, content moderation, Digital Services Act, transparency obligations, risk management, non-discrimination, pre-trained models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
