Regulating ChatGPT and other Large Generative AI Models

“Transparent LGAIMs connected to (good-old) knowledge bases, such as AtMan [43], present a promising way forward in this direction.”

“LGAIMs are usually trained with several billion, if not hundreds of billions, parameters [43, 44], requiring large amounts of training data and computing power [45].”

“OpenAI’s “CLIP” image classifier, for example, was built using a set of 400 million image-text pairs [46].”

“Although the emergence of these models in recent years constitutes a significant technical advance (for foundations, see [39-43]), they harness, to a reasonable extent, existing technologies in a vastly increased scale and scope.”

“2 LGAIMs can often digest human text input [57, 58] and produce an output (text; image; audio; video) based on it.”

“…at a crucial point in AI regulation when regulators are looking to researchers for experimental insights to inform and guide their policymaking [41]; by making results interpretable to non-technical stakeholders, researchers can better inform evidence-based regulation and contribute to…”

“Consequently, our method can evade a range of existing safety defenses, such as finetuning data filtering and inference-time monitoring (Hacker et al., 2023), thereby enabling an invisible form of safety threat.”

“Moreover, the rise of generative AI intensifies these challenges, necessitating real-time, user-side interventions [2, 20, 42, 45, 62].”

“In fact, some scholars suggest to extend and adapt DSA rules for social media platforms to generative AI (Hacker et al., 2023).”

“Existing regulations and guidelines often struggle to keep pace with the evolving capabilities of LLMs [11], leaving room for ambiguity and potential misuse.”

“Both security and safety measures are needed to ensure the good and constructive use of generative AI [16].”

“Some proposals have come from researchers (Hacker et al. 2023; Bengio et al. 2024).”

“Researchers discussed safety issues of LLMs worries such as false information, social bias, privacy infringement[9], and hallucinations[6,10,18].”

“…in homes with adults under extreme strains suffered violence during the pandemic; the elderly have been swayed more easily by scams and digital propaganda; ChatGPT and similar AI models have not been regulated suf fi ciently; Hacker et al., 2023; Lillywhite and Wolbring, 2023; Waymer, 2020).”

“Instead, GAI focuses on creating new content by using the very data it was trained upon [16, 28].”

“This layered architecture reflects the recognition that model providers may see their products integrated into a range of downstream systems and must therefore enable, but cannot substitute for, system-level compliance (Hacker et al., 2023).”

“Fortunately, a de-fender can detect the harmful data provided by users by moderation models to mitigate these attacks (Hacker et al., 2023; Ji et al., 2023; Kolla et al., 2024; Kumar et al., 2024).”

“Regarding monitoring and reporting obligations, the EU’s AI Act encompasses provisions for regulatory oversight that address the transparency of AI models, risk management, non-discrimination clauses for large-scale generative AI models, and guidelines for content review [ 106 ].”

“Hacker et al. (2023) argue that regulation should prioritize transparency and risk management and broaden content moderation guidelines to include governance in Gen AI models.”

“The bot then goes through three meticulous checks that serve as guardrails to ensure the safety and integrity of the responses [27].”

“In [23], the authors tackle the legal obstacles presented by the presence of large generative AI models (LGAIMs) such as ChatGPT [24] and offer suggestions on how the legal framework can be adjusted to account for their capabilities.”

“The term “foundation model” may also be colloquially used interchangeably with adjacent terms, such as “frontier,” “general-purpose,” “generative AI,” or “large generative AI models” [28].”

“Existing AI regulatory frameworks, such as the European AI Act, are deemed inadequate for large generative AI models like ChatGPT [ 15 ], prompting calls for new laws focusing on data protection, consumer protection, cybersecurity, and intellectual property.”

“Based on transformers and pre-trained unsupervised deep-learning models, a class of AI technology, large generative AI models (LGAIM), has emerged with the capability to produce novel outputs of a sophistication not seen earlier [10].”

“Generative AI empowers virtual assistants to evaluate a customer's financial history and present circumstances to deliver customised advice on savings, investments, or debt management [21].”

“However, possible regulation of large generative AI models such as ChatGPT are being seriously considered elsewhere (e.g. Hacker, Engel, & Mauer, 2023; Kandeel & Eldakak, 2024; Rao, 2023).”

“E, Synthesia, MusicLM) are a technology that is rapidly transforming the way we find information, communicate, illustrate, and create [11].”

“[7] GPT-3.”

“Research on collaboration between humans and AI is being developed in various forms, including research on the Data Communication Model[34], and research on the impact of AI on human cognition[35].”

“Generative AI is a branch of artificial intelligence capable of producing new content such as text, images, and music (Hacker, Engel and Mauer, 2023).”

“Others (LAION.ai, 2023; Hacker et al., 2023; Tumad ´ ottir, 2023) highlight the need to differentiate open-source systems from a regulatory stand-point, to avoid compliance costs unsustainable for open source contributors (Parliament, 2023).”

“Regarding the specific case of ChatGPT, researchers [19] [20] have outlined five key priorities for its role in research: focusing on human verification, developing rules for accountability, investing in truly open LLMs, embracing the benefits of AI, and widening the debate on LLMs. Open questions…”

“Although we are interested in policy and responses to advances in large generative AI models (LGAIMs) [35] such as ChatGPT or Stable Diffusion, regulatory frameworks, including the EU AI Act, tend to address both conventional AI systems and LGAIMs [35].”

“While these contributions hold potential for providing emotional support [34, 140], they also raise new challenges, including copyright disputes [58], targeted harms toward marginalized groups [44, 59], and the production of hate speech [60].”

“In the past, Hacker et al. [40] suggested using statistics to differentiate between writing produced by machines and humans.”

“Studies show that the GDPR provoked heated discussions among experts who compliment the data protection norm for simplifying standards [50] yet others point out its technical difficulties and insuffi-cient tech transformation capabilities [28, 41].”

“…the application of the algorithm leads to adverse consequences, the controller usually takes advantage of the complexity and unexplainability of the technology to shirk responsibility, claiming that they have no intention for infringement or subjective motivation to do so intentionally [13].”

“Additionally, production-grade chatbots such as ChatGPT and Gemini have raised concerns in potential failures, including policy compliance issues (Hacker et al., 2023), susceptibility to generating or endorsing violent content (Zhuo et al., 2023), and systemic bias (Fleisig et al., 2024).”

“Generative AI represents a significant leap in artificial intelligence, evolving from systems that analyze data or follow preset rules to those that can create original content [1].”

“In fact, ChatGPT brought a change so significant that not only its regulation [7, 8, 9], but extreme measures such as complete prohibition have been proposed and adopted at the institutional education level [10] and even by an entire country [11] (in this last case, temporarily).”

“A primary concern is their ability to generate false information or propagate misinformation [2].”

“The bot then goes through three meticulous checks that serve as guardrails to ensure the safety and integrity of the responses [18].”

“In other words, discriminative AI can tell you whether the animal in a picture is a cat; generative AI can actually draw you a picture of a cat (Hacker, Engel, and Mauer 2023, 3).”

“Nonetheless, individuals determined to use ChatGPT and similar LGAIMs, such as Mixtral and Llama 2, to create deceptive or harmful content will discover methods to elicit such responses.”

“…more such as Luminous, Bard, Bing for text generation, Stable Diffusion, DALL·E 2 for images, Syntesia for video and MusicLM for audio; see P. HACKER et al. Regulating ChatGPT and other Large Generative AI Models , p. 12. technology today you can be convinced that this goal has been reached 17 .”

“Regula - tory frameworks continue to evolve rapidly, while techni - cal capabilities and application contexts change at a similar pace, making it difficult to establish stable conditions under which operationalization methods can be meaningfully compared or evaluated [ 35 ].”

“The second concerns automated compliance for AI systems, with Hacker, Engel, and Mauer [36] and Kop [37] as anchor references for the European context, joined more recently by Veale and Borgesius [38].”

“In this paradigm, accidents at work are interpreted as the result of identifiable and separable causes, such as human error, technical failure, or improper work organization [8, 14].”

“When extended into multimodal agentic systems [10, 58], LLMs become central orchestrators that integrate heterogeneous inputs, plan complex workflows, invoke specialized tools, and coordinate multiple agents.”

“…Kumari et al., 2025) - Bias detection and fairness in financial AI (Tekale, 2024; Saha et al., 2025) - Regulatory uncertainty and global compliance (Hacker et al., 2023; G’sell, 2024) - Human-in-command and risk literacy (Aldasoro et al., 2024; Chauhan et al., 2025) offer forward-looking values…”

“…by an acute threat of confabulation and professional duty violation, is the development of generative AI - such as large language models that can be asked to draft pleadings and memoranda, along with correspondence to clients (Linna, 2025; Hacker et al., 2023; Larson, 2025; Ben-Daniel, 2025).”

“17 But this process has technical limitations.”

“Consequently, law-making bodies in many countries and regions are passing new regulations regarding the development and use of AI to reduce or avoid risks associated with this technology (Anderljung et al. 2023; Hacker et al. 2023).”

“This field faces challenges in regulating fine-tuning and feedback datasets to prevent privacy breaches and copy-right infringement.”

“With the dramatic growth of generative AI models, data, and parameters [18], [30], the application of Federated Learning (FL), as a collaborative cloud-edge inference mechanism, not only facilitates cooperation among multiple clients but also offers significant advantages in enhancing resource…”

“Furthermore, the intense employment of large generative AI models to manufacture and program AI machines enhances their online data generativity to produce online texts, images, etc. (Hacker et al. 2023, 1113).”

“Users may struggle to discern between original and erroneous content, potentially leading to its unwitting use in research (Hacker et al., 2023).”

“…(Arbel and Hoffman 2024; Engel and McAdams 2024; Liu and Li 2024; Posner and Saran 2025; Choi 2025), how governments ought to regulate or govern AI (Hacker et al. 2023; Korinek 2024; Agrawal et al. 2025; O'Keefe et al. 2025; Bullock et al. 2024), how generative AI is likely to shape the practice…”

“In addition, BLOOM and GPT-2 are building blocks for generative AI, which also needs to follow trustworthy AI requirements to achieve beneficial societal uses [30].”

“Our findings are in line with previous claims that LLMs are statisticians (Hacker et al., 2023; van Dijk et al., 2023), and LLMs rely on shallow heuristics for tasks requiring social intelligence (Aru et al., 2023; Shapira et al., 2024).”

“In response, society is crafting new strategies to temper AI’s influence, creating mitigation strategies [5, 10], regulations [11,12], governance frameworks [13,14], and other benchmarks and safeguards [15] that reflect a deep-seated commitment to responsibly shaping technological advancements.”