Regulating ChatGPT and other Large Generative AI Models

“Transparent LGAIMs connected to (good-old) knowledge bases, such as AtMan [43], present a promising way forward in this direction.”

“However, we note that recent studies have shown that LGAIMs are vulnerable to inversion attacks, even to a greater extent than previous generative models, such as generative adversarial networks (GANs [116]) [28].”

“LGAIMs are usually trained with several billion, if not hundreds of billions, parameters [43, 44], requiring large amounts of training data and computing power [45].”

“OpenAI’s “CLIP” image classifier, for example, was built using a set of 400 million image-text pairs [46].”

“Although the emergence of these models in recent years constitutes a significant technical advance (for foundations, see [39-43]), they harness, to a reasonable extent, existing technologies in a vastly increased scale and scope.”

“2 LGAIMs can often digest human text input [57, 58] and produce an output (text; image; audio; video) based on it.”

“Generally, it applies, in the US as well as the EU, in a technology-neutral way [4, 109-112, 113 ; but LGBTI, para.”

“Third, we highlight key data protection risks under the GDPR, with a particular focus on model inversion [27-29].”

“The AI models covered by this contribution are often referred to as ‘foundation models’ [36], ‘large language models’ (LLMs) [37] or ‘large generative models’ (LGAIMs–the term adopted in this article) [38].”

“By sampling from and mixing them, the model can generate content beyond the training data set–thus something new, as some commentators put it [55, 56].”

“and, in particular, transformers, more efficient [48, 49], the energy required to train models this large has triggered concerns from a climate policy perspective [25, 50-54] (see also Part 6).”

“Third, we highlight key data protection risks under the GDPR, with a particular focus on model inversion [27-29].”

“However, errors will be costly, and risks ranging from discrimination and privacy to disrespectful content need to be adequately addressed [4-6].”

“The AI models covered by this contribution are often referred to as ‘foundation models’ [36], ‘large language models’ (LLMs) [37] or ‘large generative models’ (LGAIMs–the term adopted in this article) [38].”

“Generative models, as virtually any novel technology, may be used for better (think: birthday cards) or worse purposes ( think: shitstorm) [119].”

“28-28b AI Act EP Version2) contain provisions to explicitly regulate LGAIMs, even if their providers are based outside of the EU [14, cf. also 23].”

“This would raise the typical competition law issues of vertical integration [76-78].”

“The content generated by these models can, therefore, be biased, prejudiced or harmful [15, 60].”

“AI Research has made progress in this area recently [120-122].”

“While the carbon footprint of IT and AI is significant and steadily rising [50-54], and training of LGAIMs is particularly resource intensive [136], large models may ultimately create significantly fewer greenhouse gas emissions than their smaller brethren [136, p.”

“To avoid or at least mitigate this issue, model developers need to use proper curating techniques [61, 62].”

“However, errors will be costly, and risks ranging from discrimination and privacy to disrespectful content need to be adequately addressed [4-6].”

“While the carbon footprint of IT and AI is significant and steadily rising [50-54], and training of LGAIMs is particularly resource intensive [136], large models may ultimately create significantly fewer greenhouse gas emissions than their smaller brethren [136, p.”

“The situation is slightly different with non-discrimination law, which generally applies, in the US as well as the EU, in a technology-neutral way [55-60].”

“Typically, LGAIM developers will pre-train a large model, which may then be fine-tuned by deployers, potentially in collaboration with developers [81, 82], while users ultimately make the decision what the AI system is used for specifically (e.”

“According to that rule, every simple image or speech recognition system seems to qualify, irrespective of the breadth of its capabilities; rightly, this only corresponds to a minority position in the technical GPAIS literature [14, 73].”

“AI Research has made progress in this area recently [64-66].”

“Beyond this point, only the users should be the subject of regulation and civil liability (and vice versa, for example in control-via-API cases): incentives for action only make sense where the person incentivized is actually in a position to act [89, 90].”

“To avoid or at least mitigate this issue, model developers need to use proper curating techniques [61, 62].”

“5(1)(a) GDPR, the fairness principle [103].”

“Third, we highlight key data protection risks under the GDPR, with a particular focus on model inversion [27-29].”

“Beyond this point, only the users should be the subject of regulation and civil liability (and vice versa, for example in control-via-API cases): incentives for action only make sense where the person incentivized is actually in a position to act [89, 90].”

“The AI models covered by this article are often referred to as ‘foundation models’ [18], ‘large language models’ (LLMs) [19] or ‘large generative models’ (LGAIMs) [20].”

“Importantly, more interdisciplinary research is necessary to develop markings that are easy to use and recognize, but hard to remove by average users [139].”

“Due to space constraints, we cannot address all social and regulatory concerns regarding LGAIMs and have to bracket, for example, questions of IP law, power dynamics [35, 36], a deeper exploration of the comparative advantages of technology-neutral and technology-specific regulation [37], or the use of LGAIMs in milit…”

“[28, 53] that aim at allowing them “to find patterns and relationships”

“Furthermore, the speed and syntactical accuracy of LGAIMs make them the perfect tool for the mass creation of highly polished, seemingly fact-loaded, yet deeply twisted fake news [7, 17].”

“This should be coupled with research on AI-content detection to highlight such output where watermarks fail [121, 140].”

“As consent was typically not obtained [105], developers need to avail themselves of the balancing test and/or the purpose change test (Article 6(1)(f) and (4) GDPR) [106-109], potentially in conjunction with specific research exceptions (Article 89 GDPR) [110-112].”

“The liability directives proposed by the EU Commission, for example, contain elaborate evidence disclosure rules pitting the compensation interests of injured persons against the secrecy interests of AI developers and deployers [24, 77, 78].”

“This would raise the typical competition law issues of vertical integration [76-78].”

“Already now, LGAIMs’ unbridled capacities may be harnessed to take manipulation, fake news, and harmful speech to an entirely new level [7-11].”

“, MusicLM), while further alternatives are already in the pipeline [1-3].”

“Scholars and regulators have long suggested that technology-neutral laws may be better prepared to tackle emerging risks given the rapid pace of innovation in machine learning [144-146].”

“This is crucial not only for innovation and or consumer welfare [33, 159, 160], but also for environmental sustainability.”

“As consent was typically not obtained [105], developers need to avail themselves of the balancing test and/or the purpose change test (Article 6(1)(f) and (4) GDPR) [106-109], potentially in conjunction with specific research exceptions (Article 89 GDPR) [110-112].”

“and, in particular, transformers, more efficient [48, 49], the energy required to train models this large has triggered concerns from a climate policy perspective [25, 50-54] (see also Part 6).”

“As a result, the debate on how (not) to regulate LGAIMs is becoming increasingly intense [12-22].”

“The Russian attack on Ukraine, the Corona pandemic, climate change, and the political feuds in the US and beyond already fuel hate crimes and fake news [127, 128].”

“Third, and related to the first and second aspect, individual actors in the AI value chain may simply not have the allencompassing knowledge and control that would be required if they were the sole addressees of regulatory duties [93].”

“Compliance with the GDPR will depend on a range of factors [113-116], such as the intended purposes of the model, the type of personal data used, the likelihood of model inversion, and the probability of re-identification of concrete data subjects.”

“As consent was typically not obtained [105], developers need to avail themselves of the balancing test and/or the purpose change test (Article 6(1)(f) and (4) GDPR) [106-109], potentially in conjunction with specific research exceptions (Article 89 GDPR) [110-112].”

“LGAIMs are usually trained with several billion, if not hundreds of billions, parameters [43, 44], requiring large amounts of training data and computing power [45].”

“, MusicLM), while further alternatives are already in the pipeline [1-3].”

“Additional factors, such as the balance of power among developers, must also be considered [149].”

“While scholars are divided about whether the use of personal data for machine learning purposes should be permitted under the balancing test (permissive view: [26]; more restrictive view: [109, 117]; see also [118]), the threat of model inversion arguably weighs in favor of data subjects.”

“This notice-and-action mechanism, structurally known from copyright law [64-66] and the DSA [32, 34, 67, 68], complements the novel, active production monitoring obligation (Articles 61, 4(2) AI Act), which had so far only been contained in the product liability tort law of some Member States [69] and will be partiall…”

“Due to space constraints, we cannot address all social and regulatory concerns regarding LGAIMs and have to bracket, for example, questions of IP law, power dynamics [35, 36], a deeper exploration of the comparative advantages of technology-neutral and technology-specific regulation [37], or the use of LGAIMs in milit…”

“AI moderator, with controversial help from Kenyan contractors , to detect and block harmful content [12].”

“Generally, it applies, in the US as well as the EU, in a technology-neutral way [4, 109-112, 113 ; but LGBTI, para.”

“While the carbon footprint of IT and AI is significant and steadily rising [26-30], and training of LGAIMs is”

“This is less of a concern for copyrighted material found in the training data by way of model inversion as the new text-and-data-mining exceptions (Articles 3 and 4 Copyright in the Digital Single Market Directive5), at least generally and where rightsholders have not opted out, allow the use of publicly accessible co…”

“Article 15(4) GDPR contains a similar provision, which by way of analogy also applies to the access right in Article 15(1) GDPR [99, 100].”

“We show that the EU’s prime instrument to combat harmful speech, the Digital Services Act (DSA) [16, 17], does not apply to LGAIMs, creating a dangerous regulatory loophole.”

“However, non-discrimination provisions may apply more broadly to the pre-trained model itself (more precisely: to its developers) to mitigate bias at its data source [see also 25].”

“The situation is slightly different with non-discrimination law, which generally applies, in the US as well as the EU, in a technology-neutral way [55-60].”

“Compliance with the GDPR will depend on a range of factors [127-130], such as the intended purposes of the model, the type of personal data used, the likelihood of model inversion, and the probability of re-identification of concrete data subjects.”

“As consent was typically not obtained [105], developers need to avail themselves of the balancing test and/or the purpose change test (Article 6(1)(f) and (4) GDPR) [106-109], potentially in conjunction with specific research exceptions (Article 89 GDPR) [110-112].”

“According to that rule, every simple image or speech recognition system seems to qualify, irrespective of the breadth of its capabilities; rightly, this only corresponds to a minority position in the technical GPAIS literature [14, 66].”

“Prompt engineering is becoming a new art to elicit any content from LGAIMs [123] and fake news is harder to detect than hate speech, even though industry efforts are underway via increased model and source transparency [124].”

“The enforcement of any user-focused transparency rule being arduous, it must be supported by technical measures such as digital rights management and watermarks imprinted by the model [138].”

“Due to space constraints, we cannot address all social and regulatory concerns regarding LGAIMs and have to bracket, for example, questions of IP law, power dynamics [35, 36], a deeper exploration of the comparative advantages of technology-neutral and technology-specific regulation [37], or the use of LGAIMs in milit…”

“Scholars and regulators have long suggested that technology-neutral laws may be better prepared to tackle emerging risks given the rapid pace of innovation in machine learning [144-146].”

“The unprovoked Russian attack on Ukraine, the Corona pandemic, climate change, and the political feuds in the US and beyond already fuel hate crimes and fake news [152, 153].”

“Compliance with the GDPR will depend on a range of factors [113-116], such as the intended purposes of the model, the type of personal data used, the likelihood of model inversion, and the probability of re-identification of concrete data subjects.”

“Extensive literature and practical experience concerning this problem exists in the realm of the US pretrial discovery system [88-92].”

“As consent was typically not obtained [119], developers need to avail themselves of the balancing test and/or the purpose change test (Article 6(1)(f) and (4) GDPR) [120-123], potentially in conjunction with specific research exceptions (Article 89 GDPR) [124-126].”

“This is crucial not only for innovation and or consumer welfare [16, 68, 69], but also for environmental sustainability.”

“Similar effects have already been established concerning the GDPR [47].”

“LGAIM risk regulation, in our view, should generally focus on deployed applications, not the pre-trained model [12, 25].”

“with developers [73, 74], while users ultimately make the decision what the AI system is used for specifically (e.”

“While scholars are divided about whether the use of personal data for machine learning purposes should be permitted under the balancing test (permissive view: [26]; more restrictive view: [109, 117]; see also [118]), the threat of model inversion arguably weighs in favor of data subjects.”

“We show that the EU’s prime instrument to combat harmful speech, the Digital Services Act (DSA) [16, 17], does not apply to LGAIMs, creating a dangerous regulatory loophole.”

“” While the notice and action mechanism applies to all hosting services, instruments like trusted flaggers, obligatory dispute resolution, and risk management systems are reserved for the narrower group of “online platforms” [131].”

“Already now, LGAIMs’ unbridled capacities may be harnessed to take manipulation, fake news, and harmful speech to an entirely new level [7-11].”

“The liability directives proposed by the EU Commission, for example, contain elaborate evidence disclosure rules pitting the compensation interests of injured persons against the secrecy interests of AI developers and deployers [25, 70, 71].”

“This notice-and-action mechanism, structurally known from copyright law [64-66] and the DSA [32, 34, 67, 68], complements the novel, active production monitoring obligation (Articles 61, 4(2) AI Act), which had so far only been contained in the product liability tort law of some Member States [69] and will be partiall…”

“Extensive literature and practice concerning this problem exists in the realm of the US pretrial discovery system [49-53].”

“unsurmountable trade secrets or IP rights [96-98].”

“The liability directives proposed by the EU Commission, for example, contain elaborate evidence disclosure rules pitting the compensation interests of injured persons against the secrecy interests of AI developers and deployers [25, 70, 71].”

“Hence, we introduce the following distinction within the category of users: o professional user: an entity using AI output for professional purposes, generally as defined in EU consumer law [89, 90].”

“Hence, we introduce the following distinction within the category of users: o professional user: an entity using AI output for professional purposes, generally as defined in EU consumer law [83, 84].”

“Even the most stringent application of DSA enforcement mechanisms, potentially coupled with GDPR provisions on erasure of data (Article 17(2) and 19 GDPR), cannot undo the harm done, and often cannot prevent the forward replication of problematic content [133].”

“, MusicLM), while further alternatives are already in the pipeline [1-3].”

“In details, regulation should focus on concrete high-risk applications, and not the pre-trained model itself, and should include (i) obligations regarding transparency, (ii) risk management, (iii) non-discrimination provisions, and (iv) content moderation rules 15 .”

“Regarding relevant policies and regulations, Hacker et al. [25] discussed the nature and rules of large generative AI models, including ChatGPT, which are rapidly changing the way we communicate, explain, and create.”

“However, they also promote reliability, legal accountability, and ethical standards in AI development, fostering trust among users and stakeholders and encouraging the responsible and wider adoption of AI technologies in healthcare [135].”

“…the European Union AI Act or US National AI Initiative) serve as promising initial steps toward opening pathways for future research to effectively address these issues, ensuring that AI-generated content is morally, ethically, and legally appropriate and cannot be mis-used (Hacker et al., 2023).”

“These generated datasets can reflect real-world complexities while preserving the privacy and anonymity necessary in many use cases (Hacker, Engel, and Mauer 2023).”

“For example, improperly secured LLMs could enable bad actors to generate fake patient data or insurance claims [62].”

“…article, only a few countries have begun a legislative process regulating AI and its applications, it is likely that several others will follow [37, 65] Parliament approved the European AI Act [32], determining a set of norms to be followed with respect to several aspects of AI at large [41,…”

“[37] Generic Conf.”

“For example, academic proposals to regulate generative AI have collectively argued that regulation should occur at the stage of deployment (e.g., at the advertiser and platform level), rather than at the level of message reception (e.g., at the time point consumers are seeing the specific ad) 79 .”

“, 2021), we call for more research on the regulation aspect of LLMs (Anderljung et al., 2023; Ho et al., 2023; Trager et al., 2023; Hacker et al., 2023; Barrett et al., 2022) and countermeasures against LLM-”

“Therefore, the question of how LLMs should be governed has attracted much attention, with proposals ranging from structured access protocols designed to prevent malicious use [76] to hard regulation prohibiting the deployment of LLMs for specific purposes [86].”

“Regarding relevant policies and regulations, Hacker et al. [25] discussed the nature and rules of large generative AI models, including ChatGPT, which are rapidly changing the way we communicate, explain, and create.”

“Last but not least, managing the accompanying risks of prompt engineering and large models requires the close collaboration of society, research institutions, and government [214, 215].”

“This issue arises due to the model's training data, which can contain biased or discriminatory language (Hacker et al., 2023).”

“12 Similarly, general users of GenAI are unlikely to understand the technology well enough to prescribe a reasonable amount of trust on its output or be able to identify potential misinformation.”

“Issues related to data privacy, transparency, and intellectual property must be addressed to ensure that these tools are used responsibly [18].”

“In the context of foundation models, prompting is highly relevant because the ability of the respective system to generate diverse and possibly objectionable output demands the possibility to specify and constrain desirable generations (Hacker et al. 2023).”

“Furthermore, training ChatGPT has caused severe mental stress on its content curators [21,22].”

“However, such a system would require output that could be validated, and moderation of content generated by LLMs has been highlighted as a critical concern [11].”

“Recent studies evaluated LLMs from various aspects including natural language processing [2, 41, 26], ethics [17, 45], robustness [56, 69], and education [21, 16].”

“Similarly, in the 2021 NLME exam, ChatGPT correctly answered 275 out of 600 items (45.8%).”

“One prominent aspect of AI safety is the process of model alignment , which is an umbrella term referring to algorithms which ensure that LLM outputs are consistent with human values [27]–[29].”

“These articles pose a unique challenge for Generative AI due to the nature and scope of the data they process [2].”

“technologies, including data privacy, intellectual property infringement, and security vulnerabilities [70, 138, 161].”

“Specifically, this survey concentrates on the detection of text and code generated by LLMs, as well as the attacks.”

“A significant milestone in this evolution is Generative AI (GAI), which exceeds mere analytics to enable machine creativity [2], [3], [4], [5].”

“…in creative applications such as those associated with art and music, but it can also be used for more practical applications by learning patterns and structures in a large dataset during training, and then generating new data that follows the same patterns and structures (Hacker et al., 2023).”

“Regulation of large generative models is also proposed to ensure trans-parency, risk management, non-discrimination, and content moderation obligations [19].”

“Hacker, Engel, and Mauer (2023) situate the emerging generative models in the current debate on AI regulation and suggest that more differentiated regulation would better fit the realities of the evolving AI value chain. de Rassenfosse, Jaffe, and Wasserman (2023) propose that current policy…”

“Hacker et al. [60] presented a brief discussion on various issues regulating large generative AI models, such as ChatGPT.”

“[18, 41] expressed concerns that ChatGPT and other large models should be regulated since they are double-edged swords.”

“Others argue that such a move would stifle innovation and make it more difficult to develop safer overall models [6, 17, 25].”

“Themes and sub-themes, with examples selected mainly based on participant votes, are presented below:”

“Furthermore, Hacker et al. (2023) call for specific regulation of LLM-based tools, “large generative AI models”, under the EU Digital Services Act and provide four concrete, workable suggestions that include transparency obligations, mandatory yet limited risk management, non-discrimination data…”

“Hacker, Engel, and Mauer (2023) [22] discuss the necessity for a comprehensive risk management framework for large generative AI models like ChatGPT, emphasising the need to address potential high-risk applications and innate vulnerabilities of these technologies.”

“[104, 105, 130].”

“The need to cite the sources for medical statements goes beyond gaining clinician and patient trust -- there is also an urgent regulatory case as well 19 .”

“Therefore, it is necessary to urgently reconsider the regulation of general-purpose LLMs. 9 The parroting and hallucination issues show that minimal transparency obligations are insufficient, since LLMs often lull users into misplaced trust.”

“…Borge-sius, 2021; Gstrein, 2022, pp. 756–760), the unprecedented popularity of general-purpose AI sparked a debate on whether it requires specific laws (Hacker et al., 2023; Helberger & Diakopoulos, 2023), or even legally binding international treaties (Harris, 2023; Milmo & Stacey, 2023).”

“These principles are aligned with concerns raised in the literature regarding the risks associated with data privacy and ethical data handling in AI applications [66, 67].”

“Our study revealed the intricate ways in which practitioners individually confronted and navigated complex ethical challenges, especially when GenAI was operated in a ‘grey area’ lacking clear policies or regulations [1, 34].”

“For this reason, throughout this paper, the broader term ‘ large generative AI models (LGAIMs) ’ will be adopted to encompass all the different types of generative AI models, of which large language models (LLMs) are only a subset (10).”

“In a semi-technical setting – e.g., technically informed researchers writing for a broader or interdisciplinary audience – GenAI is usually understood to somehow refer to generative models (Hacker et al., 2023), but this relationship is often not specified further.”

“Utilising GenAI tools comes with risks and errors (Hacker et al., 2023).”

“To address harmful fine-tuning attacks, recent mainstream defenses can be divided into three categories: alignment-stage defense [17], [18], [19], [20], fine-tuning-stage defense [21], [22], and post-fine-tuning stage defense [23], [24], [25].”

“Regulation of large generative models is also proposed to ensure transparency, risk management, and non-discrimination [21].”

“The use cases of these computing paradigms will be tempered by stiffer data privacy laws across these computing paradigms, as emphasized in [63,64].”

“…discussions are also proposed around its practical compliance with LLMs. Hacker et al. argue that this direct regulation is unsatisfactory and could be further enhanced from 1) the definition of GPAIS, 2) the risk management of GPAIS, and 3) the adverse consequences for competi - tion [101].”

“It is even listed as the first objective in the EU AI Act (Artificial Intelligence Act, 2021, sec.”

“ChatGPT has led the revolutionary development of generative artificial intelligence/large language models (hereafter, GenAI LLMs) [1 – 3] and has gradually been applied to important fields such as healthcare [4,5].”

“Participants in the sandbox were generally cautiously optimistic about the need for AI regulation [23].”

“…integrating LLMs in conversational interaction include combinations of rule-based and heuristic methods for determining the correct next utterance [33], where the results of this paper can be directly applied to guide the VA in both informing the user of the current state and in how they can…”

“Forexample,undertheEUAIAct,theuseofgenerative AIineducationwouldbesubjecttorigorousethicalandregulatory standards,focusingonriskassessment,transparency,dataprotection, andfairness 105 .”

“The advent of LLMs has led to further concerns about their potential to encode harmful biases due to their reliance on large-scale, uncu-rated datasets [8, 34].”

“The need to cite the sources for medical statements goes beyond gaining clinician and patient trust – there is also an urgent regulatory case as well (Hacker et al., 2023).”

“Regulatory bodies can later extract this information to trace and regulate AI-generated content (Atal-lah et al., 2001; Wilson et al., 2014; Hacker et al., 2023).”

“Recent evolutions in generative AI tools (Hacker et al., 2023) have led to their increased sophistication, utility, and accessibility”

“Malicious actors use it to propagate fake news on social media platforms [19], [9], and educational institutions are employing it to provide mental health education to students without their consent [2], among other uses.”

“To enhance safety during fine-tuning, numerous defense mechanisms employ model-level regularization [64, 56] and detection architectures [24, 38, 39, 11], complemented by data manipulation techniques to mitigate harmful data influences [5, 107, 19, 77, 51].”

“Indeed, the propensity of LLMs to reproduce this objectionable content has invigorated the field of AI alignment [2–4], wherein various mechanisms are used to “align” the output text generated by LLMs with ethical and legal standards [5–7].”