Regulating ChatGPT and other Large Generative AI Models

“Transparent LGAIMs connected to (good-old) knowledge bases, such as AtMan [43], present a promising way forward in this direction.”

“LGAIMs are usually trained with several billion, if not hundreds of billions, parameters [43, 44], requiring large amounts of training data and computing power [45].”

“OpenAI’s “CLIP” image classifier, for example, was built using a set of 400 million image-text pairs [46].”

“Although the emergence of these models in recent years constitutes a significant technical advance (for foundations, see [39-43]), they harness, to a reasonable extent, existing technologies in a vastly increased scale and scope.”

“2 LGAIMs can often digest human text input [57, 58] and produce an output (text; image; audio; video) based on it.”

“For example, some researchers identify a regulatory stack, seeing public priorities like transparency, data protection, risk management, and non-discrimination as essential normative concerns (Hacker, Engel, and Mauer 2023).”

“Third, there is a growing need for research on governance and regulation, including studies on how institutions and governments can design safe-guards, accountability mechanisms, and disclosure standards for ChatGPT in public-facing digital services [168].”

“However, concerns have been increasingly raised about the inclusion of copyrighted, private, or sensitive information in the training data [5], [6].”

“Given the tendency for foundation models to output the content they see during training (Nasr et al., 2023), both text-and image-based models are fine-tuned to refuse to respond to queries requesting objectionable responses Hacker et al. (2023); Ouyang et al. (2022).”

“As such, the most accurate answers it will provide relate to the most dogmatic or consistent messaging used during the training process, which includes human moderation, albeit in a currently non-transparent way (Hacker et al. , 2023).”

“8,11,16 Therefore, future studies should investigate whether ChatGPT can also be used for educational purposes to support the process of writing explanations for occupational therapy-related questions.”

“GenAI models are normally trained on several billion, sometimes hundreds of billions of different parameters, necessitating vast amounts of data and computing power [30].”

“…existing approaches to both coordination and safety problems presume critical use of traditional social technologies like policies and regulation [42] [86] [33], which are insufficient and too slow for coordinating games with a mixture of human and algorithmic participants, especially in…”

“The same holds true for the AI Act, where the literature, so far, focuses almost exclusively on the legal, economic and policy issues [101, 74, 50, 55, 81, 92] or on explainability questions [83, 97].”

“Hacker et al [48] argue that regulation should focus on specific high-risk applications, rather than on the pre-trained model itself, and should include transparency obligations, risk management, non-discrimination provisions, and content moderation rules.”

“Moreover, the texts ChatGPT generates may sometimes include inappropriate content, biased information, or misleading ideas (Borji, 2023; Hacker et al., 2023; Sallam, 2023).”

“AI-Powered Marketing in today's digital age, marketing must incorporate individualized and specific strategies (Hacker et al., 2023).”

“However, as the QA pairs used in the attack are explicitly harmful, such attacks are easily detected and blocked by moderation models [8, 9, 10, 11].”

“The challenges that Gen AI models pose involve and affect not one but a series of human stakeholders, including those whose data is used for model training and those who moderate the data, develop the models, create systems using these models, and use the results of these systems [44, 89, 120, 147].”

“A flurry of research has already emerged on how to set up safety guardrails in such models by finetuning them to align with harmless human preferences (Bai et al., 2022b; Hacker et al., 2023; Ouyang et al., 2022; Sun et al., 2023).”

“We focus on LLM supply chains due to their complexity [21] and timely relevance.”

“The collaborative emergence of generative artificial intelligence (AI) models has given rise to new AI paradigms, rapidly changing the way we communicate, learn, and create (Hacker, Engel, and Mauer 2023).”

“However, growing concerns have been raised about the potential misuse of undesirable data in training, such as copyrighted materials, privacy-sensitive information, or harmful content [5, 6].”

“However, these traditional methods may fall short as GAI content becomes advanced [205].”

“13 For related discussion concerning regulatory recommendations, see, e.g. Frosio [16] and Hacker et al. [18].”

“This issue stems from the model’s training data, which may include biased or discriminatory language (Hacker et al., 2023).”

“,71,73 Also, ChatGPT can be trained to generate extensive synthetic data for analysis.”

“Whereas NLP is a process in which these ML algorithms are applied to analyze and interpret human language (e.g., Lund & Wang, 2023), LLMs are a type of structure used for NLP and to output natural language text (Hacker et al., 2023).”

“Within just a short period of time, the global community was faced with the questions of how to best deal with such AI models and how to regulate them [41, 50, 96, 140].”

“…discussions are also proposed around its practical compliance with LLMs. Hacker et al. argue that this direct regulation is unsatisfactory and could be further enhanced from 1) the definition of GPAIS, 2) the risk management of GPAIS, and 3) the adverse consequences for competi - tion [101].”

“Instead, GAI focuses on creating new content by using the data it was trained on (Hacker et al., 2023; Murphy, 2022).”

“Growing scholarly and industry attention has been given to recognizing and addressing their risks [4, 17, 28, 36].”

“Hacker et al. (2023) argue that regulation should prioritize transparency and risk management and broaden content moderation guidelines to include governance in Gen AI models.”

“[5] Associated Press and OpenAI partner to explore generative AI to license news stories.”

“This diffusion strains legal and academic norms (Hacker et al., 2023; Mittelstadt and Floridi, 2016; Kosseff, 2019).”

“Unlike existing checklists (e.g., TRIPOD for prediction model development and its upcoming AI extension 45,46 or CLAIM for AI in medical imaging 47,48 ) that provide detailed guidance on key considerations and information to report in each section of a manuscript based on well-established best…”

“Students are using it to generate assignments and coding projects [8], [12], scholars rely on it for producing academic papers [20], and malicious actors exploit it to spread fake news on social media platforms [24], [13].”

“The challenges to prevent misuse of chatbots were not discussed [91, 92].”

“More recent studies have recognized that regulating content generation in GAI products is an emerging issue worthy of attention [2, 32, 69].”

“Regulatory mechanisms for GenAI have been proposed advocating three layers of regulation: universal technology-neutral regulation, regulation on high-risk applications of GenAI rather than pre-trained models, and regulation on information access [Hacker et al., 2023].”

“The recent widespread proliferation of artificial intelligence (AI), and especially Generative AI (GAI), is reshaping our world by redefining how we live, work, communicate, and create (Hacker, Engel, & Mauer, 2023).”

“by AI providers [35] or by the middlemen using it in products [32] have been proposed so as to better assess context-specific harms of a general-purpose AI system.”

“These techniques generally imply having in place both a pre-moderation (Hacker et al., 2023, p. 3) and a moderation procedure during which humans detect and remove any inappropriate or illegal content (input or output).”

“…examines how power relations, the economy, culture, and knowledge are shaped by colonial legacies, and seeks to promote the decolonization of these structures [ 23]. framework but also offers tools to address the underrepre - sentation of Latin American perspectives in the literature on AI ethics.”

“These frameworks are designed to facilitate the lawful and ethical use of generative AI, particularly concerning intellectual property rights, data security, and privacy protection (Hacker et al., 2023; Dixon, 2023).”

“Designers increasingly use tools like Midjourney, Stable Diffusion, and DALL-E for creating posters, brand identities, and typography, while ChatGPT aids in drafting design descriptions and refining creative expression [4, 5].”

“These models, while transformative in their capabilities to generate content based on patterns in existing data, bring to the fore pressing concerns about data privacy, copyright infringements, and the broader ethics of data usage [3].”

“This definition has been heavily criticised [26], as it is overly inclusive, so that, simple image or speech recognition systems would qualify as GPAIS.”

“The relationship between AI memorization and data protection law under the GDPR has garnered considerable attention [14, 33, 34, 82].”

“Secondly, although a few studies have discussed the governance countermeasures of potential GAI risks, they are mostly from the perspective of a single stakeholder and lack a comprehensive consideration of the governance of the dynamic game among multi-stakeholders (Hacker, Engel, and Mauer 2023).”

“FMs must align with the legal requirements of their deployment regions; for example, using ChatGPT-4-based FMware in China may not be viable due to local regulations [97, 98].”

“The use of Generative AI thereby opens avenues for civil litigation [ 27 , 28 , 32 ].”

“The generator network learns to generate new data samples, such as images or text, while the discriminator network learns to distinguish between real and generated samples [7].”

“Hacker et al. (2023) advocate technology-neutral regulations that are more appropriate for addressing emerging risks.”

“Additionally, Hacker (2023) highlights the transformative prospects as well as risks associated with large generative AI models (LGAIMs), such as ChatGPT, and how current regulations are not suited to manage this class of AI [41].”

“Generative AI, another subset of AI, is centred around models that generate new data instances that resemble the training data [16].”

“While LLMs are rapidly transforming the way people communicate, create, and work [26], significant advancements made by LLMs give rise to emerging ethical concerns like potential biases and fairness, reliability, and toxicity [88].”

“…applicable functions such as image and speech recognition, audio and video generation, pattern detection, question answering, translation and others; a general purpose AI system may be used in a plurality of contexts and be integrated in a plurality of other AI systems” (Article 3(1b) AI Act) [53].”

“For an in-depth overview of these legislative implications, also outstanding the EU ground, refer to the working paper of [64].”

“We will now discuss a suggestion of how to regulate foundation models in the context of trustworthy AI, concentrating on ChatGPT as an operational LLM [55].”

“A possible example is given by Hacker et al. [67], highlighting the malicious use of improperly secured LLMs to generate fake patients for insurance claims.”

“The Artificial Intelligence Act needs to keep pace with the rapid advancements in AI technology (Hacker et al., 2023).”

“The extraordinary capabilities of LLMs raise considerable concerns about their safety and responsibility (Floridi et al., 2021; Cath et al., 2018; Bommasani et al., 2021; Hacker et al., 2023).”

“With the development of ChatGPT, a crucial turning point in the field of conversational AI has emerged, in which all aspects of society can be influenced, particularly the field of medicine, from research to teaching (16, 17).”

“…than a year after the DSA was passed, commentators were already pointing out that it had not anticipated the rapidly increasing availability and popularity of generative AI, leaving concerning regulatory gaps and ambiguities, 80 while major platforms raced to implement tools with well-known risks.”

“12 Despite these advancements, there remains a notable oversight: the specific challenges posed by RAG GenAI have not been adequately addressed by existing frameworks.”

“Similarly, [4] highlighted the critical role of discriminant validity in their study on regulating ChatGPT and other large generative AI models.”

“Consequently, the argument over how to regulate generative AI systems is intensifying [86,87] .”