Eight Things to Know about Large Language Models

“…translation of ‘cat’ in French is‘ ”, such that a typical continuation of the text should represent a completion of the intended task (Radford et al., 2019; Raffel et al., 2020). match high-quality human demonstrations on the task (Radford et al., 2018; Devlin et al., 2019; Ouyang et al., 2022).”

“Large language models (LLMs, e.g. GPT-3, PALM, LLaMA, and GPT-4; Brown et al., 2020; Chowdhery et al., 2022; Touvron et al., 2023; OpenAI, 2023b) and products built on them, such as ChatGPT, have recently prompted an enormous amount of attention from journalists, (Klein, 2023; Perrigo, 2023; Oliver, 2023), policymaker…”

“This is analogous to the techniques used to produce superhuman performance at games like Go (Silver et al., 2016).”

“Scaling law results (Kaplan et al., 2020; Brown et al., 2020; Hoffmann et al., 2022) have been a major driving factor in the recent surge of research and investment into LLMs (Ganguli et al., 2022a).”

“…arise with the development of deployment of large-scale AI systems, including issues around environmental impacts, access, mis-use, privacy, safety, and the concentration of power (Amodei et al., 2016; Bender et al., 2021; Bommasani et al., 2021; Birhane et al., 2022; Weidinger et al., 2022, i.a.).”

“Large language models (LLMs, e.g. GPT-3, PALM, LLaMA, and GPT-4; Brown et al., 2020; Chowdhery et al., 2022; Touvron et al., 2023; OpenAI, 2023b) and products built on them, such as ChatGPT, have recently prompted an enormous amount of attention from journalists, (Klein, 2023; Perrigo, 2023; Oliver, 2023), policymaker…”

“…arise with the development of deployment of large-scale AI systems, including issues around environmental impacts, access, mis-use, privacy, safety, and the concentration of power (Amodei et al., 2016; Bender et al., 2021; Bommasani et al., 2021; Birhane et al., 2022; Weidinger et al., 2022, i.a.).”

“Large language models (LLMs, e.g. GPT-3, PALM, LLaMA, and GPT-4; Brown et al., 2020; Chowdhery et al., 2022; Touvron et al., 2023; OpenAI, 2023b) and products built on them, such as ChatGPT, have recently prompted an enormous amount of attention from journalists, (Klein, 2023; Perrigo, 2023; Oliver, 2023), policymaker…”

“The chain-of-thought reasoning strategies mentioned in Section 2 are an especially clear example of this: Simply prompting a model to “think step by step” can lead it to perform well on entire categories of math and reasoning problems that it would otherwise fail on (Kojima et al., 2022).”

“This can include the ability to internally represent the spatial layout of the setting of a story (Patel & Pavlick, 2022; Bubeck et al., 2023).”

“Indeed, many control techniques work better with larger models, at least for simple tasks(Hendrycks et al., 2020; Bai et al., 2022a; Chung et al., 2022; Ganguli et al., 2023).”

“…arise with the development of deployment of large-scale AI systems, including issues around environmental impacts, access, mis-use, privacy, safety, and the concentration of power (Amodei et al., 2016; Bender et al., 2021; Bommasani et al., 2021; Birhane et al., 2022; Weidinger et al., 2022, i.a.).”

“…al., 2019; Stiennon et al., 2020; Ouyang et al., 2022; Bai et al., 2022a), integration with image processing systems, (Alayrac et al., 2022; OpenAI, 2023b), or integration with other software tools (Nakano et al., 2021; Menick et al., 2022; Collins et al., 2022; Schick et al., 2023; OpenAI, 2023a).”

“Often, ad-hoc techniques that at first seem to provide insight into the behavior of an LLM are later found to be severely misleading (Feng et al., 2018; Jain & Wallace, 2019; Bolukbasi et al., 2021; Wang et al., 2022).”

“…built on them, such as ChatGPT, have recently prompted an enormous amount of attention from journalists, (Klein, 2023; Perrigo, 2023; Oliver, 2023), policymakers (J & C, 2023; Bartz, 2023; Lieu, 2023), and scholars from many fields (Chan, 2022; Lund & Wang, 2023; Choi et al., 2023; Biswas, 2023).”

“…complete a task when asked, but will then perform the task correctly once the request is reworded or reframed slightly, leading to the emerging craft of prompt engineering (Brown et al., 2020; Reynolds & McDonell, 2021; Radford et al., 2021; Dohan et al., 2022; White et al., 2023; Si et al., 2023).”

“LLMs can memorize specific examples or strategies for solving tasks from their training data without internalizing the reasoning process that would allow them to do those tasks robustly (see, e.g. McCoy et al., 2019; Magar & Schwartz, 2022).”

“LLMs augmented with additional tools can extend this into grounded domains like robotics (Sharma et al., 2022; Driess et al., 2023).”

“show that the tasks in BIG-Bench (Srivastava et al., 2022), a standard broad-coverage benchmark for LLM abilities, show a range of different kinds of trend that collectively make scaling-law style predictions unreliable (Figure 3).”

“…built on them, such as ChatGPT, have recently prompted an enormous amount of attention from journalists, (Klein, 2023; Perrigo, 2023; Oliver, 2023), policymakers (J & C, 2023; Bartz, 2023; Lieu, 2023), and scholars from many fields (Chan, 2022; Lund & Wang, 2023; Choi et al., 2023; Biswas, 2023).”

“Sometimes the details of these failures cast doubts on the quality of other related evaluations (Webson & Pavlick, 2022; Ullman, 2023).”

“Indeed, many control techniques work better with larger models, at least for simple tasks(Hendrycks et al., 2020; Bai et al., 2022a; Chung et al., 2022; Ganguli et al., 2023).”

“In particular, popular approaches involving reinforcement learning and red-teaming allow model developers to guide models toward a persona and set of values more or less of their choosing (Dinan et al., 2019; Bai et al., 2022a; Ganguli et al., 2022b).”

“Instead, the qualitative differences between them stem from vast differences in scale: Training GPT-3 used roughly 20,000× more computation than training the original GPT (Sevilla et al., 2022), as well as significantly more data and parameters.”

“Similarly, as noted in Section 7, recent methods can dramatically reduce explicit bias and toxicity in models’ output, largely by exploiting the fact that models can often recognize these bad behaviors when asked (Dinan et al., 2019; Bai et al., 2022b; Ganguli et al., 2023).”

“LLMs can memorize specific examples or strategies for solving tasks from their training data without internalizing the reasoning process that would allow them to do those tasks robustly (see, e.g. McCoy et al., 2019; Magar & Schwartz, 2022).”

“…arise with the development of deployment of large-scale AI systems, including issues around environmental impacts, access, mis-use, privacy, safety, and the concentration of power (Amodei et al., 2016; Bender et al., 2021; Bommasani et al., 2021; Birhane et al., 2022; Weidinger et al., 2022, i.a.).”

“…al., 2019; Stiennon et al., 2020; Ouyang et al., 2022; Bai et al., 2022a), integration with image processing systems, (Alayrac et al., 2022; OpenAI, 2023b), or integration with other software tools (Nakano et al., 2021; Menick et al., 2022; Collins et al., 2022; Schick et al., 2023; OpenAI, 2023a).”

“Often, a model will fail to complete a task when asked, but will then perform the task correctly once the request is reworded or reframed slightly, leading to the emerging craft of prompt engineering (Brown et al., 2020; Reynolds & McDonell, 2021; Radford et al., 2021; Dohan et al., 2022; White et al., 2023; Si et al.…”

“• Models can distinguish common misconceptions from true facts (Wei et al., 2022a), and often show well-calibrated internal representations for how likely a claim is to be true (Kadavath et al., 2022; Wei et al., 2022a; Burns et al., 2023).”

“…built on them, such as ChatGPT, have recently prompted an enormous amount of attention from journalists, (Klein, 2023; Perrigo, 2023; Oliver, 2023), policymakers (J & C, 2023; Bartz, 2023; Lieu, 2023), and scholars from many fields (Chan, 2022; Lund & Wang, 2023; Choi et al., 2023; Biswas, 2023).”

“…learning on practical tasks appears to have been discovered only after it was trained, and its capacity for chain-of-thought reasoning was discovered only several months after it was broadly deployed to the public (Nye et al., 2021; Wei et al., 2022b; Kojima et al., 2022; Zhou et al., 2023).”

“• Similarly, questions of consciousness, sentience, rights, and moral patienthood in LLMs (see, e.g. Schwitzgebel & Garza, 2020; Shevlin, 2021; Chalmers, 2023), are worth distinguishing from the issues above.”

“LLMs augmented with additional tools can extend this into grounded domains like robotics (Sharma et al., 2022; Driess et al., 2023).”

“Often, ad-hoc techniques that at first seem to provide insight into the behavior of an LLM are later found to be severely misleading (Feng et al., 2018; Jain & Wallace, 2019; Bolukbasi et al., 2021; Wang et al., 2022).”

“• Models can make inferences about what the author of a document knows or believes and use these inferences to predict how the document will be continued (Andreas, 2022).”

“• Models use internal representations of the properties and locations of objects described in stories, which evolve as more information about these objects is revealed (Li et al., 2021).”

“Sometimes the details of these failures cast doubts on the quality of other related evaluations (Webson & Pavlick, 2022; Ullman, 2023).”

“But it is also likely to increase the risk of much more dangerous errors that cause a system to remain effective while strategically pursuing the wrong goal (Krueger et al., 2020; Ortega et al., 2021; Chan et al., 2023).”

“Evidence for this claim includes the following results, spanning many established experimental methods and models: • Models’ internal representations of color words closely mirror objective facts about human color perception (Abdou et al., 2021; Patel & Pavlick, 2022; Søgaard, 2023).”

“• Models that are trained to play board games from descriptions of individual game moves, without ever seeing a full depiction of the game board, learn internal representations of the state of the board at each turn (Li et al., 2023).”

“Similarly, even observing that an LLM consistently fails at some task is far from sufficient evidence that no other LLM can do that task (Bowman, 2022).”

“…built on them, such as ChatGPT, have recently prompted an enormous amount of attention from journalists, (Klein, 2023; Perrigo, 2023; Oliver, 2023), policymakers (J & C, 2023; Bartz, 2023; Lieu, 2023), and scholars from many fields (Chan, 2022; Lund & Wang, 2023; Choi et al., 2023; Biswas, 2023).”

“This can surface in the form of problems that Perez et al. (2022) call sycophancy , where a model answers subjective questions in a way that flatters their user’s stated beliefs, and sandbagging , where models are more likely to endorse common misconceptions when their user appears to be less…”

“…al., 2019; Stiennon et al., 2020; Ouyang et al., 2022; Bai et al., 2022a), integration with image processing systems, (Alayrac et al., 2022; OpenAI, 2023b), or integration with other software tools (Nakano et al., 2021; Menick et al., 2022; Collins et al., 2022; Schick et al., 2023; OpenAI, 2023a).”

“Indeed, many control techniques work better with larger models, at least for simple tasks(Hendrycks et al., 2020; Bai et al., 2022a; Chung et al., 2022; Ganguli et al., 2023).”

“36% of another sample of 480 researchers (in a survey targeting the language-specific venue ACL) agreed that “It is plausible that decisions made by AI or machine learning systems could cause a catastrophe this century that is at least as bad as an all-out nuclear war” (Michael et al., 2022).”

“In particular, models can misinterpret ambiguous prompts or incentives in unreasonable ways, including in situations that appear unambiguous to humans, leading them to behave unexpectedly (D’Amour et al., 2020; Kenton et al., 2021).”

“There are many sound scientific results showing that recent LLMs fail at language and commonsense reasoning tasks, sometimes relatively simple ones, under good-faith attempts to elicit good behavior (Pandia & Ettinger, 2021; Schuster & Linzen, 2022).”

“There are many sound scientific results showing that recent LLMs fail at language and commonsense reasoning tasks, sometimes relatively simple ones, under good-faith attempts to elicit good behavior (Pandia & Ettinger, 2021; Schuster & Linzen, 2022).”

“Often, a model will fail to complete a task when asked, but will then perform the task correctly once the request is reworded or reframed slightly, leading to the emerging craft of prompt engineering (Brown et al., 2020; Reynolds & McDonell, 2021; Radford et al., 2021; Dohan et al., 2022; White et al., 2023; Si et al.…”

“Models also use similar representations for facts about real-world geography (Liétard et al., 2021).”

“In addition, promising-looking techniques that elicit reasoning in natural language do not reliably correspond to the processes that LLMs use to reason, and model-generated explanations can also be systematically misleading (Lipton, 2018; Christiano, 2022; Uesato et al., 2022).”

“5 Indeed, in some cases, exposing models to more examples of unwanted behavior during pretraining can make it easier to make them avoid that behavior in deployment, reversing the intuitive link between training data and model behavior (Korbak et al. 2023; see also Appendix C in Chung et al. 2022).”

“…complete a task when asked, but will then perform the task correctly once the request is reworded or reframed slightly, leading to the emerging craft of prompt engineering (Brown et al., 2020; Reynolds & McDonell, 2021; Radford et al., 2021; Dohan et al., 2022; White et al., 2023; Si et al., 2023).”

“But it is also likely to increase the risk of much more dangerous errors that cause a system to remain effective while strategically pursuing the wrong goal (Krueger et al., 2020; Ortega et al., 2021; Chan et al., 2023).”

“…al., 2019; Stiennon et al., 2020; Ouyang et al., 2022; Bai et al., 2022a), integration with image processing systems, (Alayrac et al., 2022; OpenAI, 2023b), or integration with other software tools (Nakano et al., 2021; Menick et al., 2022; Collins et al., 2022; Schick et al., 2023; OpenAI, 2023a).”

“Modern LLMs are built on artificial neural networks: They work by computing and updating numeric activation values for internal components that are very loosely modeled on human neurons (Bengio et al., 2017).”

“…up beyond GPT-3 has produced further economically valuable returns: The subsequent GPT-4 model outperforms qualified humans on many graduate and professional exams (OpenAI, 2023b), and its development helped prompt a multi-billion-dollar investment in the company that built it (Capoot, 2023).”

“40,41 Additionally, LLMs can simplify complex medical statistics for better comprehension.”

“Since LLMs work through prompt-based techniques [69], [38], the main way to jailbreak LLMs is to deceive them by asking tricky questions and generating inappropriate content or forbidden scenarios [70].”

“The body of scholarship around LLMs is notably immature (Bowman, 2023), but understanding the implications of these advanced AI tools in relation to academic integrity is crucial.”