Eight Things to Know about Large Language Models

“…translation of ‘cat’ in French is‘ ”, such that a typical continuation of the text should represent a completion of the intended task (Radford et al., 2019; Raffel et al., 2020). match high-quality human demonstrations on the task (Radford et al., 2018; Devlin et al., 2019; Ouyang et al., 2022).”

“Large language models (LLMs, e.g. GPT-3, PALM, LLaMA, and GPT-4; Brown et al., 2020; Chowdhery et al., 2022; Touvron et al., 2023; OpenAI, 2023b) and products built on them, such as ChatGPT, have recently prompted an enormous amount of attention from journalists, (Klein, 2023; Perrigo, 2023; Oliver, 2023), policymaker…”

“This is analogous to the techniques used to produce superhuman performance at games like Go (Silver et al., 2016).”

“Scaling law results (Kaplan et al., 2020; Brown et al., 2020; Hoffmann et al., 2022) have been a major driving factor in the recent surge of research and investment into LLMs (Ganguli et al., 2022a).”

“…arise with the development of deployment of large-scale AI systems, including issues around environmental impacts, access, mis-use, privacy, safety, and the concentration of power (Amodei et al., 2016; Bender et al., 2021; Bommasani et al., 2021; Birhane et al., 2022; Weidinger et al., 2022, i.a.).”

“Large language models (LLMs, e.g. GPT-3, PALM, LLaMA, and GPT-4; Brown et al., 2020; Chowdhery et al., 2022; Touvron et al., 2023; OpenAI, 2023b) and products built on them, such as ChatGPT, have recently prompted an enormous amount of attention from journalists, (Klein, 2023; Perrigo, 2023; Oliver, 2023), policymaker…”

“• Misuse: LLMs, if not properly calibrated or deployed by malicious actors, can generate content that disrespects, misunderstands, or illegitimately appropriates information [36, 51].”

“In Section 5.1 we examine whether a model can accurately target a specific accuracy score (calibration), and in Section 5.2 we examine whether a model can consistently emulate a lower capability profile, e.g., that of a weaker model (emulation).”

“In this case, we are then unable to interpret the inner workings of LLMs, and “any attempt at a precise explanation of an LLM’s behavior is doomed to be too complex for any human to understand” (Bowman, 2023).”

“OpenAI’s models are not only proprietary but also too complex for even those within the company to fully understand (Bowman, 2023).”

“These models, including GPT-4, FLAN-T5, and PaLM-2, have demonstrated exceptional performance in understanding and generating human-like text [10], solving complex language tasks [11], and integrating multimodal information [12,13,14,15].”

“"AI" is a shorthand term for a vast array of interconnecting, and sometimes, competing, technologies, protocols, and methodologies (Bowman, 2023).”

“While ZTNs focus on self-optimizing network architectures, LLMs introduce novel approaches to natural language understanding and generation [9].”

“Yet , the origins, reliability, and veracity of AI-generated content remain deeply uncertain (Bender et al., 2021; Bowman, 2024).”

“However, challenges such as interpretability, ethical considerations, and the substantial computational resources required for training and deploying these models remain significant [Bubeck et al., 2023, Bommasani et al., 2021].”

“It is important to understand different facts about the foundation LLMs models [93]:”

“To address this gap, this paper presents a diagnostic case study that dissects the systemic risks inherent in LLM-based agentic workflows.”

“Consequently, comprehensive evaluation of LLM capabilities is essential for identifying promising research directions and optimizing resource allocation, particularly given the substantial computational and energy requirements of AI development [46, 5].”

“Safety risks and potential dangers arising from the deployment of LLMs are increasingly studied in the literature (Phuong et al., 2024; Shevlane et al., 2023; Bengio et al., 2023; Bowman, 2023).”

“10 10 For more on the state-of-the-art, see (Bowman 2023; Zhao et al. 2023).”

“73 Weighing risks for their severity will be increasingly difficult, due to the variability of risks across a broad spectrum based on the model ’ s capabilities.”

“Rather, it generates a response text by continually predicting which token should be next in the sentence it is constructing [14].”

“…Chan et al. , 2022; Elhage et al. , 2021; Lovering y Pavlick, 2022)–, ninguna técnica que permita comprender completamente la forma en que el modelo efectúa estas predicciones o el tipo de conocimiento, razonamiento u objetivos subyacentes del modelo cuando produce un resultado (Bowman, 2023).”

“They make it very difficult to ‘explain’ in a transparent way what is happening inside an LLM. Bowman (2023) commented that “As of early 2023, there is no technique that would allow us to lay out in any satisfactory way what kinds of knowledge, reasoning, or goals a model is using when it produces…”

“LLMs such as OpenAI’s ChatGPT, Anthropic’s Claude, Meta’s Llama, and Google’s Gemini and Gemma models have received a great deal of attention recently, as they show potential to achieve advances in multiple fields and applications (Bommasani et al., 2021; Bowman, 2023).”

“They furthermore claim that reasoning emerges spontaneously from the training procedures and Wei et al. [2022] and Bowman [2023] claim that certain behaviors emerge unpredictably.”

“For a comprehensive evaluation, we select four datasets: AWA2 (Xian et al., 2017), CUB (Wah et al., 2011), FGVCAircraft (Maji et al., 2013), and ImageNet + , spanning natural and human-made artifacts, coarse and fine-grained categories, and varying scales ( 40 ≤ | S | ≤ 1000 ).”

“Consequently, the accurate and efficient fulfillments of task-oriented queries have become an essential benchmark for satisfactory, efficient, or delightful human-computer interactions in the era of large language models (LLMs) [8][16] and GenAI.”

“Understanding the evolution of LLMs provides crucial context for assessing their current capabilities and anticipating future developments Bowman, 2023; Chang et al., 2023; Clusmann et al., 2023; Fan et al., 2023; Hadi et al., 2023; Hou et al., 2023; Thirunavukarasu et al., 2023; J. Wu et al.,…”

“…reasons including the known shortcomings of LLMs such as privacy concerns, ethical implications, as well as the crucial requirement in the MH domain for machine learning model outputs to be theoretically-grounded and explainable in view of potential risk to the target care-seekers [10, 18, 25].”

“Scalable over-sight [24] addresses trust and verification in multi-tenant AI systems.”

“FAI-C-ST results align with a growing body of research demonstrating that large language models do not operate free of substantive normative commitments [Liang et all, 2022], [Bowman et al, 2023].”

“Large language models (LLMs) have demonstrated remarkable capabilities in open-domain question answering [1-4] .”

“Several recent works argue that AI benchmarking would benefit from similar principles (Wang et al., 2023; Raji et al., 2021), while others critique how benchmarks are built, saturated, and deployed (Kiela et al., 2021; Bowman, 2023; Dehghani et al., 2021).”

“From their structure and training (Bowman 2023 ; Zhao et al. 2023 ), we know that LLMs capture language by processing large amounts of data and by learning statistical regularities in human text corpora.”

“Avec de telles masses d’informations ingurgitées, nous pouvons estimer que ces modèles sont entraînés sur des volumes de connaissances paraissant dépasser les capacités mémorielles humaines (Bowman (2023)), laissant penser à tort qu’ils auraient réponse à tout.”

“Started by small PLMs such as GPT and BERT in 2018, training LLMs has received significant attention of research and investment from both industry and academia (Bowman, 2023).”

“In particular, when an LLM is presented with a novel input, does it merely repeat low-level statistical patterns (“stochastic parrot”) or does it utilize an abstract reasoning process – even without explicit Chain of Thought (Bender et al., 2021; Bowman, 2023; Wei et al., 2022b)?”

“Such alignment will improve monitorability of LLMs, enabling more reliable interpretation of model behavior and fostering greater trust in high-stakes applications (Bowman, 2023; Anwar et al., 2024).”

“This challenge is compounded by the black-box nature of modern models, which make their outputs challenging to predict [Bowman, 2024].”

“Bowman (2024) explained that LLMs still have limitations in reasoning, factual accuracy, reliability, and context interpretation, as models depend on language distribution patterns and training data.”

“These characteristics of LLMs make retraining or fine-tuning them with new data computationally expensive and resource-intensive (Bowman, 2023; Chen et al., 2021).”

“…design of the technology itself: GenAI-assisted tools are trained on a vast amount of publicly available data harvested from the Internet (UNESCO, 2023, pp. 9–11) and recycled without proper attribution or a traceable pathway that can help identify the provenance of individual information items…”

“Recent advances in large-scale deep learning—particularly transformer-based architectures have led to the development of models exhibiting emergent cognitive capabilities that were neither explicitly programmed nor anticipated at smaller scales [23, 24].”

“39 O’Keefe et al., 2025; Bowman, 2023, point 4.”

“The more general question is whether a model trained only on natural language can develop representations of concepts grounded in the real world (Bowman, 2024; Naveed et al., 2023).”

“However, Bowman [12] emphasizes that straightforward attempts to manage hallucination might fail silently, making the models appear more trustworthy than they are.”

“Generative AI [3], large language models (LLMs) [4], and text embeddings are among the latest achievements in NLP. Today, the research in textual IR revolves largely around the combination of Machine Learning (ML) and NLP for solving IR tasks such as question answering [5], [6], text retrieval [7]…”

“Researchers are actively exploring methods to align LLMs with ethical standards and mitigate these risks, ensuring their safe and responsible deployment [20,21].”

“Current discussions of Large Language Models are one example where the technology works for what it was built for without its designers fully understanding how or why it works (Bowman 2023).”

“While no method fully reveals the knowledge, reasoning, or concepts a model uses to generate outputs [100], we argue that their inefficiency in these tasks stems largely from limited competence in discerning semantic similarity between events [101] and evaluating key phrases [102].”