CoChat
Sign Up
PreprintCited by 2,676Ethics and Social Impacts of AIArtificial Intelligence ApplicationsTopic Modeling

The DOI matches its DataCite record; 1,042 of the 1,094 references it cites were found in a registry or an index, and 8 are not checked yet; none of the 381 sources its claims cite is checked for support yet.

On the Opportunities and Risks of Foundation Models

Open access
Bommasani, Rishi, Hudson, Drew A., Adeli, Ehsan, Altman, Russ, Arora, Simran, von Arx, Sydney, Bernstein, Michael S., Bohg, Jeannette, Bosselut, Antoine, Brunskill, Emma, Brynjolfsson, Erik, Buch, Shyamal, Card, Dallas, Castellon, Rodrigo, Chatterji, Niladri, Chen, Annie, Creel, Kathleen, Davis, Jared Quincy, Demszky, Dora, Donahue, Chris, Doumbouya, Moussa, Durmus, Esin, Ermon, Stefano, Etchemendy, John, Ethayarajh, Kawin, Fei-Fei, Li, Finn, Chelsea, Gale, Trevor, Gillespie, Lauren, Goel, Karan, Goodman, Noah, Grossman, Shelby, Guha, Neel, Hashimoto, Tatsunori, Henderson, Peter, Hewitt, John, Ho, Daniel E., Hong, Jenny, Hsu, Kyle, Huang, Jing, Icard, Thomas, Jain, Saahil, Jurafsky, Dan, Kalluri, Pratyusha, Karamcheti, Siddharth, Keeling, Geoff, Khani, Fereshte, Khattab, Omar, Koh, Pang Wei, Krass, Mark, Krishna, Ranjay, Kuditipudi, Rohith, Kumar, Ananya, Ladhak, Faisal, Lee, Mina, Lee, Tony, Leskovec, Jure, Levent, Isabelle, Li, Xiang Lisa, Li, Xuechen, Ma, Tengyu, Malik, Ali, Manning, Christopher D., Mirchandani, Suvir, Mitchell, Eric, Munyikwa, Zanele, Nair, Suraj, Narayan, Avanika, Narayanan, Deepak, Newman, Ben, Nie, Allen, Niebles, Juan Carlos, Nilforoshan, Hamed, Nyarko, Julian, Ogut, Giray, Orr, Laurel, Papadimitriou, Isabel, Park, Joon Sung, Piech, Chris, Portelance, Eva, Potts, Christopher, Raghunathan, Aditi, Reich, Rob, Ren, Hongyu, Rong, Frieda, Roohani, Yusuf, Ruiz, Camilo, Ryan, Jack, Ré, Christopher, Sadigh, Dorsa, Sagawa, Shiori, Santhanam, Keshav, Shih, Andy, Srinivasan, Krishnan, Tamkin, Alex, Taori, Rohan, Thomas, Armin W., Tramèr, Florian, Wang, Rose E., Wang, William, Wu, Bohan, Wu, Jiajun, Wu, Yuhuai, Xie, Sang Michael, Yasunaga, Michihiro, You, Jiaxuan, Zaharia, Matei, Zhang, Michael, Zhang, Tianyi, Zhang, Xikun, Zhang, Yuhui, Zheng, Lucia, Zhou, Kaitlyn, Liang, Percy·2021·arXiv
DOI 10.48550/arxiv.2108.07258arXiv 2108.07258
Open this paper in CoChat — free accountOpen PDF

Abstract

AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles(e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities,and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature.

Checked 2026-10-06 · 8 references not checked yet

Overview

AI snapshot

Written from a shortened version of the paper’s full text (it is longer than we read).

TL;DR

This report from Stanford's Center for Research on Foundation Models (CRFM) surveys the emerging paradigm of "foundation models" — models trained on broad data with self-supervision that can be adapted to many downstream tasks — and argues their significance lies in two properties: emergence and homogenization. It maps opportunities and risks across capabilities, applications, technology, and society, concluding that foundation models are poorly understood and that deep interdisciplinary collaboration is needed to derisk them.

Key findings

  • Foundation models are defined as models trained on broad data (generally self-supervised at scale) that can be adapted to a wide range of downstream tasks; examples cited include BERT, GPT-3, and CLIP. GPT-3 has 175 billion parameters versus GPT-2's 1.5 billion, and permits in-context learning as an emergent property "neither specifically trained for nor anticipated to arise."
  • Homogenization provides leverage but creates single points of failure: "any flaws in the model are blindly inherited by all adapted models," and almost all state-of-the-art NLP models are now adapted from one of a few foundation models (BERT, RoBERTa, BART, T5).
  • Scale required three ingredients: GPU throughput and memory increased 10× over the last four years; the Transformer architecture; and more training data. Google search, with 4 billion users, now depends on BERT as one of its signals.
  • Domain-specific figures cited: healthcare expenditures account for 17% of US GDP and ~30% of healthcare spending may be wasteful; drug development typically takes over 10 years and costs more than one billion dollars; 80% of clinical trials fail; the US has over 1.3M lawyers and legal services revenues exceed $300B, yet ~86% of low-income individuals with civil legal problems report inadequate or no legal help; US student private education debt has reached $1.6 trillion.
  • On language: there are over 6,000 languages in the world, but multilingual foundation models to date (mBERT, mT5, XLM-R) are each trained on around 100 languages; Fula has over 65 million speakers but few if any NLP resources. An adapted foundation model scored 91.6% on the NY Regents 8th grade science exam in 2019, versus 73.1% for the best 2018 pre-foundation-model system.
  • On law, a GPT-3 exchange shows the model failing to infer that $1M in liquidated damages for a Toyota Corolla is exorbitant, even when given the rule and its application in the prompt.

Methods

  • This is a survey/report, not an empirical study. It was written by over 100 people from different backgrounds at Stanford, organized into 26 sections grouped into four parts: capabilities, applications, technology, and society.
  • A large part of the report is a survey of existing work, unified through many discussions to highlight interdisciplinary connections; the authors describe the writing itself as "an experiment."
  • Sections cover capabilities (language, vision, robotics, reasoning and search, interaction, philosophy of understanding), applications (healthcare/biomedicine, law, education), technology (modeling, training, adaptation, evaluation, systems, data, security and privacy, robustness, AI safety and alignment, theory, interpretability), and society (inequity and fairness, misuse, environment, legality, economics, ethics of scale).

Limitations

  • Stated by the paper: the report "will inevitably be incomplete, especially as the field evolves quickly"; many applications (e.g., natural sciences, music, finance, agriculture) are not included though they are as likely to be affected; and "not all the views expressed in this report are held by all the authors."
  • Stated by the paper: existing foundation models are "very much research prototypes that are poorly understood," with a lack of agreement on basic questions such as when models are "safe" to release; the authors note we "cannot characterize whether the foundation is trustworthy or not."
  • Evident from the text: the report is a broad, non-systematic survey rather than a study with a defined sample or analysis protocol, so its claims are largely qualitative and forward-looking rather than empirically tested within the document itself.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2108.07258

Cites

1042 of 1050 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 of 381 cited sources checked · 0 supported · 0 flagged

Show all 216 claims

  • It may state two separate claims as one. 117
  • Probably not a substantial claim. 262
323 of 323

  • The paper relies on this without support.

    Foundation models will be widely deployed in the near future.

    “Despite the impending widespread deployment of foundation models”

  • The paper relies on this without support.

    Government investment in public infrastructure can close the resource gap for foundation models.

    “One way to close the resource gap is for the government to invest in public infrastructure.”

  • The paper relies on this without support.

    Volunteer computing can be harnessed for training foundation models.

    “Recently, the Learning@home project is attempting to harness volunteer computing for training foundation models [Ryabinin and Gusev 2020].”

  • The authors state this as an assumption.

    Shared structures and patterns between languages can lead to sharing and transfer from high-resource languages to low-resource languages.

    “Joint multilingual training relies on the reasonable assumption that the shared structures and patterns between languages can lead to sharing and transfer from the high-resource languages to the low-resource ones”

  • The paper relies on this without support.

    BLEU does not correlate with causal judgements from humans.

    “parallel ones in natural language processing (e.g., metrics like BLEU do not correlate with causal judgements from humans).”

  • The paper relies on this without support.

    Due to high costs, labeled training data is scarce in legal applications.

    “At the same time, due to its high costs, labeled training data is scarce.”

  • The paper relies on this without support.

    The majority of legal tasks in which reliance on foundation models will be beneficial involve text-based inputs and outputs.

    “Yet, the majority of legal tasks in which reliance on foundation models will be beneficial involve text-based inputs and outputs.”

  • The paper relies on this without support.

    The process of biomedical discovery involves extrapolation.

    “The process of biomedical discovery involves extrapolation.”

  • The paper relies on this without support.

    Foundation models have a strong generative capability, such as coherent text generation in GPT-3.

    “First, foundation models have a strong generative capability (e.g., coherent text generation in GPT-3), which can help generative tasks in biomedical research”

  • The paper relies on this without support.

    It is very costly to label data in criminal justice contexts because annotators must be given access to sensitive data and appropriate background checks are often required.

    “In these contexts, it is very costly to label data since annotators must be given access to sensitive data and appropriate background checks are often required.”

  • The paper relies on this without support.

    The cost of annotating legal data is very high.

    “First, the cost of annotating data is very high.”

  • The paper relies on this without support.

    Certain legal data may be sensitive and cannot be pooled together to train a large language model.

    “Even after labels are obtained, certain data may be sensitive and cannot be pooled together to training a large language model.”

  • The paper relies on this without support.

    Legal decision-making requires context at various scales: knowledge of all historical decisions and standards, knowledge of case law that remains relevant, and knowledge of nuances of individual case.

    “Second, legal decision-making requires context at various scales: knowledge of all historical decisions and standards, knowledge of the case law that remains relevant in the present, and knowledge of the nuances of the individual case at hand.”

  • The paper relies on this without support.

    Specificity and truthfulness are of heightened importance in legal contexts, where imprecise statements can have drastic, unanticipated consequences, and false statements can lead to sanctions against attorneys.

    “Specificity and truthfulness are of heightened important in legal contexts, where imprecise statements can have drastic, unanticipated consequences, and false statements can lead to sanctions against attorneys.”

  • The paper relies on this without support.

    Costly training data for more representative scenarios may be concentrated in the biggest law firms.

    “Costly training data for more representative scenarios may be concentrated in the biggest law firms.”

  • The paper relies on this without support.

    These law firms may have the ability to retain and accumulate data across many cases and clients.

    “These law firms may have the ability to retain and accumulate data across many cases and clients.”

  • The paper relies on this without support.

    Existing work has focused on custom solutions to highly specific tasks for which large amounts of training data has to be collected from scratch.

    “One particular challenge is that existing work has focused on custom solutions to highly specific tasks for which large amounts of training data has to be collected from scratch.”

  • The paper relies on this without support.

    These systems may experience a high degree of 'feedback', where the collected data continually reinforces the model's decisions.

    “Moreover, these systems may experience a high degree of”

  • The paper relies on this without support.

    In the United States, student information is protected by FERPA.

    “For example, in the United States, student information is protected by the Family Education Rights and Privacy Act (FERPA).”

  • The paper relies on this without support.

    These laws and regulations are especially important for children under 13, who have their data privacy and security additionally protected by COPPA.

    “These laws and regulations are especially important for children under 13, who have their data privacy and security additionally protected by the Children's Online Privacy Protection Act.”

  • The paper relies on this without support.

    FERPA limits teachers from sharing personally identifiable student work.

    “Among other things, FERPA limits teachers from sharing personally identifiable student work.”

  • The paper relies on this without support.

    Across classrooms, and across learning tasks in a given domain, there are generalizable patterns in how students arrive at their answers.

    “Across classrooms, and across learning tasks in a given domain, there are generalizable patterns in how students arrive at their answers.”

  • The paper relies on this without support.

    The language used by teachers is often different from the language used by the general population.

    “The language used by teachers is often different from the language used by the general population.”

  • The paper relies on this without support.

    Microsoft's 2016 Twitter bot 'Tay' started generating hate speech within 24 hours of being deployed live.

    “Cautionary examples like Microsoft's 2016 Twitter bot”

  • The paper relies on this without support.

    Humans seem to learn well when presented with real-world analogies and contrasts which may be cross-cutting between their current context and past experiences.

    “Humans seem to learn well when presented with real-world analogies and contrasts which may be cross-cutting between their current context and past experiences.”

  • The paper relies on this without support.

    Multimodality is a key component of intelligence and a crucial factor for thorough and broad comprehension of the world.

    “Essentially, multimodality serves as a key component of intelligence, and is a crucial factor for the development of both thorough and broad comprehension of the world.”

  • The paper relies on this without support.

    Language learning is more effective when occurring in a grounded environment rather than in a vacuum.

    “Concretely, language learning is more effective when occurring in a grounded environment rather than in a vacuum.”

  • The paper relies on this without support.

    Language encourages the emergence of abstractions that link low-level perceptual signals and statistics to semantic concepts of objects, properties, agents and motivations, thereby enriching and elevating visual representations.

    “And inversely, from the vision perspective, language encourages the emergence of abstractions that link between low-level perceptual signals and statistics to semantic concepts of objects, properties, agents and motivations, thereby enriching and elevating visual representations.”

  • The paper relies on this without support.

    The whole can sometimes be greater than the sum of its parts (e.g., red wine is not the same as red onion).

    “In other words, the whole can sometimes be greater than the sum of its parts, where for instance, red wine is not the same as red onion.”

  • The paper relies on this without support.

    Facts' correctness and validity can change over time as the world keeps evolving.

    “it is crucial to note the dynamic nature of knowledge, where facts' correctness and validity can change over time as the world keeps evolving — and what was true or relevant yesterday may not be so tomorrow.”
30 of 30

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Education is assigned the law section number

    Minor

    The applications overview points to §3.2 for education, but the report's section heading identifies §3.3 as education; §3.2 is law.

1 of 1

Citations

Cites

Items 1,001–1,050 of 1,050

Flagged first, then by citations

44 of the 1094 references couldn’t be matched to a known paper.

Research works printed but not linked: 71

  1. §4.6: Data. Data is the lifeblood of foundation models; the training data of these models largely determines what these capabilities these models can acquire. The centrality of data is not unique to foundation models; recent calls for data-centric AI [Press 2021; Ré 2021] indicate the pervasive importance of managing, understanding, and documenting data used to train machine learning models. For foundation models specifically, the current modus operandi is for training data to be selected using unspecified or unclear principles with a general lack of transparency regarding the nature of training data. We believe an alternative approach is needed to re-imagine the data ecosys- tem surrounding foundation models: we draw upon work on data visualization and management to propose a data hub for foundation models. We articulate how this proposal relates to many of the relevant data-centric considerations for foundation models: selection, curation, documentation, access, visualization and inspection, quality assessment, and legal regulation.
  2. From family court to criminal justice and from environmental policy to corporate transactions, the reach of the law is vast. In the United States,31 there are over 1.3M lawyers [American Bar Association 2021] and annual revenues for legal services exceed $300B [MarketLine 2021]. Yet "access to justice" remains far out of reach for most. Legal services can be prohibitively expensive. Roughly 86% of low-income individuals with civil legal problems in the United States, for instance, report receiving inadequate or no legal help [Legal Services Corporation 2017]. Evenwhen counsel is appointed, lawyers might be strained by increasingly large caseloads. Studies have shown that public defenders, for example, are often overworked and underfunded [Lefstein and Spagenberg 2009; Schumm 2012; American Bar Association 2004]. The U.S. Department of Justice reported that in 2007, 73% of county-based public defender offices exceeded the maximum recommended limit of cases received per attorney and 15 of 19 reporting state public defender programs exceeded the maximum recommended limit of felony or misdemeanor cases per attorney [Farole and Langston 2010; Langston and Farole 2010]. Even in a country with one of the highest per capita rates of attorneys, justice can appear out of reach. U.S. President Jimmy Carter once opined, "Ninety percent of our lawyers serve ten percent of our people. We are overlawyered and underrepresented" [Carter 1978]. According to a leading voice in access to justice, technology may provide a path forward [Rhode 2014], a view echoed by many others [Cabral et al. 2012].
  3. 60 Center for Research on Foundation Models (CRFM) can pose insurmountable obstacles to the successful deployment of traditional models. In contrast, their flexibility and capability to learn from few examples suggest that foundation models could be uniquely positioned to address the aforementioned challenges. Throughout this section, foundation models may take as context many modalities as evidence: audio during trial proceedings, video and images during discovery, and text in conducting legal research. Yet, the majority of legal tasks in which reliance on foundation models will be beneficial involve text-based inputs and outputs. As such, we mainly focus on text-based domains while only briefly discussing others. To ground the discussion, Figure 13 describes the stages of a civil lawsuit in the United States and where foundation models might come into play in this process. Figure 14 shows the logic flow required to generate just part of one paragraph of a legal brief, which might serve as a concrete example of a task that foundation models might one day be used for. An Important Consideration. Before proceeding, we note that the ethical, legal, and fairness considerations expanded on in §5.6: ethics, §5.4: legality, and §5.1: fairness are particularly important to examine before using foundation models in an applied legal or government context, as these applications often have important, real-world consequences to those affected [Surden 2020]. Foundation models must also be thoroughly scrutinized before deployment, as discussed in §4.4: evaluation. For example, the legal system places particular emphasis on — and may even mandate — transparency, accountability, and explainability. Consequently, it is questionable whether current models are positioned to solve many of the most pressing, legal problems. Nonetheless, the need to expand and improve access to legal and government services provides a worthy goal for foundation models.
  4. Once a client speaks with an attorney, prior to civil litigation, the attorney may seek to avoid a costly trial. At this stage, they can rely on foundation models to evaluate contracts, review terms of service, find relevant patents, and conduct other pre-litigation processes in order to ensure that their clients are at an advantage [Betts and Jaep 2017; Elwany et al. 2019; Lippi et al. 2019; Lee and Hsiang 2019; Hendrycks et al. 2021c; Hegel et al. 2021]. Notably, recent work has both described the challenges and benefits of using foundation models for contract review [Leivaditi et al. 2020; Hegel et al. 2021; Hendrycks et al. 2021c]. In addition to reviewing and drafting legal documents, client interactions and documents can be translated to reduce costs and barriers to the provision of legal services [Cuéllar 2019]. But translation of legal documents requires precision and an understanding of highly technical language, which makes collecting training data costly. Additionally, translating client statements or trial proceedings often requires an understanding of local dialects and language. This, too, makes it difficult to collect enough ground truth translation data to train on. As a result, traditional supervised methods rarely achieve the level of accuracy required in the legal domain [Vieira et al. 2020]. Foundation models may improve performance in this area over fully supervised mechanisms by adapting quickly in these low-resource contexts.
  5. Long Documents and Narratives. To achieve this goal, the model must be able to read long contexts and produce long narratives. Legal documents tend to be far longer than documents in any other context. The average U.S. Supreme Court opinion contains around 4,700 words,37 a brief on the merits to the Supreme Court can have as many as 15,000 words,38 a law review article often contains 20,000 to 30,000 words,39 parole transcripts can be hundreds of pages long [Bell et al. 2021], and trial records can be even longer. Current foundation models have struggled with such long contexts and outputs (see §4.1: modeling for more discussion).
  6. Alec Radford and Karthik Narasimhan. 2018. Improving Language Understanding by Generative Pre-Training. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. Technical Report. OpenAI. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019). Kira Radinsky. 2015. Data monopolists like Google are threatening the economy. Harvard Business Review 2 (2015). Evani Radiya-Dixit and Florian Tramèr. 2021. Data Poisoning Won't Save You From Facial Recognition. arXiv preprint arXiv:2106.14851 (2021). JackWRae, Sebastian Borgeaud, Trevor Cai, KatieMillican, JordanHoffmann, Francis Song, John Aslanides, SarahHenderson, · arXiv
  7. Roman Ring, Susannah Young, et al. 2021. Scaling Language Models: Methods, Analysis & Insights from Training Gopher. arXiv preprint arXiv:2112.11446 (2021). · arXiv
  8. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683 (2019). · arXiv
  9. Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. 2020. ZeRO: Memory Optimizations toward Training Trillion Parameter Models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, 1–16.
  10. Alexander Ratner, Stephen H. Bach, Henry Ehrenberg, Jason Fries, Sen Wu, and Christopher Ré. 2017. Snorkel: Rapid Training Data Creation with Weak Supervision. Proceedings of the VLDB Endowment (PVLDB) (2017).
  11. Theodoros Rekatsinas, Manas Joglekar, Hector Garcia-Molina, Aditya Parameswaran, and Christopher Ré. 2017b. Slimfast: Guaranteed results for data fusion and source reliability. In Proceedings of the 2017 ACM International Conference on Management of Data. 1399–1414.
  12. Press. Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. 2020. Beyond accuracy: Behavioral testing of
  13. Transactions of the Association for Computational Linguistics (TACL) 8 (2020), 842–866. David Rolnick, Priya L Donti, Lynn H Kaack, Kelly Kochanski, Alexandre Lacoste, Kris Sankaran, Andrew Slavin Ross,
  14. Keshav Santhanam, Siddharth Krishna, Ryota Tomioka, Andrew Fitzgibbon, and Tim Harris. 2021. DistIR: An Intermediate Representation for Optimizing Distributed Neural Networks. In Proceedings of the 1st Workshop on Machine Learning and Systems. 15–23.
  15. Nikunj Saunshi, Sadhika Malladi, and Sanjeev Arora. 2020b. A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks. arXiv preprint arXiv:2010.03648 (2020). · arXiv
  16. Manolis Savva, Abhishek Kadian, Oleksandr Maksymets, Yili Zhao, Erik Wijmans, Bhavana Jain, Julian Straub, Jia Liu, Vladlen Koltun, Jitendra Malik, Devi Parikh, and Dhruv Batra. 2019b. Habitat: A Platform for Embodied AI Research. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).
  17. Londa Schiebinger. 2013. Machine Translation: Analyzing Gender. (2013). http://genderedinnovations.stanford.edu/case- studies/nlp.html#tabs-2 https://doi.org/10.1146/annurev-linguistics-011817-045438 https://doi.org/10.1146/annurev-linguistics-011817-045438 https://arxiv.org/abs/https://doi.org/10.1146/annurev-linguistics-011817-045438 https://www.therobotreport.com/wp-content/uploads/2021/01/2020-Research-Brief-Sanneman-Fourie-Shah.pdf https://www.therobotreport.com/wp-content/uploads/2021/01/2020-Research-Brief-Sanneman-Fourie-Shah.pdf http://proceedings.mlr.press/v48/santoro16.html https://doi.org/10.18653/v1/P19-1163 https://doi.org/10.1086/667722 https://arxiv.org/abs/https://doi.org/10.1086/667722 https://doi.org/10.1146/annurev-soc-071312-145639 https://doi.org/10.1146/annurev-soc-071312-145639 https://arxiv.org/abs/https://doi.org/10.1146/annurev-soc-071312-145639 https://aclanthology.org/2021.eacl-main.20 https://doi.org/10.18653/v1/2021.naacl-main.185 https://doi.org/10.18653/v1/2021.naacl-main.185 http://genderedinnovations.stanford.edu/case-studies/nlp.html#tabs-2 http://genderedinnovations.stanford.edu/case-studies/nlp.html#tabs-2 · DOI
  18. Abigail See, Aneesh Pappu, Rohun Saxena, Akhila Yerukola, and Christopher D. Manning. 2019. Do Massively Pretrained Language Models Make Better Storytellers?. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL). Association for Computational Linguistics, Hong Kong, China, 843–861. https://doi.org/10.18653/v1/ K19-1079 · DOI
  19. Rico Sennrich, B. Haddow, and Alexandra Birch. 2016. Neural Machine Translation of Rare Words with Subword Units. ArXiv abs/1508.07909 (2016). · arXiv
  20. Ali Shafahi, Parsa Saadatpanah, Chen Zhu, Amin Ghiasi, Christoph Studer, David Jacobs, and Tom Goldstein. 2019. Adver- sarially robust transfer learning. arXiv preprint arXiv:1905.08232 (2019). · arXiv
  21. Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. 2017. Outra- geously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538 (2017). · arXiv
  22. Noam Shazeer and Mitchell Stern. 2018. Adafactor: Adaptive Learning Rates with Sublinear Memory Cost. In International Conference on Machine Learning. PMLR, 4596–4604.
  23. Bokui Shen, Fei Xia, Chengshu Li, Roberto Martın-Martın, Linxi Fan, Guanzhi Wang, Claudia D'Arpino, Shyamal Buch, Sanjana Srivastava, Lyne P Tchapmi, Kent Vainio, Li Fei-Fei, and Silvio Savarese. 2021a. iGibson, a Simulation Environment for Interactive Tasks in Large Realistic Scenes. International Conference on Intelligent Robots and Systems (IROS) (2021).
  24. Haichen Shen, Lequn Chen, Yuchen Jin, Liangyu Zhao, Bingyu Kong, Matthai Philipose, Arvind Krishnamurthy, and Ravi Sundaram. 2019. Nexus: A GPU Cluster Engine for Accelerating DNN-Based Video Analysis. In Proceedings of the 27th ACM Symposium on Operating Systems Principles. 322–337.
  25. Ben Shneiderman and Pattie Maes. 1997. Direct manipulation vs. interface agents. In Interactions. ACM. Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. 2019. Megatron-LM:
  26. Nate Soares, Benja Fallenstein, Stuart Armstrong, and Eliezer Yudkowsky. 2015. Corrigibility. In Workshops at the Twenty- Ninth AAAI Conference on Artificial Intelligence.
  27. Irene Solaiman and Christy Dennison. 2021. Process for Adapting LanguageModels to Society (PALMS) with Values-Targeted Datasets. arXiv preprint arXiv:2106.10328 (2021). · arXiv
  28. Steven J. Spencer, Christine Logel, and Paul G. Davies. 2016. Stereotype threat. Annual Review of Psychology 67 (2016), 415–437.
  29. Linguistics (ACL). R. Srivastava, Pranav Shyam, Filipe Wall Mutz, Wojciech Jaśkowski, and J. Schmidhuber. 2019. Training Agents using
  30. Cell 180, 4 (2020), 688–702.
  31. Sainbayar Sukhbaatar, JasonWeston, Rob Fergus, et al. 2015. End-to-endmemory networks. InAdvances in neural information processing systems. 2440–2448.
  32. Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip HS Torr, and Timothy M Hospedales. 2018. Learning to compare: Relation network for few-shot learning. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1199–1208.
  33. Harry Surden. 2020. The ethics of artificial intelligence in law: Basic questions. Forthcoming chapter in Oxford Handbook of Ethics of AI (2020), 19–29.
  34. Abhijit Suresh, Jennifer Jacobs, Vivian Lai, Chenhao Tan, Wayne Ward, James H Martin, and Tamara Sumner. 2021. Using Transformers to Provide Teachers with Personalized Feedback on their Classroom Discourse: The TalkMoves Application. https://ojs.aaai.org/index.php/ICWSM/article/view/15009 https://doi.org/10.2139/ssrn.3489440 https://www.cambridge.org/core/journals/european-review/article/abs/improving-ratings-audit-in-the-british-university-system/FC2EE640C0C44E3DB87C29FB666E9AAB https://www.cambridge.org/core/journals/european-review/article/abs/improving-ratings-audit-in-the-british-university-system/FC2EE640C0C44E3DB87C29FB666E9AAB https://doi.org/10.1145/3313831.3376315 https://doi.org/10.2172/1172643 https://doi.org/10.2172/1172643 https://arxiv.org/abs/1912.04838 https://arxiv.org/abs/1909.11825 · DOI · Paper page
  35. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. In International Conference on Learning Representations (ICLR).
  36. Andrew Szot, Alex Clegg, Eric Undersander, Erik Wijmans, Yili Zhao, John Turner, Noah Maestre, Mustafa Mukadam, Devendra Chaplot, Oleksandr Maksymets, Aaron Gokaslan, Vladimir Vondrus, Sameer Dharur, Franziska Meier, Wojciech Galuba, Angel Chang, Zsolt Kira, Vladlen Koltun, Jitendra Malik, Manolis Savva, and Dhruv Batra. 2021. Habitat 2.0: Training Home Assistants to Rearrange their Habitat. arXiv:2106.14405 [cs.LG] · arXiv
  37. Alex Tamkin, Miles Brundage, Jack Clark, and Deep Ganguli. 2021a. Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models. arXiv:2102.02503 [cs.CL] · arXiv
  38. Hao Tan and Mohit Bansal. 2020. Vokenization: Improving Language Understanding via Contextualized, Visually-Grounded Supervision. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2066–2080.
  39. Navlab. IEEE Transactions on Pattern Analysis and Machine Intelligence 10, 3 (1988), 362–373. Simon Thorpe, Denis Fize, and Catherine Marlot. 1996. Speed of processing in the human visual system. Nature 381, 6582
  40. Christopher Tosh, Akshay Krishnamurthy, and Daniel Hsu. 2021. Contrastive learning, multi-view redundancy, and linear models. In Algorithmic Learning Theory. PMLR, 1179–1206.
  41. Florian Tramèr, Fan Zhang, Ari Juels, Michael K. Reiter, and Thomas Ristenpart. 2016. Stealing machine learning models via prediction APIs. In USENIX Security.
  42. Yao-Hung Hubert Tsai, Yue Wu, Ruslan Salakhutdinov, and Louis-Philippe Morency. 2020. Self-supervised learning from a multi-view perspective. arXiv preprint arXiv:2006.05576 (2020). · arXiv
  43. 13th International Conference, CICM 2020, Bertinoro, Italy, July 26-31, 2020, Proceedings (Lecture Notes in Computer Science, Vol. 12236), Christoph Benzmüller and Bruce R. Miller (Eds.). Springer, 315–323. https://doi.org/10.1007/978-3-030-53518- 6_24 · DOI
  44. Manasi Vartak, Harihar Subramanyam, Wei-En Lee, Srinidhi Viswanathan, Saadiyah Husnoo, Samuel Madden, and Matei Zaharia. 2016. ModelDB: a system for machine learning model management. In Proceedings of the Workshop on Human- In-the-Loop Data Analytics. 1–3.
  45. Pascal Vincent, Hugo Larochelle, Yoshua Bengio, , and Pierre-Antoine Manzagol. 2008. Extracting and Composing Robust Features with Denoising Autoencoders. In International Conference on Machine Learning (ICML).
  46. Soroush Vosoughi, Deb Roy, and Sinan Aral. 2018. The spread of true and false news online. Science 359, 6380 (2018),
  47. Alex Wang, Amapreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2019b. GLUE: A Multi- Task Benchmark and Analysis Platform for Natural Language Understanding. In International Conference on Learning Representations (ICLR).
  48. Zihan Wang, K Karthikeyan, Stephen Mayhew, and Dan Roth. 2020b. Extending Multilingual BERT to Low-Resource Languages. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings. 2649– 2656.
  49. Zirui Wang, Zachary C Lipton, and Yulia Tsvetkov. 2020d. On Negative Interference in Multilingual Language Models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 4438–4450.
  50. Ken Weiner. 2018. Can AI Create True Art? Scientific American (2018). https://github.com/kingoflolz/mesh-transformer-jax https://github.com/kingoflolz/mesh-transformer-jax https://github.com/kingoflolz/mesh-transformer-jax https://proceedings.neurips.cc/paper/2020/hash/d2a27e83d429f0dcae6b937cf440aeb1-Abstract.html http://arxiv.org/abs/2004.13640 https://doi.org/10.18653/v1/W17-3012 https://arxiv.org/abs/2106.09226 · DOI
  51. Jason Weston, Sumit Chopra, and Antoine Bordes. 2014. Memory networks. arXiv preprint arXiv:1410.3916 (2014). Michelle Whirl-Carrillo, Ellen M McDonagh, JM Hebert, Li Gong, K Sangkuhl, CF Thorn, Russ B Altman, and Teri E Klein. · arXiv
  52. Bernard Williams. 1973. In Utilitarianism: For and Against, J.C. Smart and Bernard Williams (Eds.). Cambridge University Press, 82–118.
  53. Langdon Winner. 1980. Do Artifacts Have Politics? Daedalus 109, 1 (1980), 121–136. http://www.jstor.org/stable/20024652 Publisher: The MIT Press.
  54. L. Wittgenstein. 1953. Philosophical Investigations. Blackwell, Oxford. Thomas Wolf, Julien Chaumond, Lysandre Debut, Victor Sanh, Clement Delangue, Anthony Moi, Pierric Cistac, Morgan
  55. Student Feedback. arxiv (2021). https://doi.org/10.1177/0891243215584758 https://arxiv.org/abs/https://doi.org/10.1177/0891243215584758 https://doi.org/10.1145/3330430.3333636 https://doi.org/10.1001/jamadermatol.2019.1735 https://arxiv.org/abs/https://jamanetwork.com/journals/jamadermatology/articlepdf/2740808/jamadermatology_winkler_2019_oi_190038.pdf http://www.jstor.org/stable/20024652 https://doi.org/10.1609/aimag.v34i4.2490 · DOI · Paper page
  56. Shijie Wu and Mark Dredze. 2020. Are All Languages Created Equal in Multilingual BERT?. In Proceedings of the 5th Workshop on Representation Learning for NLP. 120–130. https://aclanthology.org/2020.repl4nlp-1.16
  57. Learning. arXiv:2008.05659 [cs.CV] Michael Xie, Neal Jean, Marshall Burke, David Lobell, and Stefano Ermon. 2016. Transfer Learning from Deep Features for · arXiv
  58. Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. 2021c. An Explanation of In-context Learning as Implicit Bayesian Inference. arXiv preprint arXiv:2111.02080 (2021). · arXiv
  59. Qian Yang, Aaron Steinfeld, Carolyn P Rose, , and John Zimmerman. 2020. Re-examiningWhether, Why, and HowHuman-AI Interaction Is Uniquely Difficult to Design. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.18653/v1/D19-1077 https://aclanthology.org/2020.repl4nlp-1.16 https://openreview.net/forum?id=O6LPudowNQm https://arxiv.org/abs/2008.05659 https://doi.org/10.18653/v1/2021.naacl-main.190 · DOI
  60. Journal of experimental psychology: General 130, 1 (2001), 29.
  61. Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, et al. 2020. Big Bird: Transformers for Longer Sequences.. In NeurIPS.
  62. Elad Ben Zaken, Shauli Ravfogel, and Yoav Goldberg. 2021. BitFit: Simple Parameter-efficient Fine-tuning for Transformer- based Masked Language-models. CoRR abs/2106.10199 (2021). arXiv:2106.10199 https://arxiv.org/abs/2106.10199 · arXiv
  63. Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019b. Defending Against Neural Fake News. In Advances in Neural Information Processing Systems (NeurIPS). 9054–9065.
  64. Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer. 2021. Scaling vision transformers. arXiv preprint arXiv:2106.04560 (2021). · arXiv
  65. T. Zhang and T. Hashimoto. 2020. On the Inductive Bias of Masked Language Modeling: From Statistical to Syntactic Dependencies. In Association for Computational Linguistics (ACL).
  66. Xingliang Zhang and Degan Shu. 2021. Current understanding on the Cambrian Explosion: questions and answers. Paläontologische Zeitschrift 95 (2021), 641–660.
  67. Mengjie Zhao, Tao Lin, Fei Mi, Martin Jaggi, and Hinrich Schütze. 2020b. Masking as an Efficient Alternative to Finetuning for Pretrained Language Models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Online, 2226–2241. https://doi.org/10.18653/v1/2020.emnlp- main.174 · DOI
  68. Nature communications 11, 1 (2020), 1–9.
  69. Jin Peng Zhou, Yuhuai Wu, Colin Li, and Roger Grosse. 2021c. REFACTOR: Learning to Extract Theorems from Proofs. The First Mathematical Reasoning in General Artificial Intelligence Workshop, ICLR 2021 (2021). https://mathai-iclr.github.io/ papers/papers/MATHAI_22_paper.pdf
  70. Kaitlyn Zhou, Kawin Ethayarajh, and Dan Jurafsky. 2021a. Frequency-based Distortions in ContextualizedWord Embeddings. ArXiv abs/2104.08465 (2021). https://arxiv.org/abs/2104.08465 · arXiv
  71. Annette Zimmermann. 2021. Stop Building Bad AI. Boston Review (July 2021). https://bostonreview.net/science-nature/ annette-zimmermann-stop-building-bad-ai

Cited by

Sorted by citations

See all 2,676

Related

See all

Metadata

Authors
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N., Chen, A., Creel, K., Davis, J. Q., Demszky, D., Donahue, C., Doumbouya, M., Durmus, E., Ermon, S., Etchemendy, J., Ethayarajh, K., Fei-Fei, L., Finn, C., Gale, T., Gillespie, L., Goel, K., Goodman, N., Grossman, S., Guha, N., Hashimoto, T., Henderson, P., Hewitt, J., Ho, D. E., Hong, J., Hsu, K., Huang, J., Icard, T., Jain, S., Jurafsky, D., Kalluri, P., Karamcheti, S., Keeling, G., Khani, F., Khattab, O., Koh, P. W., Krass, M., Krishna, R., Kuditipudi, R., Kumar, A., Ladhak, F., Lee, M., Lee, T., Leskovec, J., Levent, I., Li, X. L., Li, X., Ma, T., Malik, A., Manning, C. D., Mirchandani, S., Mitchell, E., Munyikwa, Z., Nair, S., Narayan, A., Narayanan, D., Newman, B., Nie, A., Niebles, J. C., Nilforoshan, H., Nyarko, J., Ogut, G., Orr, L., Papadimitriou, I., Park, J. S., Piech, C., Portelance, E., Potts, C., Raghunathan, A., Reich, R., Ren, H., Rong, F., Roohani, Y., Ruiz, C., Ryan, J., Ré, C., Sadigh, D., Sagawa, S., Santhanam, K., Shih, A., Srinivasan, K., Tamkin, A., Taori, R., Thomas, A. W., Tramèr, F., Wang, R. E., Wang, W., Wu, B., Wu, J., Wu, Y., Xie, S. M., Yasunaga, M., You, J., Zaharia, M., Zhang, M., Zhang, T., Zhang, X., Zhang, Y., Zheng, L., Zhou, K., & Liang, P.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2021
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2108.07258
arXiv
2108.07258
OpenAlex
W3195577433
Semantic Scholar
76e9e2ec3de437ffb30d8b7b629f7fe3e61de5c2
Cited by
2,676
References
1,094
Citations by year
2026: 287 · 2025: 669 · 2024: 816 · 2023: 640 · 2022: 215 · 2021: 46
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
foundation models, large language models, large vision models, transfer learning, emergent capabilities, homogenization, AI risks, societal impact of AI, AI ethics, AI security, artificial intelligence in healthcare, artificial intelligence in law, artificial intelligence in education, environmental impact of AI, economic impact of AI, AI failure modes, interdisciplinary collaboration

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review