On the Opportunities and Risks of Foundation Models

Foundation models have shown promising results for standard vision tasks like image classification and object detection.

“These models have shown promising results for standard tasks in the field, like image classification and object detection”

Big tech companies are on a completely different level in resources than startups and academia, especially in infrastructure, users, and data from market position.

“big tech companies are on a completely different level in terms of resources, especially in terms of the infrastructure, users, and data that come from their market position”

Foundation models are on the rise in computer vision.

“Now, pretraining on web-scale raw data instead of curated datasets, foundation models are on the rise in computer vision [e.g., Radford et al. 2021].”

• cites Radford et al. 2021Not checked yet

  Not checked in the free run.

Adapting a foundation model trained on a broad range of unlabeled data improves the robustness of adapted models across a wide variety of shifts.

“Existing work shows that adapting a foundation model trained on a broad range of unlabeled data improves the robustness of adapted models across a wide variety of shifts.”

Current training of foundation models uses modality-specific objectives that are often chosen heuristically.

“The current status quo for training foundation models involves modality-specific objectives (e.g., masked language modeling [Devlin et al. 2019] for text and SimCLR [Chen et al. 2020c] for images) that are often chosen heuristically.”

• cites Devlin et al. 2019Not checked yet

  Not checked in the free run.

• cites Chen et al. 2020cNot checked yet

  Not checked in the free run.

The model is unable to infer that $1M for a Toyota Corolla is exorbitant.

“The model is unable to infer that $1M for a Toyota Corolla is exorbitant.”

Even if the model is provided with both the abstract rule and its precise translation to the context in the prompt, GPT-3 is still unable to conduct the logical reasoning required.

“Even if the model is provided with both the abstract rule and its precise translation to the context in the prompt — and the application should thus be trivial — GPT-3 is still unable to conduct the logical reasoning required.”

Current foundation models have struggled with long contexts and outputs.

“Current foundation models have struggled with such long contexts and outputs (see §4.1: modeling for more discussion).”

• cites §4.1Not checked yet

  Not checked in the free run.

In-context learning is an emergent property that was neither specifically trained for nor anticipated to arise.

“an emergent property that was neither specifically trained for nor anticipated to arise”

Homogenization is a liability because all AI systems might inherit the same problematic biases of a few foundation models.

“it is also a liability; all AI systems might inherit the same problematic biases of a few foundation models [Bolukbasi et al. 2016; Caliskan et al. 2017; Abid et al. 2021, inter alia]”

• cites Bolukbasi et al. 2016Not checked yet

  Not checked in the free run.

• cites Caliskan et al. 2017Not checked yet

  Not checked in the free run.

• cites Abid et al. 2021Not checked yet

  Not checked in the free run.

We also see the homogenization of actual models across research communities in the form of multimodal models.

“Besides the homogenization of approaches, we also see the homogenization of actual models across research communities in the form of multimodal models”

• cites Luo et al. 2020Not checked yet

  Not checked in the free run.

• cites Kim et al. 2021aNot checked yet

  Not checked in the free run.

• cites Cho et al. 2021Not checked yet

  Not checked in the free run.

The scale of foundation models results in new emergent capabilities.

“their scale results in new emergent capabilities”

Homogenization produces extremely high leverage because any improvements in the foundation models can lead to immediate benefits across all of NLP.

“While this homogenization produces extremely high leverage (any improvements in the foundation models can lead to immediate benefits across all of NLP)”

We are beginning to see a homogenization across research communities.

“We are also beginning to see a homogenization across research communities.”

Similar Transformer-based sequence modeling approaches are now applied to text, images, speech, tabular data, protein sequences, organic molecules, and reinforcement learning.

“similar Transformer-based sequence modeling approaches are now applied to text [Devlin et al. 2019; Radford et al. 2019; Raffel et al. 2019], images [Dosovitskiy et al. 2020; Chen et al. 2020d], speech [Liu et al. 2020d], tabular data [Yin et al. 2020], protein sequences [Rives et al. 2021], organic molecules [Rothchild et al. 2021], and reinforcement learning [Chen et al. 2021b; Janner et al. 202”

• cites Devlin et al. 2019Not checked yet

  Not checked in the free run.

• cites Radford et al. 2019Not checked yet

  Not checked in the free run.

• cites Raffel et al. 2019Not checked yet

  Not checked in the free run.

Foundation models have led to surprising emergence which results from scale.

“Foundation models have also led to surprising emergence which results from scale.”

The gap between private models that industry can train and open community foundation models will likely remain large if not grow.

“the gap between the private models that industry can train and the ones that are open to the community will likely remain large if not grow”

The tech industry has little incentive to devote significant resources to technologies designed for improving the condition of poor and marginalized people.

“the tech industry has little incentive to devote significant resources to technologies designed for improving the condition of poor and marginalized people [Reich et al. 2021]”

• cites Reich et al. 2021Not checked yet

  Not checked in the free run.

Commercial incentives can also lead to market failures and underinvestment in domains where shareholders are unable to capture the value of innovation.

“However, commercial incentives can also lead to market failures and underinvestment in domains where shareholders are unable to capture the value of innovation.”

The direct social impact of foundation models is through the actual deployment of these models, which is governed by proprietary practices on often private data.

“the direct social impact is through the actual deployment of these models, which is governed by proprietary practices on often private data”

Foundation models offer the possibility of closing the reasoning/search gap due to their multi-purpose nature and generative and multimodal capabilities.

“Foundation models offer the possibility of closing this gap: their multi-purpose nature along with their strong generative and multimodal capabilities offer new leverage for controlling the combinatorial explosion inherent to search.”

Existing foundation models are hard to understand.

“existing foundation models are hard to understand”

Volunteer computing for training foundation models is an open technical challenge due to high latency and high bandwidth requirements.

“The high latency connections between nodes and the high bandwidth requirements for training foundation models make this an open technical challenge.”

Commercial incentives can lead companies to ignore social externalities such as the technological displacement of labor, the health of an informational ecosystem required for democracy, the environmental cost of computing resources, and the profit-driven sale of technologies to non-democratic regimes.

“commercial incentives can lead companies to ignore social externalities [Acemoglu 2021; Reich et al. 2021] such as the technological displacement of labor, the health of an informational ecosystem required for democracy, the environmental cost of computing resources, and the profit-driven sale of technologies to non-democratic regimes.”

• cites Acemoglu 2021Not checked yet

  Not checked in the free run.

• cites Reich et al. 2021Not checked yet

  Not checked in the free run.

Robotics faces fundamental challenges due to being anchored to the physical world.

“robotics faces fundamental challenges due to being anchored to the physical world”

AlphaGo has shown that deep neural networks can be effective in guiding the search space.

“existing work such as AlphaGo have already shown that deep neural networks can be effective in guiding the search space”

Google search, which boasts 4 billion users, now depends on foundation models like BERT as one of its signals.

“Google search, which boasts 4 billion users, now depends on foundation models like BERT [Devlin et al. 2019] as one of its signals.”

• cites Devlin et al. 2019Not checked yet

  Not checked in the free run.

Humans operate intuitively even in the most mathematical domains.

“humans are known to operate intuitively even in the most mathematical of domains [Lakoff and Núñez 2000]”

• cites Lakoff and Núñez 2000Not checked yet

  Not checked in the free run.

Foundation models start to roll back the positive trend of increased reproducibility and open science.

“Foundation models start to roll back this positive trend.”

While trained models may be available (e.g., BERT), the actual training of foundation models is unavailable to the vast majority of AI researchers, due to the much higher computational cost and the complex engineering requirements.

“While trained models may be available (e.g., BERT), the actual training of foundation models is unavailable to the vast majority of AI researchers, due to the much higher computational cost and the complex engineering requirements.”

Research on building foundation models themselves has occurred almost exclusively in industry - big tech companies such as Google, Facebook, Microsoft, or Huawei, or startups such as OpenAI or AI21 Labs, though AI2 is a notable exception.

“research on building foundation models themselves has occurred almost exclusively in industry — big tech companies such as Google, Facebook, Microsoft, or Huawei, or startups such as OpenAI or AI21 Labs, though AI2 is a notable exception [Peters et al. 2018; Zellers et al. 2019b].”

• cites Peters et al. 2018Not checked yet

  Not checked in the free run.

• cites Zellers et al. 2019bNot checked yet

  Not checked in the free run.

The principal challenge in developing foundation models for robotics is acquiring sufficient data of the right form conducive to learning.

“The principal challenge in developing new types of foundation models for robotics — different in nature than their language and vision counterparts — is acquiring sufficient data of the right form that is conducive to learning”

Existing foundation models have unexpected failure modes.

“they have unexpected failure modes”

The political economy in which foundation models are designed, developed, and deployed provides an inevitable incentive structure for decision-making at every stage.

“The political economy in which foundations models are designed, developed, and deployed provides an inevitable incentive structure for decision-making at every stage.”

The barrier to entry for developing foundation models will continue to rise, making it difficult for even startups to compete.

“The fundamental centralizing nature of foundation models means that the barrier to entry for developing them will continue to rise, so that even startups, despite their agility, will find it difficult to compete”

• cites Radinsky 2015Not checked yet

  Not checked in the free run.

Non-experts have difficulty distinguishing short-form English text written by GPT-3 from human-written text.

“Clark et al. [2021] demonstrate that non-experts have difficulty distinguishing short-form English text that was written by GPT-3 from that written by humans.”

• cites Clark et al. 2021Not checked yet

  Not checked in the free run.

Existing work demonstrates a variety of security vulnerabilities and privacy risks for foundation models.

“existing work demonstrates a variety of security vulnerabilities (e.g., adversarial triggers to generate undesirable outputs) or privacy risks (e.g., memorization of training data) for these models”

The nature of foundation models introduces new challenges for interpretability research.

“the nature of foundation models (i.e., the wide array of tasks these models are beneficial for and the unexpected emergent properties they acquire) introduces new challenges for interpretability research”

The most impactful feature of foundation models in NLP is their generality and adaptability, not their raw generation abilities.

“However, the feature of foundation models that has been most impactful in NLP is not their raw generation abilities but their surprising generality and adaptability: a single foundation model can be adapted in different ways in order to achieve many linguistic tasks.”

The theory of standard supervised learning is inadequate to fully explain foundation models.

“the theory of standard supervised learning, while relatively mature, is inadequate to fully explain foundation models”

The dominant modern approach for each NLP task is to use a single foundation model and adapt it slightly with small amounts of task-specific annotated data.

“By contrast, the dominant modern approach for performing each task is to use a single foundation model and adapt it slightly using relatively small amounts of annotated data specific to each task (sentiment classification, named entity tagging, translation, summarization) to create an adapted model.”

Standard machine learning produces models that are not robust to distribution shifts.

“A major limitation of standard machine learning is that it produces models that are not robust to distribution shifts”

Machine learning has been shown to contribute to, and potentially amplify, societal inequity in many contexts.

“In many contexts, machine learning has been shown to contribute to, and potentially amplify, societal inequity.”

The generality of foundation models compounds security and privacy concerns, intensifying the risk for function creep or dual use.

“the generality of foundation models compounds these concerns, intensifying the risk for function creep or dual use (i.e., use for unintended purposes)”

Systems are a key bottleneck for scaling in terms of data and model size.

“Systems are a key bottleneck for scaling in terms of data and model size”

Generative foundation models are now the primary vehicle for machine learning for language, including analysis and understanding tasks.

“These generative models now constitute the primary vehicle through which machine learning for language is done — including the analysis and understanding tasks that were once considered prerequisites for generation.”

For the vast majority of NLP tasks, a slightly adapted foundation model greatly outperforms previous task-specific models or pipelines.

“This has proved to be an extremely successful approach: for the vast majority of the tasks described above, a foundation model that is slightly adapted for a task greatly outperforms previous models or pipelines of models that were built specifically to perform that one task.”

Since the introduction of ELMo and BERT in 2018, the field of NLP has become largely centered around using and understanding foundation models.

“Since the introduction of the early foundation models ELMo [Peters et al. 2018] and BERT [Devlin et al. 2019] in 2018, the field of NLP has become largely centered around using and understanding foundation models.”

• cites Peters et al. 2018Not checked yet

  Not checked in the free run.

• cites Devlin et al. 2019Not checked yet

  Not checked in the free run.

Foundation models challenge the ability of standard evaluation paradigms in machine learning because they are one step removed from specific tasks.

“Foundation models challenge the ability of standard evaluation paradigms in machine learning to achieve these goals since they are one step removed from specific tasks.”

The training data of foundation models largely determines what capabilities these models can acquire.

“the training data of these models largely determines what these capabilities these models can acquire”

Until around 2018, generating general-purpose language was considered very difficult and essentially unapproachable except through other linguistic sub-tasks.

“Until around 2018, the problem of generating general-purpose language was considered very difficult and essentially unapproachable except through other linguistic sub-tasks [Paris et al. 2013].”

• cites Paris et al. 2013Not checked yet

  Not checked in the free run.

It is now possible to train highly coherent foundation models with a simple language generation objective.

“Now, it is possible to train highly coherent foundation models with a simple language generation objective”

Human language is grounded to the real world.

“One salient difference between foundation models and human language acquisition is that human language is grounded to the real world [Saxton 2017].”

• cites Saxton 2017Not checked yet

  Not checked in the free run.

Multilingual foundation models remain useful for some multilingual applications, such as adapting them for low-resource languages not in their original training set.

“Nevertheless, they remain useful for some multilingual applications, for example through adapting multilingual models for low-resource languages not in their original training set [Wang et al. 2020b].”

• cites Wang et al. 2020bNot checked yet

  Not checked in the free run.

When a foundation model produces a linguistic construction accurately once, future uses of that construction are not guaranteed to be mostly consistent, especially after a significant domain shift.

“when a foundation model produces a linguistic construction accurately one time there is no guarantee that future uses of that construction will be mostly consistent, especially after a significant domain shift in the subject matter”

• cites Lake and Baroni 2018Not checked yet

  Not checked in the free run.

• cites Kim and Linzen 2020Not checked yet

  Not checked in the free run.

• cites Bahdanau et al. 2018Not checked yet

  Not checked in the free run.

RoBERTa models express abstract syntactic features before usable meaning.

“Zhang et al. [2021] show that RoBERTa models express abstract syntactic features before usable meaning.”

• cites Zhang et al. 2021Not checked yet

  Not checked in the free run.

Foundation models are not blank-slate learners.

“foundation models are not blank-slate learners [Baroni 2021]”

• cites Baroni 2021Not checked yet

  Not checked in the free run.

Improvements have been observed in traditional computer vision tasks, particularly with respect to generalization capability.

“we have observed improvements in traditional computer vision tasks (particularly with respect to generalization capability) [Radford et al. 2021; Ramesh et al. 2021]”

• cites Radford et al. 2021Not checked yet

  Not checked in the free run.

• cites Ramesh et al. 2021Not checked yet

  Not checked in the free run.

It would be infeasible to create a corpus that is balanced and representative in all axes of language variation.

“there are so many axes of language variation that it would be infeasible to create a corpus that is balanced and representative in all regards.”

The potential for foundation models to reduce dependence on explicit annotations may lead to progress on essential cognitive skills like commonsense reasoning.

“However, in the longer-term, the potential for foundation models to reduce dependence on explicit annotations may lead to progress on essential cognitive skills (e.g., commonsense reasoning) which have proven difficult in the current, fully-supervised paradigm [Zellers et al. 2019a; Martin-Martin et al. 2021].”

• cites Zellers et al. 2019aNot checked yet

  Not checked in the free run.

• cites Martin-Martin et al. 2021Not checked yet

  Not checked in the free run.

For most of the over 6,000 languages in the world, available text data is not enough to train a large-scale foundation model.

“For most of the over 6,000 languages in the world, the text data available is not enough to train a large-scale foundation model.”

Foundation models often do not acquire the systematic abstractions expected from humans.

“Foundation models, on the other hand, often do not acquire the systematic abstractions that we expect from humans.”

The future, versatility, and equity of foundation models depend on robustly handling language variation despite unbalanced data.

“The future, versatility, and equity of foundation models all depend on robustly handling language variation despite unbalanced data [e.g., Oren et al. 2019].”

• cites Oren et al. 2019Not checked yet

  Not checked in the free run.

Multilingual models show better performance in languages similar to the highest-resource languages in their training data.

“Multilingual models show better performance in languages that are similar to the highest-resource languages in their training data”

• cites Wang et al. 2020dNot checked yet

  Not checked in the free run.

This transition moved computer vision beyond classic approaches and task-specific feature engineering towards models trained once on large data and adapted for broad tasks.

“This transition marked a new era, where we moved beyond the classic approaches and task-specific feature engineering of earlier days [Lowe 2004; Bay et al. 2006; Rosten and Drummond 2006] towards models that could be trained once over large amounts of data, and then adapted for a broad variety of tasks, such as image recognition, object detection, and image segmentation [Krizhevsky et al. 2012; Sz”

• cites Lowe 2004Not checked yet

  Not checked in the free run.

• cites Bay et al. 2006Not checked yet

  Not checked in the free run.

• cites Rosten and Drummond 2006Not checked yet

  Not checked in the free run.

Multilingual foundation models to date (mBERT, mT5, XLM-R) are each trained on around 100 languages.

“The multilingual foundation models to date (mBERT, mT5, XLM-R) are each trained on around 100 languages [Devlin et al. 2019; Goyal et al. 2021; Xue et al. 2020].”

• cites Devlin et al. 2019Not checked yet

  Not checked in the free run.

• cites Goyal et al. 2021Not checked yet

  Not checked in the free run.

• cites Xue et al. 2020Not checked yet

  Not checked in the free run.

Traditional supervised techniques rely on expensive and carefully-collected labels and annotations, limiting their robustness, generalization and applicability.

“Traditional supervised techniques rely on expensive and carefully-collected labels and annotations, limiting their robustness, generalization and applicability”

Most foundation models used in NLP learn from distributional information of raw, ungrounded text.

“Most foundation models used in NLP, on the other hand, learn from the distributional information of raw, ungrounded text”

Joint multilingual training makes foundation models possible for languages where a stand-alone model could not be trained.

“Joint multilingual training relies on the reasonable assumption that the shared structures and patterns between languages can lead to sharing and transfer from the high-resource languages to the low-resource ones, making foundation models possible for languages where we could not train a stand-alone model.”

• AssumptionThe authors state this as an assumption: Shared structures and patterns between languages can lead to sharing and transfer from high-resource languages to low-resource languages.

Babies and caretakers point to objects during language development.

“For example babies and caretakers point to objects during language development [Colonnesi et al. 2010]”

• cites Colonnesi et al. 2010Not checked yet

  Not checked in the free run.

It is generally agreed that humans learn language in a way that allows them to easily slot new knowledge into existing abstractions and productively create new grammatical sentences.

“it is generally agreed that humans learn language in a way that allows them to easily slot new knowledge into existing abstractions and productively create new grammatical sentences.”

• cites Comrie 1989Not checked yet

  Not checked in the free run.

• cites Chomsky 2014Not checked yet

  Not checked in the free run.

• cites Croft 2001Not checked yet

  Not checked in the free run.

The introduction of ImageNet and supervised pretraining led to a deep learning paradigm shift in computer vision.

“The introduction of ImageNet [Deng et al. 2009] and the advent of supervised pretraining led to a deep learning paradigm shift in computer vision.”

• cites Deng et al. 2009Not checked yet

  Not checked in the free run.

Current multilingual foundation models in raw form, and naive unsupervised multilingual training, may not model the subtleties of languages and language varieties to their full extent.

“Current multilingual foundation models in their raw form, and naive unsupervised multilingual training as a method, may not model the subtleties of languages and language varieties to their full extent.”

Languages in multilingual models compete for model parameters.

“it has been shown that languages in multilingual models compete for model parameters”

• cites Wang et al. 2020dNot checked yet

  Not checked in the free run.

Recent advances in self-supervised learning suggest an alternative route for developing foundation models that use large quantities of raw data to attain contextual understanding of the visual world.

“in contrast, recent advances in self-supervised learning [Chen et al. 2020c; He et al. 2020] suggest an alternative route for the development of foundation models that could make use of large quantities of raw data to attain a contextual understanding of the visual world.”

• cites Chen et al. 2020cNot checked yet

  Not checked in the free run.

• cites He et al. 2020Not checked yet

  Not checked in the free run.

Current capabilities of vision foundation models are early-stage relative to the broader aims of the field.

“Relative to the broader aims of the field, the current capabilities of vision foundation models are currently early-stage (§2.2.1: vision-capabilities)”

Experiments have shown a surprising amount of transfer between and parallel encoding of different languages in multilingual foundation models.

“Experiments using and analyzing multilingual foundation models have shown that there is indeed a surprising amount of transfer between and parallel encoding of the different languages in multilingual foundation models [Wu and Dredze 2019; Choenni and Shutova 2020; Pires et al. 2019; Libovickỳ et al. 2019; Chi et al. 2020; Papadimitriou et al. 2021; Cao et al. 2019].”

• cites Wu and Dredze 2019Not checked yet

  Not checked in the free run.

• cites Choenni and Shutova 2020Not checked yet

  Not checked in the free run.

• cites Pires et al. 2019Not checked yet

  Not checked in the free run.

Transferring visual abilities to machines has proved remarkably challenging.

“For a skill executed effortlessly by even simple living creatures, transferring the same abilities to machines has proved remarkably challenging”

• cites Moravec 1988Not checked yet

  Not checked in the free run.

• cites Thorpe et al. 1996Not checked yet

  Not checked in the free run.

• cites Fei-Fei et al. 2007Not checked yet

  Not checked in the free run.

Babies learn grounded meanings of words that refer to common objects before they learn many other aspects of the linguistic system.

“babies learn the grounded meanings of words that refer to common objects before they learn a lot of the other aspects of the linguistic system [Bergelson and Swingley 2012]”

• cites Bergelson and Swingley 2012Not checked yet

  Not checked in the free run.

Existing foundation models will not suffice for building generalist robots.

“To make progress towards this goal, existing foundation models will not suffice.”

Many of the same underlying bias issues continue to persist in current foundation models.

“Many of the same underlying issues continue to persist in current foundation models [Agarwal et al. 2021].”

• cites Agarwal et al. 2021Not checked yet

  Not checked in the free run.

Low-level computer vision inputs are extremely high-dimensional; a single 1080p frame contains over 2 million pixels.

“relative to word token-level inputs in language, low-level computer vision inputs are extremely high-dimensional: a single 1080p frame contains over 2 million pixels.”

A naïve approach to fully processing every individual pixel is likely prohibitive.

“Understandably, a naïve approach to fully processing every individual pixel is likely prohibitive.”

Foundation models in language have shown initial steps towards modeling longer-term coherence of events.

“Foundation models in language (§2.1: language) have shown initial steps towards modeling longer-term coherence of events”

Standard visual-question answering systems struggle to answer questions requiring commonsense understanding because these questions often require external knowledge beyond pixels alone.

“For instance, standard systems for visual-question answering struggle to answer questions that require commonsense understanding, since these questions often require external knowledge beyond what is present in the pixels alone [Zellers et al. 2019a].”

• cites Zellers et al. 2019aNot checked yet

  Not checked in the free run.

The limitations of the fully supervised paradigm motivate the transition to foundation models because reliance on external supervised annotations constrains the upper bound capability of previous approaches to capture diverse visual inputs scalably, robustly and generally.

“The limitations of this fully supervised paradigm motivate the transition to foundation models: the reliance on external supervised annotations constrains the upper bound capability of previous approaches to capture the diverse spectrum of visual inputs in a scalable, robust and generalizable manner.”

Because robots have the ability to actively and autonomously shape their environment, they should be able to generate targeted unlabeled data at scale.

“Moreover, because robots have the ability to actively and autonomously shape their environment, they should be able to generate targeted unlabeled data at scale.”

The ability to capture long-range temporal correlations and causal coherence in visual input would benefit downstream settings like robotics.

“the analogous ability to capture long-range temporal correlations and causal coherence in visual input would stand to benefit downstream settings like robotics [Dai et al. 2019; Alyamkin et al. 2019; Goel et al. 2020b; Feng et al. 2019, §2.3: robotics].”

• cites Dai et al. 2019Not checked yet

  Not checked in the free run.

• cites Alyamkin et al. 2019Not checked yet

  Not checked in the free run.

• cites Goel et al. 2020bNot checked yet

  Not checked in the free run.

Foundation models have shown early indications of understanding scene and object geometry.

“Foundation models have shown early indications of understanding scene and object geometry [Ramesh et al. 2021].”

• cites Ramesh et al. 2021Not checked yet

  Not checked in the free run.

A necessary first step towards developing generalist robots is building a new type of foundation models for reliable task specification.

“Therefore, a necessary first step towards developing generalist robots is building a new type of foundation models for reliable task specification, i.e., the intuitive and effective communication of task objectives, preferences, and constraints.”

The key stumbling block for robotic foundation models is collecting the right data.

“Yet while these opportunities exist, the key stumbling block is collecting the right data.”

GANs learn to generate visual content of high fidelity, realism and diversity by using a generator and discriminator that supervise one another from image collections alone.

“GANs, for instance, learn to generate visual content of high fidelity, realism and diversity, by featuring two competing networks of a generator and a discriminator that can supervise one another from image collections alone [e.g., Goodfellow et al. 2014; Hudson and Zitnick 2021].”

• cites Goodfellow et al. 2014Not checked yet

  Not checked in the free run.

• cites Hudson and Zitnick 2021Not checked yet

  Not checked in the free run.

Scaling up robot data collection to the size of vision and language datasets remains an open challenge.

“While scaling up robot data collection to the size of vision and language datasets [Deng et al. 2009; Krishna et al. 2017; Raffel et al. 2019; Gao et al. 2020a] remains an open challenge, the increasing scale and quality of robotic datasets suggests they can play an important role in learning foundation models for robotics.”

• cites Deng et al. 2009Not checked yet

  Not checked in the free run.

• cites Krishna et al. 2017Not checked yet

  Not checked in the free run.

• cites Raffel et al. 2019Not checked yet

  Not checked in the free run.

Current foundation models for vision have largely focused on a small subset of modalities, such as RGB images and text, because these are perhaps the most readily accessible.

“Current foundation models for vision have largely focused on a small subset ofmodalities shown in Figure 7 (e.g., datasets of RGB images and text), since these are perhaps the most readily accessible [Changpinyo et al. 2021; Radford et al. 2021].”

• cites Changpinyo et al. 2021Not checked yet

  Not checked in the free run.

• cites Radford et al. 2021Not checked yet

  Not checked in the free run.

The increasing scale and quality of robotic datasets suggests they can play an important role in learning foundation models for robotics.

“While scaling up robot data collection to the size of vision and language datasets [Deng et al. 2009; Krishna et al. 2017; Raffel et al. 2019; Gao et al. 2020a] remains an open challenge, the increasing scale and quality of robotic datasets suggests they can play an important role in learning foundation models for robotics.”

Generated deepfake images and misinformation pose greater risks as the semantic and generative capability of vision foundation models continues to grow.

“generated deepfake images and misinformation pose greater risks as the semantic and generative capability of vision foundation models continues to”

• cites Dolhansky et al. 2020Not checked yet

  Not checked in the free run.

• cites Ramesh et al. 2021Not checked yet

  Not checked in the free run.

By reducing dependence on explicit annotations, foundation models may enable further progress towards higher-order vision goals than previously feasible.

“By reducing the dependence on explicit annotations, foundation models may enable further progress towards these goals than was previously feasible.”

Safety risks from robotic foundation models are expected to be different from their language counterparts because embodied agents are empowered to manipulate and interact with their surroundings directly in the physical world.

“We can expect the safety risks from these models for robotics to be different from their language counterparts given that embodied agents are empowered to manipulate and interact with their surroundings directly in the physical world.”

Since foundation models may be trained on multiple modalities, they can assist in understanding a range of data sources such as images and texts.

“Since the models may be trained on multiple modalities, foundation models can assist in understanding a range of data sources (e.g., images, texts).”

Such flexibility is essential for many of the reasoning challenges discussed, to allow creative generation in domains such as mathematical conjecturing and synthesizing novel programs.

“Such flexibility is essential for many of the reasoning challenges we discussed, to allow creative generation in domains such as mathematical conjecturing [Li et al. 2021b] and synthesizing novel programs [Chen et al. 2021f].”

• cites Li et al. 2021bNot checked yet

  Not checked in the free run.

• cites Chen et al. 2021fNot checked yet

  Not checked in the free run.

Many reasoning problems exhibit similar latent structures.

“As we mentioned in the last section, many reasoning problems exhibit similar latent structures.”

Improving high-level reasoning capabilities is a core challenge for existing foundation models.

“Improving the high-level reasoning capabilities is a core challenge for existing foundation models.”

Grounding representations in other modalities, such as visual or physical, is essential to grasp abstract concepts in reasoning tasks and endow them with concrete meaning.

“First, grounding representations in other modalities, such as visual or physical, are essential to grasp abstract concepts in reasoning tasks and endow them with concrete meaning [Larkin and Simon 1987; Jamnik 2001].”

• cites Larkin and Simon 1987Not checked yet

  Not checked in the free run.

• cites Jamnik 2001Not checked yet

  Not checked in the free run.

The semantic and visual distribution of the real world can be simulated by scanning the real world into a simulation.

“and that the semantic and visual distribution of the real world can be simulated by scanning the real world into a simulation [Chang et al. 2017; Kolve et al. 2017b; Savva et al. 2019a; Szot et al. 2021; Shen et al. 2021a].”

• cites Chang et al. 2017Not checked yet

  Not checked in the free run.

• cites Kolve et al. 2017bNot checked yet

  Not checked in the free run.

• cites Savva et al. 2019aNot checked yet

  Not checked in the free run.

Standard techniques from language modelling, such as pretraining, can greatly improve performance on reasoning tasks.

“It has also been shown that scaling model size significantly improves reasoning capabilities [Polu and Sutskever 2020], and furthermore standard techniques from language modelling, such as pretraining, can also greatly improve performance on these tasks [Rabe et al. 2021; Polu and Sutskever 2020].”

• cites Rabe et al. 2021Not checked yet

  Not checked in the free run.

• cites Polu and Sutskever 2020Not checked yet

  Not checked in the free run.

We still lack general principled approaches in designing self-supervised tasks, as most existing works are tailored to specific problem setups.

“However, we still lack general principled approaches in designing self-supervised tasks, as most of the existing works are tailored to specific problem setups [Yasunaga and Liang 2020; Ren and Leskovec 2020; Han et al. 2021].”

• cites Yasunaga and Liang 2020Not checked yet

  Not checked in the free run.

• cites Ren and Leskovec 2020Not checked yet

  Not checked in the free run.

• cites Han et al. 2021Not checked yet

  Not checked in the free run.

Data-driven methods using neural networks have shown encouraging results, such as defeating the best humans in Go, by exploiting statistical structures and learning useful heuristics.

“More recently, data-driven methods using neural networks have shown encouraging results — e.g., defeating the best humans in Go [Silver et al. 2016], a board game with a much larger space of actions than the classic challenge of chess — by exploiting statistical structures and learning useful heuristics.”

• cites Silver et al. 2016Not checked yet

  Not checked in the free run.

One core safety challenge for learning-based systems is the chicken-and-egg problem of needing to specify system constraints for safety prior to collecting data, after which unforeseen unsafe behaviors requiring additional constraints may emerge.

“One core safety challenge for learning-based systems is the chicken-and-egg problem of needing to specify system constraints for safety prior to collecting data, after which unforeseen unsafe behaviors requiring additional constraints may emerge.”

Effective and general sim-to-real learning of manipulation and locomotion skills remains an open challenge.

“While these are promising steps towards closing the sim-to-real gap, effective and general sim-to-real learning of manipulation and locomotion skills remains an open challenge.”

The unifying framework imposed by a foundation model can transfer and share significant heuristics across tasks, ranging from generalizing low-level techniques to new scenarios to finding meta-techniques that work across numerous kinds of problems.

“We believe that the unifying framework imposed by a foundation model can transfer and share significant heuristics across tasks, ranging from generalizing low-level techniques that work well for one task to new scenarios all the way to directly finding meta-techniques that work well across numerous kinds of problems.”

As one scales up foundation models, the capabilities of capturing such statistical structures also grow immensely.

“As one scales up foundation models, the capabilities of capturing such statistical structures also grow immensely [Polu and Sutskever 2020].”

• cites Polu and Sutskever 2020Not checked yet

  Not checked in the free run.

The vast amount of data, computing resources, and skills needed to create a powerful task-specific model is frequently in conflict with the iterative prototyping process necessary to elicit and satisfy users' needs and values.

“The vast amount of data, computing resources, and skills needed to create a powerful task-specific model is frequently in conflict with the iterative prototyping process necessary to elicit and satisfy users' needs and values [Yang et al. 2016].”

• cites Yang et al. 2016Not checked yet

  Not checked in the free run.

The foundation model training and adaptation framework encourages a separation of concerns, where training learns meta-knowledge such as shared search tree structure between drug retrosynthesis and propositional logic proofs, and adaptation focuses on task-specific vocabulary.

“The foundation model training and adaptation framework encourage a separation of concerns, where foundation model training learns meta-knowledge such as the shared search tree structure between drug retrosynthesis and propositional logic proofs, and the adaptation phase can focus on learning the task specific vocabulary.”

Early learning-based approaches that used a constrained search space to make the problem tractable suffered from the limited kinds of actions the solver could issue.

“To overcome the unbounded search space challenge, researchers first started with a constrained search space to make the problem tractable [Huang et al. 2018; Bansal et al. 2019]. But such approaches suffered from the limited kinds of actions the solver could issue.”

• cites Huang et al. 2018Not checked yet

  Not checked in the free run.

• cites Bansal et al. 2019Not checked yet

  Not checked in the free run.

It is possible that collecting datasets of size comparable to those used in vision and language is insufficient for robotics.

“Given the challenging closed-loop nature of learning control, it is possible that collecting datasets of size comparable to those used in vision and language is insufficient for robotics.”

A major challenge lies in bridging the gap between simulation and the real world, both in the underlying physics and in the semantic distribution of environments and tasks.

“However, a major challenge lies in bridging the gap between simulation and the real world, both in the underlying physics and in the semantic distribution of environments and tasks.”

High-quality annotated data for reasoning problems is scarce and harder to collect than raw images and text.

“Due to the intrinsic difficulty of these problems, high-quality annotated data is scarce and harder to collect compared to raw images and text.”

Scaling model size significantly improves reasoning capabilities.

“It has also been shown that scaling model size significantly improves reasoning capabilities [Polu and Sutskever 2020], and furthermore standard techniques from language modelling, such as pretraining, can also greatly improve performance on these tasks [Rabe et al. 2021; Polu and Sutskever 2020].”

• cites Polu and Sutskever 2020Not checked yet

  Not checked in the free run.

Due to the unbounded search space, it becomes intractable to enumerate all kinds of possibilities.

“Due to the unbounded search space, it becomes intractable to enumerate all kinds of possibilities.”

Since a foundation model is trained across many domains, it can positively transfer meta-knowledge encoded in its weights across tasks and domains.

“In addition, since a foundation model is trained across many domains, it can positively transfer meta-knowledge encoded in the foundation models' weights across tasks and domains [Papadimitriou and Jurafsky 2020; Wu et al. 2021f; Lu et al. 2021a].”

• cites Papadimitriou and Jurafsky 2020Not checked yet

  Not checked in the free run.

• cites Wu et al. 2021fNot checked yet

  Not checked in the free run.

• cites Lu et al. 2021aNot checked yet

  Not checked in the free run.

Besides theorem proving, many real-world problems deal with unbounded search spaces, such as program synthesis, drug discovery, chemical synthesis, computer-aided design, combinatorial optimization, and more.

“Besides theorem proving, many real-world problems deal with unbounded search spaces, such as program synthesis [Gulwani et al. 2017], drug discovery [Drews 2000], chemical synthesis [Segler et al. 2018], computer-aided design [Haigh 1985], combinatorial optimization [Bengio et al. 2021], and more.”

• cites Gulwani et al. 2017Not checked yet

  Not checked in the free run.

• cites Drews 2000Not checked yet

  Not checked in the free run.

• cites Segler et al. 2018Not checked yet

  Not checked in the free run.

The generative capabilities of foundation models are essential for effective reasoning.

“We believe that the generative capabilities of foundation models are essential for effective reasoning.”

Preventable medical errors cause wastes in healthcare.

“preventable medical errors (e.g., hospital readmissions, surgical errors) cause wastes in healthcare [Shrank et al. 2019; Shah et al. 2020].”

• cites Shrank et al. 2019Not checked yet

  Not checked in the free run.

• cites Shah et al. 2020Not checked yet

  Not checked in the free run.

Foundation models have strong adaptation capabilities such as fine-tuning and prompting.

“As foundation models have strong adaptation capabilities (e.g., fine-tuning, prompting [Brown et al. 2020]), they can be efficiently adapted to various individual tasks in healthcare and biomedicine”

• cites Brown et al. 2020Not checked yet

  Not checked in the free run.

As demand for healthcare increases, society faces a serious shortage in healthcare providers.

“as the demand for healthcare increases, the society faces a serious shortage in healthcare providers [Kirch and Petelle 2017].”

• cites Kirch and Petelle 2017Not checked yet

  Not checked in the free run.

The same generalizability and high ceiling that give foundation models their edge can also make them difficult to work with, as they may be even more unpredictable and complex than single-purpose AI models.

“Unfortunately, the same generalizability and high ceiling that give foundation models their edge can also make these models difficult to work with, as they may be even more unpredictable and complex than single-purpose AI models.”

Language-based foundation models' ability to take natural language as input and generalize to many downstream tasks could significantly lower the difficulty threshold for application development by enabling development of sophisticated models without collecting significant amounts of data and training large models from scratch.

“For instance, language-based foundationmodels' ability to take natural language as input, and to generalize to many downstream tasks, could significantly lower the difficulty”

• cites Myers et al. 2000Not checked yet

  Not checked in the free run.

A system might have achieved the mapping that internalism or referentialism require, but we may be unable to show this with behavioral testing.

“Second, a system might have achieved the mapping that these views require, but we may be unable to show this with behavioral testing.”

Both capabilities of foundation models derive from their ability to extract co-occurrence patterns from their training data.

“Both capabilities derive from these models' ability to extract co-occurrence patterns from their training data.”

If a model receives only linguistic inputs, its capacity to learn the mapping might be fundamentally limited in ways that prevent it from learning to refer in the relevant sense.

“If the model receives only linguistic inputs, then its capacity to learn this mapping might be fundamentally limited in ways that prevent it from learning to refer in the relevant sense.”

In an urgent pandemic crisis such as COVID-19, fast diagnosis/screening and automated question answering for patients and the public are vital to reduce disease spread and allocate healthcare resources.

“in an urgent pandemic crisis such as COVID-19, fast diagnosis/screening (e.g., automatic analysis of chest X-ray images) as well as automated question answering for patients (e.g., symptom checking and care) and the public (e.g., disease prevention) are vital to reduce the spread of diseases and allocate healthcare resources for critical patients, saving more lives [Lalmuanawma et al. 2020].”

• cites Lalmuanawma et al. 2020Not checked yet

  Not checked in the free run.

If we take internalism or referentialism as the ultimate target, then behavioral tests will always be at best imperfect as a means of assessing whether understanding has been achieved.

“If we take internalism or referentialism as the ultimate target – our gold standard for what understanding is – then behavioral tests will always be at best imperfect as a means of assessing whether understanding has been achieved.”

Behavioral tests will always have gaps that could allow unsophisticated models to slip through.

“First, behavioral tests will always have gaps that could allow unsophisticated models to slip through.”

Early generations of foundation model-powered software tools for multimedia creation and editing have started to drive a new frontier that empowers even novice content creators to generate high-quality multimedia from coarse, intuitive specifications.

“Already, early generations of what can be considered foundation model-powered software tools for multimedia creation and editing have started to drive a new frontier that empowers even novice content creators to generate high-quality multimedia from coarse, intuitive specifications (e.g., collaborative authoring for writers [Lee et al. 2022], text-to-image generation for digital artists,19”

• cites Lee et al. 2022Not checked yet

  Not checked in the free run.

Recent work has shown that GPT-3 can robustly perform classification tasks in a few-shot or even zero-shot fashion when given an adequate task description in its natural language prompt.

“Recent work has shown that GPT-3, for example, can robustly perform classification tasks in a few-shot or even in zero-shot fashion when given an adequate task description in its natural language prompt [Brown et al. 2020].”

• cites Brown et al. 2020Not checked yet

  Not checked in the free run.

Healthcare providers spend unnecessary time editing electronic health records (EHRs).

“Healthcare providers spend unnecessary time editing electronic heath records (EHRs) [Kocher 2021]”

• cites Kocher 2021Not checked yet

  Not checked in the free run.

We can agree with Bender and Koller's assessment without concluding that foundation models are in general incapable of understanding.

“We can agree with this assessment without concluding that foundation models are in general incapable of understanding.”

Current foundation models' training objectives do not include explainability.

“current foundation models' training objectives do not include explainability, requiring future research in this direction [Linardatos et al. 2021].”

• cites Linardatos et al. 2021Not checked yet

  Not checked in the free run.

Many individuals, especially those with low income, struggle to secure adequate legal representation.

“many individuals, especially those with low income, struggle to secure adequate legal representation [Rhode 2004].”

• cites Rhode 2004Not checked yet

  Not checked in the free run.

Lu et al. [2021a] showed a transformer model trained on natural language could be adapted for other sequence-based tasks such as protein fold prediction.

“Lu et al. [2021a] showed how a transformer model trained on natural language (a data-rich modality) could be adapted for other sequence-based tasks such as protein fold prediction”

• cites Lu et al. [2021a]Not checked yet

  Not checked in the free run.

The ability to leverage existing datasets and extrapolate to new settings is a key machine learning challenge in biomedicine.

“The ability to leverage existing datasets and extrapolate to new settings is a key machine learning challenge in biomedicine [Snell et al. 2017; Ma et al. 2021b].”

• cites Snell et al. 2017Not checked yet

  Not checked in the free run.

• cites Ma et al. 2021bNot checked yet

  Not checked in the free run.

• AssumptionRests on an assumption: The process of biomedical discovery involves extrapolation.

Foundation models have a potential to integrate diverse data modalities in medicine, enabling investigation of biomedical concepts from multiple scales and knowledge sources.

“foundation models have a potential to integrate diverse data modalities in medicine, which enables investigating biomedical concepts (e.g., disease) from multiple scales (using molecule-, patient- and population-level data) and multiple knowledge sources (using imaging, textual and chemical descriptions).”

• cites Lanckriet et al. 2004Not checked yet

  Not checked in the free run.

• cites Aerts et al. 2006Not checked yet

  Not checked in the free run.

• cites Kong et al. 2011Not checked yet

  Not checked in the free run.

Recent work has used machine learning models to identify relevant legal issues contained in a plain-language description of facts presented by a client.

“Recent work has used machine learning models to identify the relevant legal issues contained in a plain-language description of facts presented by a client.33”

• cites [33]Not checked yet

  Not checked in the free run.

Given a set of drugs and a patient genome, foundation models may help predict which drug is likeliest to treat the patient with minimal side effects.

“given a set of drugs and a patient genome, foundation models may help predict which drug is likeliest to treat the patient with minimal side effects [Whirl-Carrillo et al. 2012; Tatonetti et al. 2012; Gerstung et al. 2017; Grinfeld et al. 2018; Adam et al. 2020].”

• cites Whirl-Carrillo et al. 2012Not checked yet

  Not checked in the free run.

• cites Tatonetti et al. 2012Not checked yet

  Not checked in the free run.

• cites Gerstung et al. 2017Not checked yet

  Not checked in the free run.

Identifying the appropriate target and generating a corresponding molecule requires years of expensive wet lab experiments.

“identifying the appropriate target and generating a corresponding molecule requires years of expensive wet lab experiments [Hughes et al. 2011; Schenone et al. 2013; Schneider 2018].”

• cites Hughes et al. 2011Not checked yet

  Not checked in the free run.

• cites Schenone et al. 2013Not checked yet

  Not checked in the free run.

• cites Schneider 2018Not checked yet

  Not checked in the free run.

Current systems for document review are costly because they use supervised learning and active learning to label documents as responsive.

“Current systems are costly because they used supervised learning and active learning to label the documents as responsive [Grossman and Cormack 2010; Oard et al. 2018; Yang et al. 2021].”

• cites Grossman and Cormack 2010Not checked yet

  Not checked in the free run.

• cites Oard et al. 2018Not checked yet

  Not checked in the free run.

• cites Yang et al. 2021Not checked yet

  Not checked in the free run.

Simultaneous solution of related drug discovery problems by a single foundation model may improve the solutions to each of them.

“the simultaneous solution of related drug discovery problems (i.e., target identification, efficacy prediction, side effect prediction, and others) by a single foundation model may improve the solutions to each of them [Ramsundar et al. 2015; Camacho et al. 2018; Duran-Frigola et al. 2020; Huang et al. 2021a].”

• cites Ramsundar et al. 2015Not checked yet

  Not checked in the free run.

• cites Camacho et al. 2018Not checked yet

  Not checked in the free run.

• cites Duran-Frigola et al. 2020Not checked yet

  Not checked in the free run.

Medical data are highly multimodal, with various data types, scales, and styles.

“Medical data are highly multimodal, with various data types (text, image, video, database, molecule), scales (molecule, gene, cell, tissue, patient, population) [Kong et al. 2011; Ruiz et al. 2020], and styles (professional and lay language) [Lavertu and Altman 2019; Li et al. 2019].”

• cites Kong et al. 2011Not checked yet

  Not checked in the free run.

• cites Ruiz et al. 2020Not checked yet

  Not checked in the free run.

• cites Lavertu and Altman 2019Not checked yet

  Not checked in the free run.

Recent work has utilized pretrained models for the recommendation of relevant citations and holding statements when writing legal texts.

“recent work has utilized pretrained models for the recommendation of relevant citations and holding statements when writing legal texts [Zheng et al. 2021; Huang et al. 2021b; Ostendorff et al. 2021].”

• cites Zheng et al. 2021Not checked yet

  Not checked in the free run.

• cites Huang et al. 2021bNot checked yet

  Not checked in the free run.

• cites Ostendorff et al. 2021Not checked yet

  Not checked in the free run.

Traditional supervised methods rarely achieve the level of accuracy required in the legal domain.

“traditional supervised methods rarely achieve the level of accuracy required in the legal domain [Vieira et al. 2020].”

• cites Vieira et al. 2020Not checked yet

  Not checked in the free run.

• AssumptionRests on an assumption: Due to high costs, labeled training data is scarce in legal applications.

Federated learning is one potential solution to keeping raw sensitive data private in the training of foundation models.

“Federated learning is one potential solution to keeping the raw, sensitive data private in the training of foundation models [Chamikara et al. 2021].”

• cites Chamikara et al. 2021Not checked yet

  Not checked in the free run.

Foundation models are uniquely powerful in their ability to integrate multimodal patient data to make an optimal prediction.

“Foundation models are uniquely powerful in their ability to integrate multimodal patient data ranging from the EHR [Rajkomar et al. 2018] to medical imaging [Bera et al. 2019; Ouyang et al. 2020] to drug and molecular measurements [Gottlieb et al. 2011; Ruiz et al. 2020] to make an optimal prediction.”

• cites Rajkomar et al. 2018Not checked yet

  Not checked in the free run.

• cites Bera et al. 2019Not checked yet

  Not checked in the free run.

• cites Ouyang et al. 2020Not checked yet

  Not checked in the free run.

The price of education per student is growing faster than economy-wide costs.

“The price of education per student is growing faster than economy-wide costs [Bowen 2012]”

• cites Bowen 2012Not checked yet

  Not checked in the free run.

Holzenberger et al. [2020] found that simply using a BERT model, even when pretrained on in-domain legal text, was not sufficient to allow the model to deduce and apply the relevant rules.

“However, they found that simply using a BERT model, even when pretrained on in-domain legal text, was not sufficient to allow the model to deduce and apply the relevant rules, a process also referred to as”

• cites Holzenberger et al. [2020]Not checked yet

  Not checked in the free run.

Existing work has leveraged NLP for facilitative moderation in public comment forums.

“existing work has leveraged NLP for facilitative moderation in public comment forums.”

• cites Park et al. 2012Not checked yet

  Not checked in the free run.

Few-shot learning is still in its infancy.

“Some challenges facing foundation models beyond those described in the above example include few-shot learning, which is still in its infancy [Perez et al. 2021].”

• cites Perez et al. 2021Not checked yet

  Not checked in the free run.

Holzenberger et al. [2020] provide a dataset and methodology for evaluating the ability to identify a relevant legal rule and determine how it applies to a new situation in current models based on the tax code.

“Holzenberger et al. [2020] provide a dataset andmethodology for evaluating this ability in current models based on the tax code.”

• cites Holzenberger et al. [2020]Not checked yet

  Not checked in the free run.

Foundation models have already started to boost the performance of some specific flagship tasks in education.

“Foundation models have already started to boost the performance of some specific flagship tasks in education.”

• cites Shen et al. 2021bNot checked yet

  Not checked in the free run.

• cites Wu et al. 2021eNot checked yet

  Not checked in the free run.

Holzenberger et al. [2020] found that domain experts can express the relevant tax law as a prolog solver to determine an individual's tax liability with perfect accuracy.

“Holzenberger et al. [2020] found that domain experts can express the relevant tax law as a prolog solver to determine an individual's tax liability with perfect accuracy.”

• cites Holzenberger et al. [2020]Not checked yet

  Not checked in the free run.

Larger legal benchmark datasets may be necessary to observe further gains from applying transfer learning techniques to foundation models.

“This work suggests that larger legal benchmark datasets may be necessary to observe further gains from applying transfer learning techniques to foundation models.”

Some recent efforts have sought to create large labeled datasets for more challenging legal benchmark tasks through automation or manual annotation by volunteer legal experts.

“Some recent efforts have sought to create large labeled datasets for more challenging legal benchmark tasks through automation [Zheng et al. 2021] or manual annotation by volunteer legal experts [Hendrycks et al. 2021c].”

• cites Zheng et al. 2021Not checked yet

  Not checked in the free run.

• cites Hendrycks et al. 2021cNot checked yet

  Not checked in the free run.

Some gains have been observed from domain-adaptive pretraining on unlabeled legal corpora.

“Some gains have been observed from domain-adaptive pretraining on unlabeled legal corpora.”

• cites Zheng et al. 2021Not checked yet

  Not checked in the free run.

Due to the difficulty and cost of creating large datasets, using this approach to solve every educational task independently is fundamentally limited.

“Due to the difficulty and cost of creating large datasets, using this approach to solve every educational task independently is fundamentally limited.”

• AssumptionRests on an assumption: Existing work has focused on custom solutions to highly specific tasks for which large amounts of training data has to be collected from scratch.

Pretrained models have been used for processing parole hearing transcripts to find instances of anomalous outcomes.

“Pretrained models have been used for processing parole hearing transcripts to find instances of anomalous outcomes [Bell et al. 2021].”

• cites Bell et al. 2021Not checked yet

  Not checked in the free run.

Providing high quality, inclusive education at a large scale poses difficult societal and economic challenges.

“However, providing high quality, inclusive education at a large scale poses difficult societal and economic challenges.”

Current foundation models have a problem of creating fake facts.

“The foundation model also cannot create fake facts in the process, a problem that has come up in current models [Gretz et al. 2020; Zellers et al. 2019b].”

• cites Gretz et al. 2020Not checked yet

  Not checked in the free run.

• cites Zellers et al. 2019bNot checked yet

  Not checked in the free run.

These gains appear to be most pronounced when the pretraining corpus is highly relevant to the downstream task and labeled training data is limited.

“These gains appear to be most pronounced when the pretraining corpus is highly relevant to the downstream task and labeled training data is limited (a setting which is common in the law) [Zheng et al. 2021].”

• cites Zheng et al. 2021Not checked yet

  Not checked in the free run.

Parole transcripts can be hundreds of pages long.

“parole transcripts can be hundreds of pages long [Bell et al. 2021]”

• cites Bell et al. 2021Not checked yet

  Not checked in the free run.

Much available legal data may be unrepresentative.

“Much available legal data may also be unrepresentative.”

Creating benchmark datasets for tasks that are legally meaningful and difficult from an NLP perspective can itself be challenging, as human expert annotation can be costly and automated methods can fail to account for unique aspects of legal text.

“However, creating benchmark datasets for tasks that are legally meaningful and difficult from an NLP perspective can itself be challenging, as human expert annotation can be costly and automated methods that utilize conventional tokenization and sentence segmentation techniques can fail to account for unique aspects of legal text, such as the structure of legal citations [Bommarito et al. 2018; S”

• cites Bommarito et al. 2018Not checked yet

  Not checked in the free run.

• cites Savelka et al. 2017Not checked yet

  Not checked in the free run.

The multiplicative interaction embodied in attention and gating structures offers a more flexible alternative to the rigid fixed-weight computation of MLPs and CNNs, dynamically adapting computation to the input.

“From another perspective, the multiplicative interaction embodied in both attention as well as gating structures (as in LSTMs [Hochreiter and Schmidhuber 1997] or Mixture-of-Experts [Shazeer et al. 2017]) offers a more flexible alternative to the rigid fixed-weight computation of MLPs and CNNs, dynamically adapting the computation to the input at hand.”

• cites Hochreiter and Schmidhuber 1997Not checked yet

  Not checked in the free run.

• cites Shazeer et al. 2017Not checked yet

  Not checked in the free run.

Attention has stronger generality than prior architectures, as it is not strongly tied to a particular task or domain, unlike the local receptive field of convolution or the sequential assumption of recurrent networks.

“A final notable advantage of attention over prior architectures stems from its stronger generality, where it is not strongly tied to a particular task or domain, as is the case for the local receptive field of convolution or the sequential assumption of recurrent networks, both reflecting inherent properties specific to the vision and language modalities respectively.”

Providing effective instruction requires an additional capability: understanding pedagogy.

“However, providing effective instruction requires an additional capability: that of understanding pedagogy [McKenzie 2003].”

• cites McKenzie 2003Not checked yet

  Not checked in the free run.

Some research has also explored effective ways of extracting, from instructors, generative descriptions of how students make mistakes.

“Some research has also explored effective ways of extracting, from instructors, generative descriptions of how students make mistakes [Malik et al. 2021; Gulwani and Singh 2013]”

• cites Malik et al. 2021Not checked yet

  Not checked in the free run.

• cites Gulwani and Singh 2013Not checked yet

  Not checked in the free run.

An algorithm was able to grade an introductory Computer Science midterm at Stanford University, with the same effectiveness as human teaching assistants.

“As a concrete case study, many of these insights were leveraged as core components of an algorithm which was able to grade an introductory Computer Science midterm at Stanford University, with the same effectiveness as human teaching assistants [Wu et al. 2021e].”

• cites Wu et al. 2021eNot checked yet

  Not checked in the free run.

This presents technical challenges beyond what would be faced in typical NLP domain shift settings.

“This presents technical challenges beyond what would be faced in typical NLP domain shift settings (e.g., question answering based on news articles vs. Reddit posts), as the foundation model would need to be fluidly adaptable in terms of its tone and language, and not just the factual content that it generates.”

The feedback task inherently requires a transfer of understanding from external data and experience.

“As such, the feedback task inherently requires a transfer of understanding from external data and experience.”

Great depths play a crucial role in enhancing networks' expressivity, allowing them to form powerful hierarchical and distributed representations that could generalize from training data to new unseen examples.

“Great depths play a crucial role in enhancing networks' expressivity, allowing them to form powerful hierarchical and distributed representations that could generalize from the training data to new unseen examples [He et al. 2016b; Levine et al. 2020].”

• cites He et al. 2016bNot checked yet

  Not checked in the free run.

• cites Levine et al. 2020Not checked yet

  Not checked in the free run.

Different education contexts vary significantly in the kind of language that would be appropriate.

“The adaptation problem above is compounded by the fact that different education contexts vary significantly in the kind of language that would be appropriate: for example, effective instruction in a 5th-grade science class would look quite different from that in a college physics class, much less a college literature class.”

Foundation models, as they currently exist, are directly helpful for understanding a specific subject matter.

“Foundation models, as they currently exist, are directly helpful for the first of these capabilities: understanding a specific subject matter.”

Recent foundation models achieve remarkable performance across a multitude of diverse downstream tasks and applications.

“These and other instances of recent foundation models not only achieve remarkable performance across a multitude of diverse downstream tasks and applications [Rajpurkar et al. 2018; Wang et al. 2019a]”

• cites Rajpurkar et al. 2018Not checked yet

  Not checked in the free run.

• cites Wang et al. 2019aNot checked yet

  Not checked in the free run.

Some research has already started exploring foundation models that can quickly adapt to questions in new subject matter domains.

“Some research in this direction has already started exploring foundation models that can quickly adapt to questions in new subject matter domains [Wu et al. 2021e; Condor et al. 2021].”

• cites Wu et al. 2021eNot checked yet

  Not checked in the free run.

• cites Condor et al. 2021Not checked yet

  Not checked in the free run.

Much of the success of neural networks over the last decade in modeling natural data is owed to the networks' high depths.

“Much of the success of neural networks over the last decade in modeling natural data is owed to the networks' high depths, as could be roughly measured by the number of stacked non-linear layers they are composed of, or the number of computational steps they take during their chain-of-reasoning.”

• cites He et al. 2016bNot checked yet

  Not checked in the free run.

• cites Levine et al. 2020Not checked yet

  Not checked in the free run.

Training an AI system to notice is an achievable goal.

“While difficult, training an AI system to notice is an achievable goal.”

• AssumptionRests on an assumption: Across classrooms, and across learning tasks in a given domain, there are generalizable patterns in how students arrive at their answers.

A salient difference between foundation model training and human language acquisition is that human language is grounded to the real world.

“As the authors point out, a salient difference between foundation model training and human language acquisition is that”

• cites §2.1Not checked yet

  Not checked in the free run.

Subject matter encoding was built on a foundation model that had been adapted on GitHub code and a corresponding small dataset for each question's subject matter.

“In this case, subject matter encoding was built on a foundation model that had been adapted on GitHub code and a corresponding small dataset for each question's subject matter.”

• cites Wu et al. 2021eNot checked yet

  Not checked in the free run.

No single model or algorithm suits best for all cases.

“As the No Free Lunch theorem suggests, there is no single model or algorithm that suits best for all cases [Wolpert and Macready 1997]”

• cites Wolpert and Macready 1997Not checked yet

  Not checked in the free run.

Even for massive courses with millions of students, supervised algorithms barely understand the complex student reasoning behind even short, four-line programs.

“Even for massive courses with millions of students, supervised algorithms barely understand the complex student reasoning behind even short, four-line programs [Malik et al. 2021].”

• cites Malik et al. 2021Not checked yet

  Not checked in the free run.

When learning to provide feedback on short programming questions, a foundation model such as GPT-3 can efficiently understand what fluent code looks like with a few examples.

“For example, when learning to provide feedback on short programming questions, a foundation model such as GPT-3 can efficiently understand what fluent code looks like with a few examples.”

Models such as Module Networks and Mixture-of-Experts exhibit structural modularity and compositional computation, supported by specialization of sub-networks to different operations.

“Models such as Module Networks [Andreas et al. 2016] and Mixture-of-Experts [Shazeer et al. 2017] go further along this direction, exhibiting not only structural modularity, but also compositional computation, supported by the specialization of sub-networks to different operations, in a manner that adapts and tailors the model behavior to the input at hand.”

• cites Andreas et al. 2016Not checked yet

  Not checked in the free run.

• cites Shazeer et al. 2017Not checked yet

  Not checked in the free run.

Separating computation from memory can enhance models' ability to transfer knowledge by applying previously acquired abstract skills to new concrete settings.

“An important design principle that could achieve these desiderata is to separate out computation from memory [Weston et al. 2014; Graves et al. 2016; Hudson and Manning 2018, 2019a] in order to enhance models' ability to transfer knowledge by applying previously acquired abstract skills to new concrete settings.”

• cites Weston et al. 2014Not checked yet

  Not checked in the free run.

• cites Graves et al. 2016Not checked yet

  Not checked in the free run.

• cites Hudson and Manning 2018Not checked yet

  Not checked in the free run.

A supervised learning task like sentiment classification may lead to a more narrow set of capabilities than language modeling.

“In contrast, a supervised learning task like sentiment classification may lead to a more narrow set of capabilities (see §2.1: language).”

Models such as GANformer and Perceiver propose transformers with linear complexity that rely on bipartite or bottleneck attention to improve computational efficiency while maintaining high expressivity.

“Models such as the GANformer [Hudson and Zitnick 2021] and the Perceiver [Jaegle et al. 2021b,a] explore ways to balance these two properties and propose transformers with linear complexity that rely on bipartite or bottleneck attention, so to improve computational efficiency while maintaining high-expressivity.”

• cites Hudson and Zitnick 2021Not checked yet

  Not checked in the free run.

• cites Jaegle et al. 2021bNot checked yet

  Not checked in the free run.

Compositionality is a crucial ingredient of human intelligence, underlying capabilities to plan, reason and learn readily and efficiently from a handful of examples.

“It is a crucial ingredient of human intelligence [Lake et al. 2017], underlying our capabilities to plan, reason and learn readily and efficiently from a handful of examples.”

• cites Lake et al. 2017Not checked yet

  Not checked in the free run.

There is a trade-off between the strong memorization skills offered by retrieval mechanisms and the richer representations learned when there is an information bottleneck.

“However, there is trade-off between the strong memorization skills offered by retrieval mechanisms on the one hand and the richer representations learned when there is an information bottleneck on the other.”

Models that rely on external retrieval mechanisms may not learn to generalize as effectively as bounded, compact and abstract representations.

“Overall, While they certainly have some merits [Guu et al. 2020], models that rely on external retrieval mechanisms may not learn to generalize as effectively as bounded, compact and abstract representations.”

• cites Guu et al. 2020Not checked yet

  Not checked in the free run.

Because broad data lacks external annotations, a major focus for researchers is designing bespoke self-supervised algorithms that leverage the unique structure within each kind of data to produce a training signal for a foundation model.

“Because this data lacks external annotations, a major focus for researchers is designing bespoke self-supervised algorithms that leverage the unique structure within each kind of data to produce a training signal for a foundation model.”

Decoupling explicit and implicit knowledge improves models' trust and reliability by increasing their knowledge provenance.

“improves models' trust and reliability by increasing their knowledge provenance [Cheney et al. 2009]”

• cites Cheney et al. 2009Not checked yet

  Not checked in the free run.

There is a trade-off between efficiency and expressivity: explicit modeling of long-distance interactions through short and direct computation paths improves expressivity but comes at the expense of scalability due to computation entailed by increased connectivity.

“This challenge essentially reflects the trade-off between efficiency and expressivity: where explicit modeling of long-distance interactions through short and direct computation paths improves expressivity on the one hand, but comes at the expense of scalability due to computation entailed by the increased connectivity on the other [Child et al. 2019; Kitaev et al. 2020; Choromanski et al. 2020].”

• cites Child et al. 2019Not checked yet

  Not checked in the free run.

• cites Kitaev et al. 2020Not checked yet

  Not checked in the free run.

• cites Choromanski et al. 2020Not checked yet

  Not checked in the free run.

Decoupling explicit and implicit knowledge mitigates the inflation in models' size and number of parameters needed to store growing quantities of knowledge.

“The separation mitigates the inflation in models' size and number of parameters needed to store the growing quantities of knowledge [Guu et al. 2020]”

• cites Guu et al. 2020Not checked yet

  Not checked in the free run.

The in-context learning abilities of GPT-3 possibly emerge as a by-product of enforcing the network to represent the input sequential data through its bounded memory architecture.

“For instance, the in-context learning abilities of GPT-3 possibly emerge as a by-product of enforcing the network to represent the input sequential data through its bounded memory architecture [Brown et al. 2020].”

• cites Brown et al. 2020Not checked yet

  Not checked in the free run.

Compositionality at the model level, in terms of architectural properties, structure, and degree of modularity, can increase training and inference efficiency of large neural models.

“Compositionality can be reflected at the model level, in terms of its architectural properties, structure, and degree of modularity — which can increase training and inference efficiency of large neural models [Shazeer et al. 2017].”

• cites Shazeer et al. 2017Not checked yet

  Not checked in the free run.

The rise of self-supervised algorithms has made model size and compute resources increasingly salient bottlenecks.

“The rise of self-supervised algorithms has made model size and compute resources increasingly salient bottlenecks [Kaplan et al. 2020; Henighan et al. 2020]”

• cites Kaplan et al. 2020Not checked yet

  Not checked in the free run.

• cites Henighan et al. 2020Not checked yet

  Not checked in the free run.

Models are evaluated not solely on their capabilities but rather on the amount and kind of compute needed to reach those capabilities.

“leading to a shift where models are evaluated not solely on their capabilities but rather on the amount and kind of compute needed to reach those capabilities (§4.4: evaluation).”

The learned representation itself, which emerges over the course of model training and adaptation, can also be compositional.

“But most notably, the learned representation itself, which emerges over the course of the model training and adaptation, can also be compositional [Andreas 2019].”

• cites Andreas 2019Not checked yet

  Not checked in the free run.

Over-reliance on retrieval reduces the opportunities to learn how to represent information in compact and abstract manners, distill key insights and concepts, and separate the wheat from the chaff.

“Indeed, over-reliance on retrieval reduces the opportunities to learn how to represent information in compact and abstract manners, distill key insights and concepts out of the vast amounts of input information the model is exposed too, and, basically, separate the wheat from the chaff.”

These constraints 'bake in' the kinds of capabilities desired when adapting models to downstream tasks.

“At some level, these constraints”

The cost of annotation varies greatly across tasks and languages.

“However, the cost of annotation varies greatly across tasks and languages; for example, annotating MRI data requires expert medical knowledge, whereas labeling sentiment for English texts requires only common sense judgement.”

SSL objectives are highly domain-specific: different methods currently prevail in natural language processing, computer vision, and speech processing.

“Right now, SSL objectives are highly domain-specific: different methods currently prevail in natural language processing, computer vision, and speech processing.”

In text summarization, appending a prompt such as TL;DR to the input article can improve foundation model performance by acting as a task specification.

“For example, in text summarization, appending a prompt such as TL;DR to the input article can improve foundation model performance [Radford et al. 2019] by acting as a task specification for the foundation model.”

• cites Radford et al. 2019Not checked yet

  Not checked in the free run.

Task specialization mostly demands task-specific labeled data as training signals.

“Task specialization mostly demands task-specific labeled data as training signals.”

There is an instance in which the performance gap between full fine-tuning and lightweight adaptation vanishes as the model size increases.

“Lester et al. [2021] shows an instance when the performance gap between full fine-tuning and lightweight adaptation vanishes as the model size increases.”

• cites Lester et al. 2021Not checked yet

  Not checked in the free run.

Approaches that model raw bytes become intractable when training models like transformers whose compute costs grow quadratically with the input size.

“These approaches also become intractable when training models like transformers [Vaswani et al. 2017] whose compute costs grow quadratically with the input size.”

• cites Vaswani et al. 2017Not checked yet

  Not checked in the free run.

Some low-memory procedures such as gradient checkpointing trade off computation and memory, potentially exacerbating the significant energy consumption of foundation models.

“However, some low-memory procedures such as gradient checkpointing [Chen et al. 2016] trade off computation and memory, potentially exacerbating the significant energy consumption of foundation models [Bender et al. 2021].”

• cites Chen et al. 2016Not checked yet

  Not checked in the free run.

• cites Bender et al. 2021Not checked yet

  Not checked in the free run.

Modeling input at the level of raw bytes may cause the model to focus on predicting less semantic aspects of the input, slowing the rate at which it acquires more generally-useful capabilities.

“However, this high dimensionality may cause the model to focus on predicting less semantic aspects of the input,50 slowing the rate at which it acquires more generally-useful capabilities.”

Adapting a foundation model before use is necessary for some applications.

“While foundation models provide a powerful general-purpose engine for processing multi-modal information, adapting a foundation model before use is necessary for some applications.”