Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 105 of the 105 references it cites were found in a registry or an index.

Ecosystem Graphs: The Social Footprint of Foundation Models

Open access
Bommasani, Rishi, Soylu, Dilara, Liao, Thomas, Creel, Kathleen, Liang, Percy · 2023 · Research Square

Abstract

Foundation models (e.g. ChatGPT, StableDiffusion) pervasively influence society, warranting immediate social attention. While the models themselves garner much attention, to accurately characterize their impact, we must consider the broader sociotechnical ecosystem. We propose Ecosystem Graphs as a documentation framework to transparently centralize knowledge of this ecosystem. Ecosystem Graphs is composed of assets (datasets, models, applications) linked together by dependencies that indicate technical (e.g. how Bing relies on GPT-4) and social (e.g. how Microsoft relies on OpenAI) relationships. To supplement the graph structure, each asset is further enriched with fine-grained metadata (e.g. the license or training emissions). We document the ecosystem extensively at this https URL. As of March 16, 2023, we annotate 262 assets (64 datasets, 128 models, 70 applications) from 63 organizations linked by 356 dependencies. We show Ecosystem Graphs functions as a powerful abstraction and interface for achieving the minimum transparency required to address myriad use cases. Therefore, we envision Ecosystem Graphs will be a community-maintained resource that provides value to stakeholders spanning AI researchers, industry professionals, social scientists, auditors and policymakers.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.21203/rs.3.rs-2961271/v1

Cites

105 of 105 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Hubs in the ecosystem. To demonstrate the value of Ecosystem Graphs , we highlight hubs : assets that feature centrally in that many assets directly depend on them. (a) The Pile is an essential resource for training foundation models from a range of institutions (e.g. EleutherAI, Meta, Microsoft, Stanford, Tsinghua, Yandex). (b) P3 is of growing importance as interest in instructiontuning grows, both directly used to train models and as a component in other instruction-tuning datasets. (c) PaLM features centrally in Google's internal foundation models for vision (PALM-E), robotics (PaLM-SayCan), text (FLAN-U-PaLM), reasoning (Minerva), and medicine (Med-PaLM), making the recent announcement of an API for external use especially relevant. (d) The ChatGPT API profoundly accelerates deployment with downstream products spanning a range of industry sectors.
Figure 1 Hubs in the ecosystem. To demonstrate the value of Ecosystem Graphs , we highlight hubs : assets that feature centrally in that many assets directly depend on them. (a) The Pile is an essential resource for training foundation models from a range of institutions (e.g. EleutherAI, Meta, Microsoft, Stanford, Tsinghua, Yandex). (b) P3 is of growing importance as interest in instructiontuning grows, both directly used to train models and as a component in other instruction-tuning datasets. (c) PaLM features centrally in Google's internal foundation models for vision (PALM-E), robotics (PaLM-SayCan), text (FLAN-U-PaLM), reasoning (Minerva), and medicine (Med-PaLM), making the recent announcement of an API for external use especially relevant. (d) The ChatGPT API profoundly accelerates deployment with downstream products spanning a range of industry sectors.CC-BY · From the PDF
Figure 2 Basic foundation model ecosystem. To conceptualize the foundation model ecosystem, we present a simplified pipeline. Image taken from Bommasani et al. [2021].
Figure 2 Basic foundation model ecosystem. To conceptualize the foundation model ecosystem, we present a simplified pipeline. Image taken from Bommasani et al. [2021].CC-BY · From the PDF
Figure 3 Primitive subgraphs in the ecosystem. We spotlight a few 3-node subgraphs to build intuition for Ecosystem Graphs . (a) The standard pipeline: The Jurassic-1 dataset is used to train the Jurassic-1 FM, which is used in the AI21 Playground. (b) A common adaptation process: BLOOM is adapted by fine-tuning on XP3 to produce the instruction-tuned BLOOMZ model. (c) Layering of applications: The GPT-4 API powers Microsoft365 Copilot, which is integrated into Microsoft Word. (d) Dependence on applications: While applications often are the end of pipelines (i.e. sinks in the graph), applications like Google Search both depend on FMs like MUM and support FMs (e.g. through knowledge retrieval) like Sparrow.
Figure 3 Primitive subgraphs in the ecosystem. We spotlight a few 3-node subgraphs to build intuition for Ecosystem Graphs . (a) The standard pipeline: The Jurassic-1 dataset is used to train the Jurassic-1 FM, which is used in the AI21 Playground. (b) A common adaptation process: BLOOM is adapted by fine-tuning on XP3 to produce the instruction-tuned BLOOMZ model. (c) Layering of applications: The GPT-4 API powers Microsoft365 Copilot, which is integrated into Microsoft Word. (d) Dependence on applications: While applications often are the end of pipelines (i.e. sinks in the graph), applications like Google Search both depend on FMs like MUM and support FMs (e.g. through knowledge retrieval) like Sparrow.CC-BY · From the PDF
Figure 4 Graph view for Ecosystem Graphs as of March 16, 2023 (Google's music-related subgraph). We highlight salient foundation models (Noise2Music, AudioLM, MusicLM) as well as the shared and intricate dependencies (e.g. on SoundStream and MuLan). We also observe that music models at present are often language-controlled (e.g. introducing a dependency on LaMDA) that thereby links Google's music models with the more extensive (and more productionized) language models (e.g. LaMDA, PaLM). More generally, beyond these language-mediated dependencies (e.g. outbound dependencies of PaLM), Google's music subgraph is fully contained to the nodes depicted in this figure at present (to our knowledge).
Figure 4 Graph view for Ecosystem Graphs as of March 16, 2023 (Google's music-related subgraph). We highlight salient foundation models (Noise2Music, AudioLM, MusicLM) as well as the shared and intricate dependencies (e.g. on SoundStream and MuLan). We also observe that music models at present are often language-controlled (e.g. introducing a dependency on LaMDA) that thereby links Google's music models with the more extensive (and more productionized) language models (e.g. LaMDA, PaLM). More generally, beyond these language-mediated dependencies (e.g. outbound dependencies of PaLM), Google's music subgraph is fully contained to the nodes depicted in this figure at present (to our knowledge).CC-BY · From the PDF
Image 5
CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 92 references

Research works printed but not linked: 4

  1. [14]14. Nature. The big question. 2021. URL https://www.nature.com/articles/ s42256-021-00395-y. Economist. Huge "foundation models" are turbo-charging ai progress. 2022. URL https://www.economist.com/interactive/briefing/2022/06/11/ huge-foundation-models-are-turbo-charging-ai-progress.
  2. [54]54. Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 610-623, 2021.
  3. [79]79. Kawin Ethayarajh, Yejin Choi, and Swabha Swayamdipta. Understanding dataset difficulty with V-usable information. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 5988-6008. PMLR, 17-23 Jul 2022. URL https://proceedings.mlr.press/v162/ethayarajh22a.html.
  4. [90]90. Rishi Bommasani. Evaluation for change. ArXiv, 2022. URL https://arxiv.org/abs/2212. 11670.

Cited by

See all 15

Metadata

Authors
Bommasani, R., Soylu, D., Liao, T., Creel, K., & Liang, P.
Affiliations
  • Stanford University (US)
  • Northeastern University (US)
Venue
Research Square
Published
2023-06-23
OpenAlex
W4381889417
Cited by
14
References
92
Citations by year
2026: 2 · 2025: 3 · 2024: 5 · 2023: 4
Type
posted content
Language
en
Open access
Locations
  • Research Square (acceptedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Funders
National Science Foundation (DGE-165651 8, 1656518)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
foundation models, AI ecosystem, socio-technical systems, dependency graph, AI transparency, documentation framework, AI licensing, large language models, artificial intelligence applications

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Ecosystem Graphs: The Social Footprint of Foundation Models. https://app.cochat.ai/papers/10.21203/rs.3.rs-2961271/v1
The paper
Bommasani, R., Soylu, D., Liao, T., Creel, K., & Liang, P. (2023). Ecosystem Graphs: The Social Footprint of Foundation Models. Research Square. https://doi.org/10.21203/rs.3.rs-2961271/v1

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.