The DOI matches its Crossref record; 152 of the 153 references it cites were found in a registry or an index, and 1 are not checked yet; 62 of the 158 sources its claims cite were checked for support.
Data Governance in the Age of Large-Scale Data-Driven Language Technology
Open accessAbstract
The recent emergence and adoption of Machine Learning technology, and specifically of Large Language Models, has drawn attention to the need for systematic and transparent management of language data. This work proposes an approach to global language data governance that attempts to organize data management amongst stakeholders, values, and rights. Our proposal is informed by prior work on distributed governance that accounts for human values and grounded by an international research collaboration that brings together researchers and practitioners from 60 countries. The framework we present is a multi-party international governance structure focused on language data, and incorporating technical and organizational tools needed to support its work.
Verified 2026-10-06 · 1 reference not checked yet
Overview
CoChat Summary
TL;DR
This paper proposes a multi-party Data Stewardship Organization (DSO) for governing large-scale language data, grounded in eight values and six actor roles. It is a conceptual governance framework developed through a year-long workshop with over 1000 participants from 60 countries, informed by distributed governance, legal, sociolinguistic, and ML data-management literature.
Key findings
- The proposed DSO organizes language-data governance around six actor roles: Data Rights-holders, Data Providers, Data Hosts, Data Modelers, Data Stewardship Organization, and Data Helpers (Table 4).
- The framework is guided by eight values: inclusion/representation/non-discrimination, autonomy/consent/contestation, privacy, just rewards, licensing/attribution, local knowledge, participation, and beneficence (Table 1).
- The paper argues existing ML data management is insufficient: centralized repositories give the governing organization full say; public repositories rely on self-governance by submitters; open data initiatives conflict with removal/update needs; Wikimedia offers a relevant distributed-governance example.
- It proposes contractual flow-down, non-dissemination clauses, contestation/removal processes, a license taxonomy, and a DSO standard Data Provider-Host agreement (Appendix B) as mechanisms to support the values.
Methods
- Conceptual/position paper developed through a year-long, distributed collaborative workshop on Large Language Models with over 1000 participants from 60 countries; the data governance working group grew from around 10 to 50+ participants.
- The analysis reviews prior work on distributed governance, sociolinguistics, legal frameworks (property, privacy, user rights), ML/NLP data management, and Wikimedia, then synthesizes a proposed governance structure.
- No empirical experiment or quantitative analysis is reported; the work is qualitative synthesis and design, with appendices documenting value-crafting and a standard Data Provider-Host agreement.
Limitations
- Paper-stated: values expressed by workshop participants may prioritize their interests and exclude other language users; Human Rights grounding is limited by diverse philosophical foundations and its historical individual-state focus; governance faces a trade-off between harmonization and independence; legal agreements are difficult to develop.
- Paper-stated: the DSO is not a direct party to agreements and relies on fallback/default licenses and interoperability; contestation and deletion depend on non-dissemination and technical/legal maturity.
- Evident from the text: the proposal is not implemented or empirically evaluated, so no outcome data on effectiveness or adoption are provided.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
31 Valid · 11 Not valid · 34 Not checked · 57 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, whereas recent advances in modeling and hardware have increased the data training capacity of LLMs, increasing from Wikipedia-scale corpora to close to three orders of magnitude more”
What the sources say
- Partly supportedcites Hoffmann et al. [71] [71] Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and L. Sifre. 2022. Training Compute-Optimal Large Language Models. ArXiv abs/2203.15556 (2022). - sourceExperimental — accuracy not yet measured
The Introduction (p. 2) reports LLMs trained on roughly 300 billion tokens and Chinchilla on 1.4 trillion, supporting the point that LLMs can train on very large datasets. Appendix A (p. 22) lists Wikipedia as one small component of Chinchilla’s dataset, but the paper does not establish a historical increase from Wikipedia-scale corpora to nearly three orders of magnitude more data, or attribute such an increase to advances in modeling and hardware. (LLM, conf=0.82)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“These practical applications of language technology are increasingly reliant on approaches based on trained Large Language Models (LLMs) [23, 41, 104, 111], whereby models are first exposed to as large and varied a collection of language data as possible with the aim of extracting 'general' properties of a language of interest.”
What the sources say
- Supportedcites [23] [23] Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020). - source
Section 2.2 (p. 8) describes training on a very large mixture of text datasets and adding corpora to increase its diversity. The Introduction (p. 3) says unsupervised pre-training develops broad skills and pattern-recognition abilities, supporting the claim's account of learning general language capabilities. The paper also emphasizes data quality rather than simply maximizing quantity. (LLM, conf=0.90)
- Not checked — no full text of the sourcecites [41] [41] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers) , Jill Burstein, Christy Doran, and Thamar Solorio (Eds.). Association for Computational Linguistics, 4171-4186. https://doi.org/10.18653/v1/n19-1423 - source
- Partly supportedcites [104] [104] Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep Contextualized Word Representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers) , Marilyn A. Walker, Heng Ji, and Amanda Stent (Eds.). Association for Computational Linguistics, 2227-2237. https://doi.org/10.18653/v1/n18-1202 - sourceExperimental — accuracy not yet measured
The Introduction describes a bidirectional language model trained on a large text corpus and says its representations capture aspects of syntax and meaning. Related work specifies a corpus of approximately 30 million sentences. The paper does not establish that training data should be as large and varied as possible, or make that claim about LLM approaches generally. (LLM, conf=0.84)
- Not matchedcites [111] [111] Alec Radford and Karthik Narasimhan. 2018. Improving Language Understanding by Generative Pre-Training. -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“As such, devising a governance structure based solely on values expressed by our participants runs the risk of prioritizing their interests and excluding visions that may be more relevant to other language users [65].”
What the sources say
- Supportedcites [65] [65] Daniel Greene, Anna Lauren Hoffmann, and Luke Stark. 2019. Better, Nicer, Clearer, Fairer: A Critical Assessment of the Movement for Ethical Artificial Intelligence and Machine Learning. In HICSS . - source
Section 5.2, “Expert oversight” (page not provided), says ethical-AI statements give experts a narrow role in deciding whose concerns count while treating users and the public differently. Section 3.1, “Values in technology design” (page not provided), stresses considering alternative courses of action and who counts in design. The citing authors apply that critique to their own participants and other language users; the source supports the underlying risk, not a finding about those particular groups. (LLM, conf=0.86)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“They note that most of these focus on endogenous notions of technical performance and novelty, and leave out considerations of broader context and impact that are necessary to shaping a governance effort.”
What the sources say
- Supportedcites [18] [18] Abeba Birhane, Pratyusha Kalluri, Dallas Card, William Agnew, Ravit Dotan, and Michelle Bao. 2021. The values encoded in machine learning research. arXiv preprint arXiv:2106.15590 (2021). - source
The Abstract reports that the 100 highly cited ML papers most frequently emphasized values including performance and novelty, while few connected their projects to societal needs or discussed negative consequences. The Conclusion likewise says these papers prioritized technical values while neglecting broader social needs and potential harms. The reference to shaping a governance effort is the citing authors’ framing, rather than a separate finding attributed to the source. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Unsurprisingly, gendered and racial disparities have been documented in a number of language technologies [37, 79, 141]”
What the sources say
- Partly supportedcites [37] [37] Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2019. Plug and play language models: Asimple approach to controlled text generation. arXiv preprint arXiv:1912.02164 (2019). - sourceExperimental — accuracy not yet measured
Section 4.4, “Language Detoxification” (page unavailable), says language models reflect biases and reports racist GPT-2 output and toxicity measurements for prompts referring to racial or ethnic groups. It does not document gender disparities or disparities across a number of language technologies, so it supports only part of the cited claim. (LLM, conf=0.85)
- Supportedcites [79] [79] Allison Koenecke, Andrew Nam, Emily Lake, Joe Nudell, Minnie Quartey, Zion Mengesha, Connor Toups, John R Rickford, Dan Jurafsky, and Sharad Goel. 2020. Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences 117, 14 (2020), 7684-7689. - source
The Results report racial differences in word error rates across five commercial speech-recognition systems and higher error rates for male than female speakers in the sample. This supports the cited statement as an example of gendered and racial disparities in language technologies, specifically speech recognition; the paper does not establish disparities across all types of language technology. (LLM, conf=0.89)
- Partly supportedcites [141] [141] Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, and Emily Dinan. 2021. Bot-Adversarial Dialogue for Safe Conversational Agents. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . Association for Computational Linguistics, Online, 2950-2968. https://doi.org/10.18653/v1/2021.naacl-main.235 - sourceExperimental — accuracy not yet measured
The Introduction says dialogue models can learn biased language, and §7 Ethical Considerations reports racial bias in hate-speech datasets. The paper does not document gendered disparities or establish disparities across a number of language technologies; its mention of gender in Appendix A.2 defines a hate-speech category, not a measured disparity. (LLM, conf=0.83)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
19 more claims · 5 with their cited sources checked
Create a free account to see all 133 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 88
- It may state two separate claims as one. 45
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, these models also raise concerns about misinformation, privacy, biases in the training data, and potential for misuse3,7–10.”
From The future landscape of large language models in medicine (2023) · cites this paper as [8]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“One approach is to carefully curate the dataset on which the model is trained 187,188 .”
From Machine culture (2023) · cites this paper as [188]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Each stakeholder brings perspectives that may specify different fairness constraints for the same application and setting.”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Jernite et al. (2022)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ 3. PURPOSE, RIGHTS GRANTED & SCOPE · p. 19
“The Data Provider grants to the Data Host a non-exclusive, non-transferable, non-sublicensable, irrevocable, perpetual, royalty-free and worldwide license to use (that is access, store, prepare, process, label and/or share) the agreed upon Data”
§ 9. TERM AND TERMINATION · p. 22
“Upon termination of this Agreement for any reason, the Data Host and Data Provider cease to use the Data and Processed Datasets and for the purposes set out in this Agreement within 14 days and upon that delete the Data and Processed Datasets immediately”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 15. SURVIVAL · p. 22
“10 (Term and Termination)”
§ 10. FORCE MAJEURE · p. 22
“## 10. FORCE MAJEURE”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 15. SURVIVAL · p. 22
“15 (Governing Law)”
§ 15. SURVIVAL · p. 22
“## 15. SURVIVAL”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 15. SURVIVAL · p. 22
“16 (Survival)”
§ 16. SEVERABILITY · p. 22
“## 16. SEVERABILITY”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 15. SURVIVAL · p. 22
“12 (Confidentiality)”
§ 12. ENTIRE AGREEMENT · p. 22
“## 12. ENTIRE AGREEMENT”
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 2×
“These practical applications of language technology are increasingly reliant on approaches based on trained Large Language Models (LLMs) [23, 41, 104, 111], whereby models are first exposed to as large and varied a collection of language data as possible with the aim of extracting “general” properties of a language of…”
BackgroundUCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,410
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective
Haraway, D. (1988) · Feminist Studies · doi:10.2307/3178066 · Cited by 18,364
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,690
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
An Integrative Framework for Collaborative Governance
Emerson, K., Nabatchi, T., & Balogh, S. (2012) · Journal of Public Administration Research and Theory · doi:10.1093/jopart/mur011 · Cited by 3,637 · cited 1×
“[46]’s proposed framework considers principled engagement and shared motivation between all the participants in a governance structure (their shared values) as the basis for resolving tensions.”
Background
Research works printed but not linked: 15
- The Ada Lovelace Institute, The AI Now Institute, and The Open Government Partnership. 2021. Algorithmic accountability for the public sector. http://www.opengovpartnership.org/documents/
- United Nations. General Assembly. 1949. Universal declaration of human rights. Vol. 3381. Department of State United States of America.
- Amber Berson, Monika Sengul-Jones, and Melissa Tamani. 2021. Unreliable Guidelines: Reliable sources and marginalized communities in French, English, and Spanish Wikipedias. https://artandfeminism.org/wp-content/uploads/2021/06/Unrealiable-Guidelines_Final.pdf
- Alina Beygelzimer Yann Dauphin Percy Liang and Jennifer Wortman Vaughan. 2021. Introducing the NeurIPS 2021 Paper Checklist. https://neuripsconf.medium.com/introducing-the-neurips-2021-paper-checklist-3220d6df500b
- Abeba Birhane Vinay Uday Prabhu and Emmanuel Kahembwe. 2021. Multimodal datasets: misogyny pornography and malignant stereotypes. ArXiv abs/2110.01963(2021). · arXiv
- 10.18653/v1 · doi:10.18653/v1
- David Freeman Engstrom, Daniel E Ho, and Cristina Isabel Ceballos. 2021. Disparate Limbo: How Administrative Law Erased Antidiscrimination. Yale Law Journal, Forthcoming(2021).
- William Labov. 1994. Principles of Linguistic Change, Volume 1: Internal Factors. John Wiley & Sons.
- Claudia Mitchell-Kernan. 1971. Language Behavior in a Black Urban Community. Monograph of the Language-Behavior Research Laboratory, No. 2 (1971).
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.
- Cornelia Roux and Petro Du Preez. 2013. Human rights literacy: A quest for meaning. Retrieved 20(2013), 2017.
- 10.1093/idpl · doi:10.1093/idpl
- Samson Tan. 2022. Linguistically-Inclusive Natural Language Processing. Ph. D. Dissertation. National University of Singapore.
- 10.18653/v1 · doi:10.18653/v1
- 10.18653/v1 · doi:10.18653/v1
Cited by
The future landscape of large language models in medicine
Clusmann, J., Kolbinger, F. R., Muti, H. S., Carrero, Z. I., Eckardt, J.-N., Laleh, N. G., Löffler, C. M. L., Schwarzkopf, S.-C., Unger, M., Veldhuizen, G. P., et al. (2023) · Communications Medicine · doi:10.1038/s43856-023-00370-1 · Cited by 1,045
“For example, they can break down complex concepts to an amateur level (Box 2, Supplementary Data, example 8, 9) and provide individualized feedback on academic topics with reasonable explanations (Supplementary Data, example 9) 6 .”
BackgroundBias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245
Auditing large language models: a three-layered approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2024) · AI and Ethics · doi:10.1007/s43681-023-00289-2 · Cited by 289
“, interactive model cards [183] and information about the initial training dataset [184], [185], to help downstream developers adapt the model appropriately.”
BackgroundHarms from Increasingly Agentic Algorithmic Systems
Chan, A., Salganik, R., Markelius, A., Pang, C., Rajkumar, N., Krasheninnikov, D., Langosco, L., He, Z., Duan, Y., Carroll, M., et al. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594033 · Cited by 149
“[97] highlight the imbalance of power between AI developers and the rest of society, and propose frameworks and”
BackgroundBrinkmann, L., Baumann, F., Bonnefon, J.-F., Derex, M., Müller, T. F., Nussberger, A.-M., Czaplicka, A., Acerbi, A., Griffiths, T. L., Henrich, J., et al. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01742-2 · Cited by 132
“One approach is to care-fully curate the dataset on which the model is trained 187,188 .”
BackgroundFoundation Models and Fair Use
Henderson, P., Li, X., Jurafsky, D., Hashimoto, T., Lemley, M. A., & Liang, P. (2023) · doi:10.2139/ssrn.4404340 · Cited by 57
Self-citationThe BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
Laurençon, H., Saulnier, L., Wang, T., Akiki, C., del Moral, A. V., Scao, T. L., Von Werra, L., Mou, C., Ponferrada, E. G., Nguyen, H., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03915 · Cited by 93
“, 2022), and data governance issues (Jernite et al., 2022) with the explicit goal of doing good for and by the people whose data we collected.”
BackgroundMethodInfluentialSelf-citationKim, J., Lee, J., Jang, K. M., & Lourentzou, I. (2023) · Telematics and Informatics · doi:10.1016/j.tele.2023.102085 · Cited by 63
The Foundation Model Transparency Index
Bommasani, R., Klyman, K., Longpre, S., Kapoor, S., Maslej, N., Xiong, B., Zhang, D., & Liang, P. (2023) · arXiv · doi:10.48550/arxiv.2310.12941 · Cited by 46
Self-citationData Authenticity, Consent, and Provenance for AI Are All Broken: What Will It Take to Fix Them?
Longpre, S., Mahari, R., Obeng-Marnu, N., Brannon, W., South, T., Kabbara, J., & Pentland, A. (2024) · doi:10.21428/e4baedd9.a650f77d · Cited by 40
Reclaiming the Digital Commons: A Public Data Trust for Training Data
Chan, A., Bradley, H., & Rajkumar, N. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604658 · Cited by 43
“[54], which provide a framework for the governance of language model data.”
BackgroundInternational AI Institutions: A Literature Review of Models, Examples, and Proposals
Maas, M. M., & Villalobos Ruiz, J. J. (2023) · doi:10.2139/ssrn.4579773 · Cited by 7
Loos, E. F., & Radicke, J. (2024) · AI and Ethics · doi:10.1007/s43681-024-00470-1 · Cited by 18
“For a writing tool such as ChatGPT, see also the role ethically responsive NLP [Natural Language Processing] technology [3, p. 587] could play, and [16] who favour the “system-atic and transparent management of language data (…) an approach to global language data governance that attempts to…”
The ROOTS Search Tool: Data Transparency for LLMs
Piktus, A., Akiki, C., Villegas, P., Laurençon, H., Dupont, G., Luccioni, A. S., Jernite, Y., & Rogers, A. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-demo.29 · Cited by 59
Self-citationBigBIO: A Framework for Data-Centric Biomedical Natural Language Processing
Fries, J. A., Weber, L., Seelam, N., Altay, G., Datta, D., Garda, S., Kang, M., Su, R., Kusa, W., Cahyawijaya, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.15076 · Cited by 17
“Jernite et al. [22] survey many aspects of the governance of language data, and propose a framework for distributed governance of large language corpora.”
BackgroundSelf-citationTen simple rules for building and maintaining a responsible data science workflow
Stoudt, S., Jernite, Y., Marshall, B., Marwick, B., Sharan, M., Whitaker, K., & Danchev, V. (2024) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1012232 · Cited by 11
Self-citationPrivacy Audit of Commercial Large Language Models with Sophisticated Prompt EngineeringPreprint
Lund, J., Macfarlane, S., & Niles, B. (2024) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/cr2tx · Cited by 14
Emerging Data Practices: Data Work in the Era of Large Language Models
Garcia, A. A., Candello, H. C. D. S. P., Badillo-Urquiola, K. A., & Wong-Villacrés, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714069 · Cited by 20
“Other concerns include perpetuating social biases or a lack of accountability when producing harmful results [54, 137], privacy, and intellectual property infringement [151].”
Karras, A., Theodorakopoulos, L., Karras, C., Krimpas, G. A., Γιάνναρος, Α., & Bakalis, C.-P. (2025) · Algorithms · doi:10.3390/a18120791 · Cited by 11
““Citizen 360” views remain tenable only under strict consent and purpose limitation with policy-aware lakes and rationale-bearing retrieval logs [87,123–125].”
The Moral Landscape of General-Purpose Large Language Models
Pistilli, G. (2024) · Chapman & Hall/CRC eBooks · doi:10.1201/9781003320791-9 · Cited by 6
Complex QA and language models hybrid architectures, SurveyPreprint
Daull, X., Bellot, P., Bruno, E., Martin, V., & Murisasco, É. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.09051 · Cited by 43
Powar, J., Janssen, H., Cloete, R., & Singh, J. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732164 · Cited by 11
Evaluating the Social Impact of Generative AI Systems
Solaiman, I., Talat, Z., Agnew, W. S., Ahmad, L., Baker, D., Blodgett, S. L., Chen, C., Daumé, H., Dodge, J., Duan, I., et al. (2025) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780198940272.013.0025 · Cited by 10
“The immense size of training datasets [118] makes scrutiny increasingly difficult.”
BackgroundSelf-citationCan Licensing Mitigate the Negative Implications of Commercial Web Scraping?
Li, H., Vincent, N., Jernite, Y., Merrill, N., Benjamin, J. J., & Tarkowski, A. (2023) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3584931.3611276 · Cited by 4
“• Examining parallels between licensing and other creatororiented responsible AI initiatives, such as data stewardship [2, 5] and refusal [4].”
Self-citationBig Data Versus Big GPU: Evolving Requirements and Governance Dynamics of AI Training Data
Cheng, L., Gong, X., & Zhao, Y. (2025) · International Journal of Digital Law and Governance · doi:10.1515/ijdlg-2025-0004 · Cited by 5
Data Governance – neuer europäischer Rechtsrahmen für Technik, Ökonomie und Nachhaltigkeit
Weber, B. (2023) · Springer eBooks · doi:10.1007/978-3-662-67556-4_1 · Cited by 3
The Emerging Artifacts of Centralized Open-Code
Choksi, M. Z., Mandel, I., Widder, D. G., & Shvartzshnaider, Y. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659019 · Cited by 5
Control and Data Integrity are Important Factors of Data Governance Technology
Chandra, Y. U., Prabowo, H., Gaol, F. L., & Purwandari, B. (2023) · International Conference on ICT for Smart Society (ICISS) · doi:10.1109/iciss59129.2023.10291563 · Cited by 4
“Data storage plays a crucial role in the realm of data governance, given the significant emphasis placed on it as a contributing factor [14], [21], [22], [23] .”
BackgroundChief Privacy Officer: A Systematic Literature Review and Future Research Directions
Bantan, M., & Shawosh, M. (2024) · Communications of the Association for Information Systems · doi:10.17705/1cais.05429 · Cited by 1
Agentic Decision Intelligence: AI Agents Driving Organizational Transformation
Shil, P. K., & Bhaskar, J. K. (2026) · Synthesis lectures on computer science · doi:10.1007/978-3-032-18322-4_9 · Cited by 1
Salama, W. M., & Aly, M. H. (2024) · International Conference on Computer Theory and Applications (ICCTA) · doi:10.1109/iccta64612.2024.10974894 · Cited by 1
GAIA Search: Hugging Face and Pyserini Interoperability for NLP Training Data Exploration
Piktus, A., Ogundepo, O., Akiki, C., Oladipo, A., Zhang, X., Schoelkopf, H., Biderman, S., Potthast, M., & Lin, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-demo.57 · Cited by 8
Self-citationResource papers as registered reports: a proposal
van Miltenburg, E. (2023) · Northern European Journal of Language Technology · doi:10.3384/nejlt.2000-1533.2023.4884
NaturalAdversaries: Can Naturalistic Adversaries Be as Effective as Artificial Adversaries?
Gabriel, S., Palangi, H., & Choi, Y. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.413
A Large Financial Chat ModelPreprint
Ding, F. (2024) · doi:10.33774/coe-2024-lsc94
Transforming Enterprise Data Management through Unified Data Governance
Suhra, R. (2024) · International Journal of Computer Applications · doi:10.5120/ijca2024924234
Securing social network user data in large language model deployments: challenges and best practices
Jalali, N. A., & Hongsong, C. (2025) · Cluster Computing · doi:10.1007/s10586-025-05482-y · Cited by 1
“Ethnic and privacy control committee is another plus point to establish a dedicated team for controlling and monitoring to ensure how users’ data is used for model training [70].”
The Commons Approach: A Proposal for a Digital Humanist Agenda to (Re)Open Artificial Intelligence
Mayer, K., Skupien, S., & Knaus, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-11108-1_28
Data Governance Approach to Integrate Techstrat Fusion for Enterprise Growth
Srivastava, S., Singh, M., & Rokade, V. (2025) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-83708-460-920251005
Foundation Vision-Language models
Laurençon, H. (2025) · theses.fr (ABES) · doi:10.70675/ec0311a0zb6a4z4249zb9fbz01bf51a02c7c · Cited by 1
A Human-Centric Framework for Data Attribution in Large Language Models
Wuehrl, A., Ruckdeschel, M., Lo, K., & Rogers, A. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812236 · Cited by 1
“Finally, we outline what role data attribution research could play in the re-negotiation of stakeholder roles for a sustainable data economy.”
Self-citationSeed Bank, Co-op, Stoop Swap: Metaphors for Governing Language Model Data for Creative Writing
Guo, A., Schnitzler, C., & Gero, K. I. (2026) · Creativity and Cognition · doi:10.1145/3803784.3807550
“This study was approved by the relevant ethics review board.”
Data Governance for Generative AI: Curation, Documentation, and Lifecycle Management
Armand, K. W. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-0368-8.ch006
Ethical Maturity Framework for Big Data Governance in Academic Libraries: A Multi-Method Study
Alahdal, A. H., Sahid, N. Z., Mohamad, A. N., & Daqal, H. A. (2026) · The International Information & Library Review · doi:10.1080/10572317.2026.2714121
García-López, I. M., & Guillén-Yparrea, N. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-37438-7_8
Metadata
- Authors
- Jernite, Y., Nguyen, H., Biderman, S., Rogers, A., Masoud, M., Danchev, V., Tan, S., Luccioni, A. S., Subramani, N., Johnson, I., Dupont, G., Dodge, J., Lo, K., Talat, Z., Radev, D., Gokaslan, A., Nikpoor, S., Henderson, P., Bommasani, R., & Mitchell, M.
- Affiliations
- Hugging Face (US)
- Ontocord, USA
- EleutherAI, USA
- University of Copenhagen (DK)
- Independent researcher, Ireland
- University of Essex (GB)
- Amazon (United States) (US)
- Allen Institute for Artificial Intelligence (US)
- Wikimedia Foundation (US)
- Independent, France
- Simon Fraser University (CA)
- Yale University (US)
- Cornell University (US)
- CAIDP, Canada
- Stanford University (US)
- ORCID
- Venue
- 2022 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2022-06-20
- Pages
- 2206-2222
- OpenAlex
- W4283167130
- Cited by
- 75
- References
- 139
- Citations by year
- 2026: 12 · 2025: 16 · 2024: 17 · 2023: 25 · 2022: 5
- FWCI
- 6.73
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
- Research at the University of Copenhagen (University of Copenhagen) (publishedVersion, cc-by-nc)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
- IT University Of Copenhagen (IT University of Copenhagen) (submittedVersion, cc-by)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Keywords
- large language models, distributed governance, data management, stakeholder management, human values, data rights, international research collaboration, data transparency, language technology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.