Data Governance in the Age of Large-Scale Data-Driven Language Technology
Open accessThe DOI matches its Crossref record; 152 of the 153 references it cites were found in a registry or an index, and 1 are not checked yet; 62 of the 158 sources its claims cite were checked for support.
Abstract
The recent emergence and adoption of Machine Learning technology, and specifically of Large Language Models, has drawn attention to the need for systematic and transparent management of language data. This work proposes an approach to global language data governance that attempts to organize data management amongst stakeholders, values, and rights. Our proposal is informed by prior work on distributed governance that accounts for human values and grounded by an international research collaboration that brings together researchers and practitioners from 60 countries. The framework we present is a multi-party international governance structure focused on language data, and incorporating technical and organizational tools needed to support its work.
Verified 2026-11-19 · 1 reference not checked yet
Overview
CoChat Summary
TL;DR
This paper proposes a multi-party Data Stewardship Organization (DSO) for governing large-scale language data, grounded in eight values and six actor roles. It is a conceptual governance framework developed through a year-long workshop with over 1000 participants from 60 countries, informed by distributed governance, legal, sociolinguistic, and ML data-management literature.
Key findings
- The proposed DSO organizes language-data governance around six actor roles: Data Rights-holders, Data Providers, Data Hosts, Data Modelers, Data Stewardship Organization, and Data Helpers (Table 4).
- The framework is guided by eight values: inclusion/representation/non-discrimination, autonomy/consent/contestation, privacy, just rewards, licensing/attribution, local knowledge, participation, and beneficence (Table 1).
- The paper argues existing ML data management is insufficient: centralized repositories give the governing organization full say; public repositories rely on self-governance by submitters; open data initiatives conflict with removal/update needs; Wikimedia offers a relevant distributed-governance example.
- It proposes contractual flow-down, non-dissemination clauses, contestation/removal processes, a license taxonomy, and a DSO standard Data Provider-Host agreement (Appendix B) as mechanisms to support the values.
Methods
- Conceptual/position paper developed through a year-long, distributed collaborative workshop on Large Language Models with over 1000 participants from 60 countries; the data governance working group grew from around 10 to 50+ participants.
- The analysis reviews prior work on distributed governance, sociolinguistics, legal frameworks (property, privacy, user rights), ML/NLP data management, and Wikimedia, then synthesizes a proposed governance structure.
- No empirical experiment or quantitative analysis is reported; the work is qualitative synthesis and design, with appendices documenting value-crafting and a standard Data Provider-Host agreement.
Limitations
- Paper-stated: values expressed by workshop participants may prioritize their interests and exclude other language users; Human Rights grounding is limited by diverse philosophical foundations and its historical individual-state focus; governance faces a trade-off between harmonization and independence; legal agreements are difficult to develop.
- Paper-stated: the DSO is not a direct party to agreements and relies on fallback/default licenses and interoperability; contestation and deletion depend on non-dissemination and technical/legal maturity.
- Evident from the text: the proposal is not implemented or empirically evaluated, so no outcome data on effectiveness or adoption are provided.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
31 Valid · 11 Not valid · 34 Not checked · 57 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, whereas recent advances in modeling and hardware have increased the data training capacity of LLMs, increasing from Wikipedia-scale corpora to close to three orders of magnitude more”
What the sources say
- Partly supportedcites Hoffmann et al. [71] [71] Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and L. Sifre. 2022. Training Compute-Optimal Large Language Models. ArXiv abs/2203.15556 (2022). - sourceExperimental — accuracy not yet measured
The Introduction (p. 2) reports LLMs trained on roughly 300 billion tokens and Chinchilla on 1.4 trillion, supporting the point that LLMs can train on very large datasets. Appendix A (p. 22) lists Wikipedia as one small component of Chinchilla’s dataset, but the paper does not establish a historical increase from Wikipedia-scale corpora to nearly three orders of magnitude more data, or attribute such an increase to advances in modeling and hardware. (LLM, conf=0.82)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“These practical applications of language technology are increasingly reliant on approaches based on trained Large Language Models (LLMs) [23, 41, 104, 111], whereby models are first exposed to as large and varied a collection of language data as possible with the aim of extracting 'general' properties of a language of interest.”
What the sources say
- Supportedcites [23] [23] Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020). - source
Section 2.2 (p. 8) describes training on a very large mixture of text datasets and adding corpora to increase its diversity. The Introduction (p. 3) says unsupervised pre-training develops broad skills and pattern-recognition abilities, supporting the claim's account of learning general language capabilities. The paper also emphasizes data quality rather than simply maximizing quantity. (LLM, conf=0.90)
- Not checked — no full text of the sourcecites [41] [41] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers) , Jill Burstein, Christy Doran, and Thamar Solorio (Eds.). Association for Computational Linguistics, 4171-4186. https://doi.org/10.18653/v1/n19-1423 - source
- Partly supportedcites [104] [104] Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep Contextualized Word Representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers) , Marilyn A. Walker, Heng Ji, and Amanda Stent (Eds.). Association for Computational Linguistics, 2227-2237. https://doi.org/10.18653/v1/n18-1202 - sourceExperimental — accuracy not yet measured
The Introduction describes a bidirectional language model trained on a large text corpus and says its representations capture aspects of syntax and meaning. Related work specifies a corpus of approximately 30 million sentences. The paper does not establish that training data should be as large and varied as possible, or make that claim about LLM approaches generally. (LLM, conf=0.84)
- Not matchedcites [111] [111] Alec Radford and Karthik Narasimhan. 2018. Improving Language Understanding by Generative Pre-Training. -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“As such, devising a governance structure based solely on values expressed by our participants runs the risk of prioritizing their interests and excluding visions that may be more relevant to other language users [65].”
What the sources say
- Supportedcites [65] [65] Daniel Greene, Anna Lauren Hoffmann, and Luke Stark. 2019. Better, Nicer, Clearer, Fairer: A Critical Assessment of the Movement for Ethical Artificial Intelligence and Machine Learning. In HICSS . - source
Section 5.2, “Expert oversight” (page not provided), says ethical-AI statements give experts a narrow role in deciding whose concerns count while treating users and the public differently. Section 3.1, “Values in technology design” (page not provided), stresses considering alternative courses of action and who counts in design. The citing authors apply that critique to their own participants and other language users; the source supports the underlying risk, not a finding about those particular groups. (LLM, conf=0.86)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“They note that most of these focus on endogenous notions of technical performance and novelty, and leave out considerations of broader context and impact that are necessary to shaping a governance effort.”
What the sources say
- Supportedcites [18] [18] Abeba Birhane, Pratyusha Kalluri, Dallas Card, William Agnew, Ravit Dotan, and Michelle Bao. 2021. The values encoded in machine learning research. arXiv preprint arXiv:2106.15590 (2021). - source
The Abstract reports that the 100 highly cited ML papers most frequently emphasized values including performance and novelty, while few connected their projects to societal needs or discussed negative consequences. The Conclusion likewise says these papers prioritized technical values while neglecting broader social needs and potential harms. The reference to shaping a governance effort is the citing authors’ framing, rather than a separate finding attributed to the source. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Unsurprisingly, gendered and racial disparities have been documented in a number of language technologies [37, 79, 141]”
What the sources say
- Partly supportedcites [37] [37] Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2019. Plug and play language models: Asimple approach to controlled text generation. arXiv preprint arXiv:1912.02164 (2019). - sourceExperimental — accuracy not yet measured
Section 4.4, “Language Detoxification” (page unavailable), says language models reflect biases and reports racist GPT-2 output and toxicity measurements for prompts referring to racial or ethnic groups. It does not document gender disparities or disparities across a number of language technologies, so it supports only part of the cited claim. (LLM, conf=0.85)
- Supportedcites [79] [79] Allison Koenecke, Andrew Nam, Emily Lake, Joe Nudell, Minnie Quartey, Zion Mengesha, Connor Toups, John R Rickford, Dan Jurafsky, and Sharad Goel. 2020. Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences 117, 14 (2020), 7684-7689. - source
The Results report racial differences in word error rates across five commercial speech-recognition systems and higher error rates for male than female speakers in the sample. This supports the cited statement as an example of gendered and racial disparities in language technologies, specifically speech recognition; the paper does not establish disparities across all types of language technology. (LLM, conf=0.89)
- Partly supportedcites [141] [141] Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, and Emily Dinan. 2021. Bot-Adversarial Dialogue for Safe Conversational Agents. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . Association for Computational Linguistics, Online, 2950-2968. https://doi.org/10.18653/v1/2021.naacl-main.235 - sourceExperimental — accuracy not yet measured
The Introduction says dialogue models can learn biased language, and §7 Ethical Considerations reports racial bias in hate-speech datasets. The paper does not document gendered disparities or establish disparities across a number of language technologies; its mention of gender in Appendix A.2 defines a hate-speech category, not a measured disparity. (LLM, conf=0.83)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
19 more claims · 5 with their cited sources checked
Create a free account to see all 133 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 88
- It may state two separate claims as one. 45
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, these models also raise concerns about misinformation, privacy, biases in the training data, and potential for misuse3,7–10.”
From The future landscape of large language models in medicine (2023) · cites this paper as [8]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“One approach is to carefully curate the dataset on which the model is trained 187,188 .”
From Machine culture (2023) · cites this paper as [188]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Each stakeholder brings perspectives that may specify different fairness constraints for the same application and setting.”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Jernite et al. (2022)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ 3. PURPOSE, RIGHTS GRANTED & SCOPE · p. 19
“The Data Provider grants to the Data Host a non-exclusive, non-transferable, non-sublicensable, irrevocable, perpetual, royalty-free and worldwide license to use (that is access, store, prepare, process, label and/or share) the agreed upon Data”
§ 9. TERM AND TERMINATION · p. 22
“Upon termination of this Agreement for any reason, the Data Host and Data Provider cease to use the Data and Processed Datasets and for the purposes set out in this Agreement within 14 days and upon that delete the Data and Processed Datasets immediately”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 15. SURVIVAL · p. 22
“10 (Term and Termination)”
§ 10. FORCE MAJEURE · p. 22
“## 10. FORCE MAJEURE”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 15. SURVIVAL · p. 22
“15 (Governing Law)”
§ 15. SURVIVAL · p. 22
“## 15. SURVIVAL”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 15. SURVIVAL · p. 22
“16 (Survival)”
§ 16. SEVERABILITY · p. 22
“## 16. SEVERABILITY”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 15. SURVIVAL · p. 22
“12 (Confidentiality)”
§ 12. ENTIRE AGREEMENT · p. 22
“## 12. ENTIRE AGREEMENT”
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 2×
“These practical applications of language technology are increasingly reliant on approaches based on trained Large Language Models (LLMs) [23, 41, 104, 111], whereby models are first exposed to as large and varied a collection of language data as possible with the aim of extracting “general” properties of a language of…”
BackgroundUCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,410
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 2×
“CL] [23] Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.”
BackgroundSituated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective
Haraway, D. (1988) · Feminist Studies · doi:10.2307/3178066 · Cited by 18,364
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,690
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
An Integrative Framework for Collaborative Governance
Emerson, K., Nabatchi, T., & Balogh, S. (2012) · Journal of Public Administration Research and Theory · doi:10.1093/jopart/mur011 · Cited by 3,637 · cited 1×
“[46]’s proposed framework considers principled engagement and shared motivation between all the participants in a governance structure (their shared values) as the basis for resolving tensions.”
Background
Research works printed but not linked: 15
- The Ada Lovelace Institute, The AI Now Institute, and The Open Government Partnership. 2021. Algorithmic accountability for the public sector. http://www.opengovpartnership.org/documents/
- United Nations. General Assembly. 1949. Universal declaration of human rights. Vol. 3381. Department of State United States of America.
- Amber Berson, Monika Sengul-Jones, and Melissa Tamani. 2021. Unreliable Guidelines: Reliable sources and marginalized communities in French, English, and Spanish Wikipedias. https://artandfeminism.org/wp-content/uploads/2021/06/Unrealiable-Guidelines_Final.pdf
- Alina Beygelzimer Yann Dauphin Percy Liang and Jennifer Wortman Vaughan. 2021. Introducing the NeurIPS 2021 Paper Checklist. https://neuripsconf.medium.com/introducing-the-neurips-2021-paper-checklist-3220d6df500b
- Abeba Birhane Vinay Uday Prabhu and Emmanuel Kahembwe. 2021. Multimodal datasets: misogyny pornography and malignant stereotypes. ArXiv abs/2110.01963(2021). · arXiv
- 10.18653/v1 · doi:10.18653/v1
- David Freeman Engstrom, Daniel E Ho, and Cristina Isabel Ceballos. 2021. Disparate Limbo: How Administrative Law Erased Antidiscrimination. Yale Law Journal, Forthcoming(2021).
- William Labov. 1994. Principles of Linguistic Change, Volume 1: Internal Factors. John Wiley & Sons.
- Claudia Mitchell-Kernan. 1971. Language Behavior in a Black Urban Community. Monograph of the Language-Behavior Research Laboratory, No. 2 (1971).
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.
- Cornelia Roux and Petro Du Preez. 2013. Human rights literacy: A quest for meaning. Retrieved 20(2013), 2017.
- 10.1093/idpl · doi:10.1093/idpl
- Samson Tan. 2022. Linguistically-Inclusive Natural Language Processing. Ph. D. Dissertation. National University of Singapore.
- 10.18653/v1 · doi:10.18653/v1
- 10.18653/v1 · doi:10.18653/v1
Cited by
The future landscape of large language models in medicine
Clusmann, J., Kolbinger, F. R., Muti, H. S., Carrero, Z. I., Eckardt, J.-N., Laleh, N. G., Löffler, C. M. L., Schwarzkopf, S.-C., Unger, M., Veldhuizen, G. P., et al. (2023) · Communications Medicine · doi:10.1038/s43856-023-00370-1 · Cited by 1,045
“For example, they can break down complex concepts to an amateur level (Box 2, Supplementary Data, example 8, 9) and provide individualized feedback on academic topics with reasonable explanations (Supplementary Data, example 9) 6 .”
BackgroundBias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245
Auditing large language models: a three-layered approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2024) · AI and Ethics · doi:10.1007/s43681-023-00289-2 · Cited by 289
“, interactive model cards [183] and information about the initial training dataset [184], [185], to help downstream developers adapt the model appropriately.”
BackgroundHarms from Increasingly Agentic Algorithmic Systems
Chan, A., Salganik, R., Markelius, A., Pang, C., Rajkumar, N., Krasheninnikov, D., Langosco, L., He, Z., Duan, Y., Carroll, M., et al. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594033 · Cited by 149
“[97] highlight the imbalance of power between AI developers and the rest of society, and propose frameworks and”
BackgroundBrinkmann, L., Baumann, F., Bonnefon, J.-F., Derex, M., Müller, T. F., Nussberger, A.-M., Czaplicka, A., Acerbi, A., Griffiths, T. L., Henrich, J., et al. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01742-2 · Cited by 132
“One approach is to care-fully curate the dataset on which the model is trained 187,188 .”
Background
Metadata
- Authors
- Jernite, Y., Nguyen, H., Biderman, S., Rogers, A., Masoud, M., Danchev, V., Tan, S., Luccioni, A. S., Subramani, N., Johnson, I., Dupont, G., Dodge, J., Lo, K., Talat, Z., Radev, D., Gokaslan, A., Nikpoor, S., Henderson, P., Bommasani, R., & Mitchell, M.
- Affiliations
- Hugging Face (US)
- Ontocord, USA
- EleutherAI, USA
- University of Copenhagen (DK)
- Independent researcher, Ireland
- University of Essex (GB)
- Amazon (United States) (US)
- Allen Institute for Artificial Intelligence (US)
- Wikimedia Foundation (US)
- Independent, France
- Simon Fraser University (CA)
- Yale University (US)
- Cornell University (US)
- CAIDP, Canada
- Stanford University (US)
- ORCID
- Venue
- 2022 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2022-06-20
- Pages
- 2206-2222
- OpenAlex
- W4283167130
- Cited by
- 75
- References
- 139
- Citations by year
- 2026: 12 · 2025: 16 · 2024: 17 · 2023: 25 · 2022: 5
- FWCI
- 6.73
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by)
- Research at the University of Copenhagen (University of Copenhagen) (publishedVersion, cc-by-nc)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
- IT University Of Copenhagen (IT University of Copenhagen) (submittedVersion, cc-by)
- Queen Mary Research Online (Queen Mary University of London) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Keywords
- large language models, distributed governance, data management, stakeholder management, human values, data rights, international research collaboration, data transparency, language technology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Data Governance in the Age of Large-Scale Data-Driven Language Technology. https://app.cochat.ai/papers/10.1145/3531146.3534637
- The paper
- Jernite, Y., Nguyen, H., Biderman, S., Rogers, A., Masoud, M., Danchev, V., Tan, S., Luccioni, A. S., Subramani, N., Johnson, I., Dupont, G., Dodge, J., Lo, K., Talat, Z., Radev, D., Gokaslan, A., Nikpoor, S., Henderson, P., Bommasani, R., & Mitchell, M. (2022). Data Governance in the Age of Large-Scale Data-Driven Language Technology. 2022 ACM Conference on Fairness Accountability and Transparency. https://doi.org/10.1145/3531146.3534637
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.