Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
PreprintCited by 5

The DOI matches its DataCite record.

The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI

Longpre, Shayne, Mahari, Robert, Chen, Anthony, Obeng-Marnu, Naana, Sileo, Damien, Brannon, William, Muennighoff, Niklas, Khazam, Nathan, Kabbara, Jad, Perisetla, Kartik, Wu, Xinyi, Shippole, Enrico, Bollacker, Kurt, Wu, Tongshuang, Villa, Luis, Pentland, Sandy, Hooker, Sara·2023·arXiv
DOI 10.48550/arxiv.2310.16787arXiv 2310.16787
Open this paper in CoChat — free account

Abstract

The race to train language models on vast, diverse, and inconsistently documented datasets has raised pressing concerns about the legal and ethical risks for practitioners. To remedy these practices threatening data transparency and understanding, we convene a multi-disciplinary effort between legal and machine learning experts to systematically audit and trace 1800+ text datasets. We develop tools and standards to trace the lineage of these datasets, from their source, creators, series of license conditions, properties, and subsequent use. Our landscape analysis highlights the sharp divides in composition and focus of commercially open vs closed datasets, with closed datasets monopolizing important categories: lower resource languages, more creative tasks, richer topic variety, newer and more synthetic training data. This points to a deepening divide in the types of data that are made available under different license conditions, and heightened implications for jurisdictional legal interpretations of copyright and fair use. We also observe frequent miscategorization of licenses on widely used dataset hosting sites, with license omission of 70%+ and error rates of 50%+. This points to a crisis in misattribution and informed use of the most popular datasets driving many recent breakthroughs. As a contribution to ongoing improvements in dataset transparency and responsible use, we release our entire audit, with an interactive UI, the Data Provenance Explorer, which allows practitioners to trace and filter on data provenance for the most popular open source finetuning data collections: www.dataprovenance.org.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2310.16787

Cites

Statistics

Figures

Fig. 1 The distributions of licences used in the DPCollection, a popular sample of the major supervised NLP datasets. We find a long tail of custom licences, adopted from software for data: 73% of all licences require attribution and 33% share-alike, but the most popular are usually commercially permissive.
Fig. 1 The distributions of licences used in the DPCollection, a popular sample of the major supervised NLP datasets. We find a long tail of custom licences, adopted from software for data: 73% of all licences require attribution and 33% share-alike, but the most popular are usually commercially permissive.CC-BY · From the PDF
Fig. 2 Across finetuning datasets, we visualize their mean input and target text lengths, measured in log-scaled number of characters. The colours indicate either their licence use category (left) or whether they were machine generated or human collected (right). Long target texts are represented in
Fig. 2 Across finetuning datasets, we visualize their mean input and target text lengths, measured in log-scaled number of characters. The colours indicate either their licence use category (left) or whether they were machine generated or human collected (right). Long target texts are represented inCC-BY · From the PDF
Fig. 3 A global heatmap of language representation scores measuring how well each country's spoken languages are represented by the composition of natural language datasets in DPCollection, as calculated in the 'Computing language representation' section. English-speaking and western European nations are best represented, while the Global South sees limited coverage.
Fig. 3 A global heatmap of language representation scores measuring how well each country's spoken languages are represented by the composition of natural language datasets in DPCollection, as calculated in the 'Computing language representation' section. English-speaking and western European nations are best represented, while the Global South sees limited coverage.CC-BY · From the PDF
Image 4
CC-BY · From the PDF
Image 5
CC-BY · From the PDF
Image 6
CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 73

  1. Touvron, H. et al. Llama: open and efficient foundation language models. Preprint at https://arxiv.org/abs/2302.13971 (2023). · arXiv
  2. Wang, Y. et al. Self-instruct: aligning language model with self generated instructions. In Proc. of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (eds Rogers, A. et al.) 13484-13508 (Association for Computational Linguistics, 2023).
  3. Anil, R. et al. Palm 2 technical report. Preprint at https://arxiv.org/ abs/2305.10403 (2023). · arXiv
  4. Achiam, J. et al. GPT-4 technical report. Preprint at https://arxiv. org/abs/2303.08774 (2023).
  5. Yoo, J., Perlin, K., Kamalakara, S. R. & Araújo J. G. Scalable training of language models using JAX-pjit and TPUv4. Preprint at https://arxiv.org/abs/2204.06514 (2022). · arXiv
  6. Wei, J. et al. Finetuned language models are zero-shot learners. In Proc. 2022 International Conference on Learning Representations https://openreview.net/pdf?id=gEZrGCozdqR (ICLR, 2022).
  7. Sanh, V. et al. Multitask prompted training enables zero-shot task generalization. In Proc. 2022 International Conference on Learning Representations https://openreview.net/pdf?id=9Vrb9D0WI4 (ICLR, 2022).
  8. Muennighoff, N. et al. Crosslingual generalization through multitask finetuning. In Proc. of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (eds Rogers, A. et al.) 15991-16111 (Association for Computational Linguistics, 2023).
  9. Lhoest, Q. et al. Datasets: a community library for natural language processing. In Proc. 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations (eds Adel, H. & Shi, S.)175-184 (Association for Computational Linguistics, 2021).
  10. Gao, L. et al. The pile: an 800 GB dataset of diverse text for language modeling. Preprint at https://arxiv.org/abs/2101.00027 (2020). · arXiv
  11. Penedo, G. et al. The RefinedWeb dataset for Falcon LLM: outperforming curated corpora with web data, and web data only. In Proc. of the 37th International Conference on Neural Information Processing Systems 79155-79172 (Curran, 2023)
  12. Wang, Y. et al. Benchmarking generalization via in-context instructions on 1,600+ language tasks. In Proc. of the 2022 Conference on Empirical Methods in Natural Language Processing (eds Goldberg, Y. et al.) 5085-5109 (Association for Computational Linguistics, 2022).
  13. Longpre, S. et al. The flan collection: designing data and methods for effective instruction tuning. In Proc. of the 40th International Conference on Machine Learning https://openreview.net/ pdf?id=ZX4uS605XV (2023).
  14. Biderman, S., Bicheno, K. & Gao, L. Datasheet for the pile. Preprint at https://arxiv.org/abs/2201.07311 (2022). · arXiv
  15. Dodge, J. et al. Documenting large webtext corpora: a case study on the colossal clean crawled corpus. In Proc. 2021 Conference on Empirical Methods in Natural Language Processing (eds Adel, H. & Shi, S.) 1286-1305 (Association for Computational Linguistics, 2021).
  16. Bandy, J. & Vincent, N. Addressing 'documentation debt' in machine learning research: a retrospective datasheet for bookcorpus. In Proc. of the Neural Information Processing Systems Track on Datasets and Benchmarks (eds Vanschoren, J. & Yeung. S.) https://datasets-benchmarks-proceedings.neurips.cc/ paper/2021/file/54229abfcfa5649e7003b83dd4755294- Paper-round1.pdf (2021).
  17. Bommasani, R. et al. The foundation model transparency index. Preprint at https://arxiv.org/abs/2310.12941 (2023). · arXiv
  18. Gebru, T. et al. Datasheets for datasets. Commun. ACM 64, 86-92 (2021).
  19. Touvron, H. et al. Llama 2: open foundation and fine-tuned chat models. Preprint at https://arxiv.org/abs/2307.09288 (2023). · arXiv
  20. Sambasivan, N. et al. 'Everyone wants to do the model work, not the data work': data cascades in high-stakes AI. In Proc. 2021 CHI Conference on Human Factors in Computing Systems (eds Kitamura, Y. & Quigley, A.) https://doi.org/10.1145/3411764.34455 (ACM, 2021). · DOI
  21. Longpre, S. et al. A pretrainer's guide to training data: measuring the effects of data age, domain coverage, quality, & toxicity. In Proc. of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) (eds Duh, K. et al.) 3245-3276 (Association for Computational Linguistics, 2024).
  22. Elangovan, A., He, J. & Verspoor, K. Memorization vs. generalization: quantifying data leakage in NLP performance evaluation. In Proc. 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume (eds
  23. Merlo, P. et al.) 1325-1335 (ACM, 2021).
  24. Carlini, N. et al. Quantifying memorization across neural language models. In Proc. 2023 International Conference on Learning Representations https://openreview.net/pdf?id=TatRHT_1cK (ICLR, 2023).
  25. Bubeck, S. et al. Sparks of artificial general intelligence: early experiments with gpt-4. Preprint at https://arxiv.org/ abs/2303.12712 (2023). · arXiv
  26. Welbl, J. et al. Challenges in detoxifying language models. In Proc. Findings of the Association for Computational Linguistics: EMNLP 2021 (eds Moens, M.-F. et al.) 2447-2469 (ACM, 2021).
  27. Xu, A. et al. Detoxifying language models risks marginalizing minority voices. In Proc. 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (eds Toutanova, K. et al.) 2390-2397 (ACM, 2021).
  28. Pozzobon, L., Ermis, B., Lewis, P. & Hooker, S. On the challenges of using black-box APIs for toxicity evaluation in research. In Proc. of the 2023 Conference on Empirical Methods in Natural Language Processing (eds Bouamor, H. et al.) 7595-7609 (Association for Computational Linguistics, 2023).
  29. Luo, Z. et al. Wizardcoder: empowering code large language models with evol-instruct. In Proc. 12th International Conference on Learning Representations https://openreview.net/pdf?id= UnUwSIgK5W (ICLR, 2024).
  30. Bommasani, R., Soylu, D., Liao, T. I., Creel, K. A. & Liang, P. Ecosystem graphs: the social footprint of foundation models. Preprint at https://arxiv.org/abs/2303.15772 (2023). · arXiv
  31. Ouyang, L. et al. Training language models to follow instructions with human feedback. In Proc. of the 36th International Conference on Neural Information Processing Systems 27730-27744 (Curran, 2024).
  32. Mitchell, M. et al. Model cards for model reporting. In Proc. Conference on Fairness, Accountability, and Transparency 220-229 (ACM, 2019).
  33. Wang, Y. et al. Super-natural instructions: generalization via declarative instructions on 1600+ NLP tasks. In Proc. 2022 Conference on Empirical Methods in Natural Language Processing (eds Goldberg, Y. et a.) 5085-5109 (Association for Computational Linguistics 2022).
  34. Xu, C. et al. WizardLM: empowering large language models to follow complex instructions. In Proc. 12th International Conference on Learning Representations https://openreview.net/ pdf?id=CfXh93NDgH (ICLR,2024).
  35. Talat, Z. et al. You reap what you sow: on the challenges of bias evaluation under multilingual settings. In Proc. BigScience Episode #5-Workshop on Challenges & Perspectives in Creating Large Language Models (eds Fan, A. et al.) 26-41 (Association for Computational Linguistics, 2022).
  36. Kreutzer, J. et al. Quality at a glance: an audit of web-crawled multilingual datasets. Trans. Assoc. Comput. Linguistics 10, 50-72 (2022).
  37. Shankar, S. et al. No classification without representation: assessing geodiversity issues in open data sets for the developing world. Preprint at https://arxiv.org/abs/1711.08536 (2017). · arXiv
  38. De Vries, T., Misra, I., Wang, C. & Van der Maaten, L. Does object recognition work for everyone? In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops 52-59 (IEEE, 2019).
  39. Mahadev, R. & Chakravarti, A. Understanding gender and racial disparities in image recognition models. Preprint at https://arxiv. org/abs/2107.09211 (2021).
  40. Ahia, O., Kreutzer, J. & Hooker, S. The low-resource double bind: an empirical study of pruning for low-resource machine translation. In Proc. Findings of the Association for Computational Linguistics: EMNLP 2021 (eds Moens, F.-M. et al.) 3316-3333 (ACM, 2021).
  41. Quang, J. Does training AI violate copyright law? Berkeley Technol. L. J. 36, 1407 (2021).
  42. Lee, K., Cooper, A. F. & Grimmelmann, J. Talkin 'bout AI generation: copyright and the generative-AI supply chain. J. Copyright Soc. USA (in the press).
  43. Gervais, D. J. AI derivatives: the application to the derivative work right to literary and artistic productions of AI machines. Seton Hall Law Rev. 52, 1111 (2021).
  44. Lemley, M. A. & Casey, B. Fair learning. Texas L. Rev. 99, 743 (2020).
  45. Longpre, S. et al. Position: data authenticity, consent, & provenance for AI are all broken: what will it take to fix them? An MIT Exploration of Generative AI https://doi.org/10.21428/ e4baedd9.a650f77d (2024). · DOI
  46. Kinney, R. M. et al. The semantic scholar open data platform. Preprint at https://arxiv.org/abs/2301.10140 (2023). · arXiv
  47. Petrov, A., La Malfa, E., Torr, P. & Bibi, A. Language model tokenizers introduce unfairness between languages. In Proc. of the 37th International Conference on Neural Information Processing Systems 36963-36990 (Curran, 2024).
  48. Bender, E. M. & Friedman, B. Data statements for natural language processing: toward mitigating system bias and enabling better science. Trans. Assoc. Comput. Linguistics 6, 587-604 (2018). Article https://doi.org/10.1038/s42256-024-00878-8 · DOI
  49. Pushkarna, M., Zaldivar, A. & Kjartansson, O. Data cards: purposeful and transparent dataset documentation for responsible AI. In Proc. 2022 ACM Conference on Fairness, Accountability, and Transparency 1776-1826 (ACM, 2022).
  50. Bender, E. M. On achieving and evaluating language-independence in NLP. Linguist. Issues Lang. Technol. https://doi.org/10.33011/lilt.v6i.1239 (2011). · DOI
  51. Bai, Y. et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. Preprint at https://arxiv.org/abs/2204.05862 (2022). · arXiv
  52. Ganguli, D. et al. Red teaming language models to reduce harms: methods, scaling behaviors, and lessons learned. Preprint at https://arxiv.org/abs/2209.07858 (2022). · arXiv
  53. Xu, C., Guo, D., Duan, N. & McAuley, J. Baize: an open-source chat model with parameter-efficient tuning on self-chat data. In Proc. of the 2023 Conference on Empirical Methods in Natural Language Processing (eds Bouamor, H. et al.) 6268-6278 (Association for Computational Linguistics, 2023).
  54. Kryściński, W., Rajani, N., Agarwal, D., Xiong, C. & Radev, D. Booksum: a collection of datasets for long-form narrative summarization. In Findings of the Association for Computational Linguistics: EMNLP 2022 (eds Goldberg, Y. et al.) 6536-6558 (Association for Computational Linguistics, 2022).
  55. Li, G., Hammoud, H., Itani, H., Khizbullin, D. & Ghanem, B. CAMEL: communicative agents for 'mind' exploration of large scale language model society. In Proc. of the 37th International Conference on Neural Information Processing Systems 51991- 52008 (Curran, 2024).
  56. Kim, S. et al. The CoT collection: improving zero-shot and few-shot learning of language models via chain-of-thought fine-tuning. In Proc. of the 2023 Conference on Empirical Methods in Natural Language Processing (eds Bouamor, H. et al.) 12685-12708 (Association for Computational Linguistics, 2023).
  57. Muennighoff, N. et al. Octopack: instruction tuning code large language models. In Proc. 12th International Conference on Learning Representations https://openreview.net/pdf?id= mw1PWNSWZP (ICLR, 2024).
  58. Peng, B., Li, C., He, P., Galley, M. & Gao, J. Instruction tuning with GPT-4. Preprint at https://arxiv.org/abs/2304.03277 (2023). · arXiv
  59. Patil, S. G., Zhang, T., Wang, X. & Gonzalez, J. E. Gorilla: large language model connected with massive APIs. Preprint at arXiv https://arxiv.org/abs/2305.15334 (2023). · arXiv
  60. Guo, B. et al. How close is ChatGPT to human experts? comparison corpus, evaluation, and detection. Preprint at https://arxiv.org/abs/2301.07597 (2023). · arXiv
  61. Zhou, C. et al. Lima: Less is more for alignment. In Proc. of the 37th International Conference on Neural Information Processing Systems 55006-55021 (Curran, 2024).
  62. Köksal, A., Schick, T., Korhonen, A. & Schütze, H. Longform: optimizing instruction tuning for long text generation with corpus extraction. Preprint at https://arxiv.org/abs/2304.08460 (2023). · arXiv
  63. Stiennon, N. et al. Learning to summarize from human feedback. In Proc. of the 34th International Conference on Neural Information Processing Systems 3008-3021 (Curran, 2020).
  64. Köpf, A. et al. OpenAssistant conversations-democratizing large language model alignment. In Proc. of the 37th International Conference on Neural Information Processing Systems 47669-47681 (Curran, 2024).
  65. Mukherjee, S. et al. Orca: progressive learning from complex explanation traces of GPT-4. Preprint at https://arxiv.org/ abs/2306.02707 (2023). · arXiv
  66. Li, R. et al. Starcoder: may the source be with you! Trans. Mach. Learn. Res. https://openreview.net/pdf?id=KoFOg41haE (2023).
  67. Sileo, D. tasksource: a dataset harmonization framework for streamlined NLP multi-task learning and evaluation. Preprint at https://arxiv.org/abs/2301.05948 (2023). · arXiv
  68. Weston, J. et al. Towards AI-complete question answering: a set of prerequisite toy tasks. In Proc. of the 4th International Conference on Learning Representations (eds Bengio, Y. & and LeCun, Y.) (ICLR, 2016).
  69. Eldan, R. & Li, Y. Tinystories: how small can language models be and still speak coherent english? Preprint at https://arxiv.org/ abs/2305.07759 (2023). · arXiv
  70. Qin, Y. et al. ToolLLM: facilitating large language models to master 16000+ real-world APIs. In Proc. 2024 International Conference on Learning Representations https://openreview.net/ pdf?id=dHng2O0Jjr (ICLR, 2024).
  71. Ding, N. et al. Enhancing chat language models by scaling high-quality instructional conversations. In Proc. of the 2023 Conference on Empirical Methods in Natural Language Processing (eds Bouamor, H. et al.) 3029-3051 (Association for Computational Linguistics, 2023).
  72. Honovich, O., Scialom, T., Levy, O. & Schick, T. Unnatural instructions: tuning language models with (almost) no human labor. In Proc. of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (eds Rogers, A. et al.) 14409-14428 (Association for Computational Linguistics, 2023).
  73. Nakano, R. et al. WebGPT: browser-assisted question-answering with human feedback. Preprint at https://arxiv.org/abs/2112.09332 (2021). 100. Hendrycks, D. et al. Measuring massive multitask language understanding. In Proc. International Conference on Learning Representations (2020). 101. Srivastava, A. et al. Beyond the imitation game: quantifying and extrapolating the capabilities of language models. Trans. Mach. Learn. Res. https://openreview.net/pdf?id= uyTL5Bvosj (2023). · arXiv

Cited by

See all 5

Metadata

Authors
Longpre, S., Mahari, R., Chen, A., Obeng-Marnu, N., Sileo, D., Brannon, W., Muennighoff, N., Khazam, N., Kabbara, J., Perisetla, K., Wu, X., Shippole, E., Bollacker, K., Wu, T., Villa, L., Pentland, S., & Hooker, S.
Venue
arXiv
Publisher
arXiv
Published
2023
DOI
10.48550/arxiv.2310.16787
arXiv
2310.16787
References
0
Type
Preprint
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI. https://app.cochat.ai/papers/10.48550/arxiv.2310.16787
The paper
Longpre, S., Mahari, R., Chen, A., Obeng-Marnu, N., Sileo, D., Brannon, W., Muennighoff, N., Khazam, N., Kabbara, J., Perisetla, K., Wu, X., Shippole, E., Bollacker, K., Wu, T., Villa, L., Pentland, S., & Hooker, S. (2023). The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI. arXiv. https://doi.org/10.48550/arxiv.2310.16787