CoChat
Sign Up
PreprintCited by 25Ethics and Social Impacts of AIDigitalization, Law, and RegulationLaw, AI, and Intellectual Property

The Subjects and Stages of AI Dataset Development: A Framework for Dataset Accountability

Open access
Khan, Mehtab, Hanna, Alex·2022·SSRN Electronic Journal
DOI 10.2139/ssrn.4217148
Open this paper in CoChat — free account

Abstract

There has been increased attention toward the datasets that are used to train and build AI technologies from the computer science and social science research communities, but less from legal scholarship. Both Large-Scale Language Datasets (LSLDs) and Large-Scale Computer Vision Datasets (LSCVDs) have been at the forefront of such discussions, due to recent controversies involving the use of facial recognition technologies, and the discussion of the use of publicly-available text for the training of massive models which generate human-like text. Many of these datasets serve as “benchmarks” to develop models that are used both in academic and industry research, while others are used solely for training models. The process of developing LSLDs and LSCVDs is complex and contextual, involving dozens of decisions about what kinds of data to collect, label, and train a model on, as well as how to make the data available to other researchers. However, little attention has been paid to mapping and consolidating the legal issues that arise at different stages of this process: when the data is being collected, after the data is used to build and evaluate models and applications, and how that data is distributed more widely. In this article, we offer four main contributions. First, we describe what kinds of objects these datasets are, how many different kinds exist, what types of modalities they encompass, and why they are important. Second, we provide more clarity about the stages of dataset development – a process that has thus far been subsumed within broader discussions about bias and discrimination – and the subjects who may be susceptible to harms at each point of development. Third, we provide a matrix of both the stages of dataset development and the subjects of dataset development, which traces the connections between stages and subjects. Fourth, we use this analysis to identify some basic legal issues that arise at the various stages in order to foster a better understanding of the dilemmas and tensions that arise at every stage. We situate our discussion within wider discussion of current debates and proposals related to algorithmic accountability. This paper fulfills an essential gap when it comes to comprehending the complicated landscape of legal issues connected to datasets and the gigantic AI models trained on them.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2139/ssrn.4217148

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Practical approaches to implement such ideas typically revolve around the systematic collection and curation of data on various aspects of systems.

    “Practical approaches to implement such ideas typically revolve around the systematic collection and curation of data on various aspects of systems [2, 28, 46].”

    From Rethinking open source generative AI: open washing and the EU AI Act (2024) · cites this paper as [28]

    On pp. 248 and 252, the paper proposes documenting stages and design choices in AI dataset development to improve accountability. That supports systematic collection of information about datasets, but not the broader claim that practical approaches typically center on collecting and curating information about systems generally. (LLM, conf=0.79)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 3

  1. Andrea Roth, Machine Testimony, 126 YALE L.J. 1972, 2026 (2017). Andrea Roth introduced a taxonomy that can be used to categorize the various kinds of machine-based evidence that may be introduced in court. This taxonomy includes standards for what constitutes sufficient information about a dataset, and documentation related to it. 31 Id at 1910-1911. 314 Id at 1917 ("determining whether a machine learning tool impinges on due process demands an examination of the fit between quality training data, the learning model, and outcome variable, and the match between the outcome variable and the latent variable.") 3151d at 1906. 316 Anytime a government distributes benefits or burdens based on individual racial classification, it will lead to the application of struct scrutiny. Parents Involved in Cmty.
  2. Sch. v. Seattle Sch. Dist. No 1, 551 US 701, 720 (2007); Jack M. Balkan & Reva B. Siegel, The American Civil Rights Tradition: Anticlassification or Antisubordination? 58
  3. U. MIAMI L. REv. 9, 10 (2003) (the anticlassification principle is that a government may not "classify people either overtly or surreptitiously" based on factors such as race).

Cited by

Sorted by citations

Related

Authors
Khan, M., & Hanna, A.
Affiliations
  • Yale University (US)
  • DAIR Institute (US)
Corresponding author
Khan, Mehtab
ORCID
Venue
SSRN Electronic Journal
Publisher
Elsevier BV
Published
2022
ISSN
1556-5068 (ISSN-L)
DOI
10.2139/ssrn.4217148
OpenAlex
W4312481174
Cited by
24
References
0
Citations by year
2026: 1 · 2025: 7 · 2024: 12 · 2023: 4
Type
posted content
Language
en
Open access
Locations
  • SSRN Electronic Journal (acceptedVersion)
  • The Knowledge Bank (The Ohio State University) (submittedVersion, public-domain)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Law › Digitalization, Law, and Regulation
  • Physical Sciences › Computer Science › Artificial Intelligence › Law, AI, and Intellectual Property
Keywords
algorithmic accountability, data collection, data annotation, data distribution, face recognition, AI training data

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review