Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs
Open accessAbstract
Disaggregated evaluations of AI systems, in which system performance is assessed and reported separately for different groups of people, are conceptually simple. However, their design involves a variety of choices. Some of these choices influence the results that will be obtained, and thus the conclusions that can be drawn; others influence the impacts---both beneficial and harmful---that a disaggregated evaluation will have on people, including the people whose data is used to conduct the evaluation. We argue that a deeper understanding of these choices will enable researchers and practitioners to design careful and conclusive disaggregated evaluations. We also argue that better documentation of these choices, along with the underlying considerations and tradeoffs that have been made, will help others when interpreting an evaluation's results and conclusions.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Algorithmic audits remain poorly defined and lack widely adopted standards.
“Algorithmic audits (or 'AI audits') are an increasingly popular mechanism for algorithmic accountability; however, they remain poorly defined.”
From Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem (2022) · cites this paper as [6]
The Introduction says that developing a clear definition of an AI audit is a much-needed research direction, supporting the claim that audits remain poorly defined. It does not establish that audits lack widely adopted standards or that they are increasingly popular. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 43 · Cited by 72
Research works printed but not linked: 4
- Alexandra Chouldechova, Emily Putnam-Hornstein, Diana Benavides Prado, Oleksandr Fialko, and Rhema Vaithianathan. 2018. A case study of algorithm- assisted decision making in child maltreatment hotline screening decisions. In Proc. of the Conference on Fairness, Accountability, and Transparency (FAT*).
- Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumeé III, and Kate Crawford. 2018. Datasheets for datasets. CoRR arXiv:1803.09010. · arXiv
- Kenneth Keppel, Elsie Pamuk, John Lynch, Olivia Carter-Pokras, Insun Kim, Vickie Mays, Jeffrey Pearcy, Victor Schoenbach, and Joel S. Weissman. 2005. Methodological issues in measuring health disparities. Vital and Health Statistics, Series 2: Data Evaluation and Methods Research 2, 141 (2005), 1-16.
- Morgan Klaus Scheuerman, Jacob M. Paul, and Jed R. Brubaker. 2019. How Computers See Gender: An Evaluation of Gender Classification in Commercial Facial Analysis and Image Labeling Services. Proc. of the ACM on Human- Computer Interaction (2019).
- Authors
- Barocas, S., Guo, A., Kamar, E., Krones, J., Morris, M. R., Vaughan, J. W., Wadsworth, W. D., & Wallach, H.
- Affiliations
- Microsoft (United States) (US)
- University of Michigan (US)
- Venue
- Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2021-07-21
- Pages
- 368-378
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W3134218834
- Cited by
- 72
- References
- 43
- Citations by year
- 2026: 14 · 2025: 11 · 2024: 15 · 2023: 16 · 2022: 12 · 2021: 4
- FWCI
- 10.60
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › AI and HR Technologies
- Keywords
- AI evaluation, evaluation documentation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
