The DOI matches its DataCite record; 33 of the 37 references it cites were found in a registry or an index.
Datasheets for Datasets
Open accessAbstract
The machine learning community currently has no standardized process for documenting datasets, which can lead to severe consequences in high-stakes domains. To address this gap, we propose datasheets for datasets. In the electronics industry, every component, no matter how simple or complex, is accompanied with a datasheet that describes its operating characteristics, test results, recommended uses, and other information. By analogy, we propose that every dataset be accompanied with a datasheet that documents its motivation, composition, collection process, recommended uses, and so on. Datasheets for datasets will facilitate better communication between dataset creators and dataset consumers, and encourage the machine learning community to prioritize transparency and accountability.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the electronic hardware industry provides datasheets with detailed characterizations of components' performances under different test conditions.”
From Model Cards for Model Reporting (2019) · cites this paper as [21]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“For example, Datasheets for Data Sets (Gebru et al., 2018) and Model Cards (Mitchell et al., 2019) are two examples of standardized reporting frameworks that may accompany clinical technologies.”
From Equipping Speech-Language Clinicians for the Critical Appraisal of an Artificial Intelligence–Driven, Evidence-Based Future (2025) · cites this paper as Gebru et al., 2018
The cited source says what the claim attributes to it.
In the citing paper
“Recently, researchers have proposed standardized forms of communicating characteristics of datasets used in machine learning [3, 21, 28] to help users understand the context in which the datasets should be used.”
From Model Cards for Model Reporting (2019) · cites this paper as [21]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Many mature industries have developed standardized methods of benchmarking various systems under different conditions.”
From Model Cards for Model Reporting (2019) · cites this paper as [21]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Advances in neural information processing systems 7
Cios, K. J., & Shields, M. E. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,093
Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments
Huang, G. B., Mattar, M., Berg, T. L., & Miller, E. G. (2008) · HAL · Cited by 4,445
Minimum information about a microarray experiment (MIAME)-toward standards for microarray data
Brazma, A., Hingamp, P., Quackenbush, J., Sherlock, G., Spellman, P., Stoeckert, C., Aach, J., Ansorge, W., Ball, C. A., Causton, H. C., et al. (2001) · Nature Genetics · doi:10.1038/ng1201-365 · Cited by 4,109
(2018) · Cited by 3,510
Self-citationPang, B., & Lee, L. (2004) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1218955.1218990 · Cited by 3,376
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681
Self-citationThe Recent Past and Near Future of Risk and/or Need Assessment
Andrews, D. A., Bonta, J. L., & Wormith, J. S. (2005) · Crime & Delinquency · doi:10.1177/0011128705281756 · Cited by 1,620
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615
The Effects of Automobile Safety Regulation
Peltzman, S. (1975) · Journal of Political Economy · doi:10.1086/260352 · Cited by 1,568
Research works printed but not linked: 3
- [7][7] Yang Trista Cao and Hal Daumé, III. 2020. Toward Gender-Inclusive Coreference Resolution. In Proceedings of the Conference of the Association for Computational Linguistics (ACL) . -
- [15][15] Kenneth Holstein, Jennifer Wortman Vaughan, Hal Daumé III, Miroslav Dudík, and Hanna M. Wallach. 2019. Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?. In 2019 ACM CHI Conference on Human Factors in Computing Systems . -
- [26][26] Semih Yagcioglu, Aykut Erdem, Erkut Erdem, and Nazli Ikizler-Cinbis. 2018. RecipeQA: A Challenge Dataset for Multimodal Comprehension of Cooking Recipes. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing .
Cited by
Sorted by citations
Items 201–300 of 1,008
Petersen, E., Potdevin, Y., Mohammadi, E., Zidowitz, S., Breyer, S., Nowotka, D., Henn, S., Pechmann, L., Leucker, M., Rostalski, P., et al. (2022) · IEEE Access · doi:10.1109/access.2022.3178382 · Cited by 92
A causal perspective on dataset bias in machine learning for medical imaging
Jones, C., Castro, D. C., Ribeiro, F. D. S., Oktay, O., McCradden, M. D., & Glocker, B. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00797-8 · Cited by 110
On Releasing Annotator-Level Labels and Information in Datasets
Prabhakaran, V., Davani, A. M., & Díaz, M. (2021) · Law · doi:10.18653/v1/2021.law-1.14 · Cited by 92
REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets
Wang, A., Liu, A., Zhang, R., Kleiman, A., Kim, L., Zhao, D., Shirai, I., Narayanan, A., & Russakovsky, O. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01625-5 · Cited by 95
Harnessing Biomedical Literature to Calibrate Clinicians’ Trust in AI Decision Support Systems
Yang, Q., Hao, Y., Quan, K., Yang, S., Zhao, Y., Kuleshov, V., & Wang, F. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581393 · Cited by 92
Can AI be racist? Color‐evasiveness in the application of machine learning to science assessments
Cheuk, T. (2021) · Science Education · doi:10.1002/sce.21671 · Cited by 86
Standards, frameworks, and legislation for artificial intelligence (AI) transparency
Lund, B. D., Orhan, Z., Mannuru, N. R., Bevara, R. V. K., Porter, B., Vinaih, M. K., & Bhaskara, P. (2025) · AI and Ethics · doi:10.1007/s43681-025-00661-4 · Cited by 101
Thylstrup, N. B. (2022) · Media Culture & Society · doi:10.1177/01634437211060226 · Cited by 73
Zhang, A., Walker, O., Nguyen, K., Dai, J., Chen, A., & Lee, M. K. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579601 · Cited by 95
How We Do Things With Words: Analyzing Text as Social and Cultural Data
Nguyen, D., Liakata, M., DeDeo, S., Eisenstein, J., Mimno, D., Tromble, R., & Winters, J. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.00062 · Cited by 136
Shen, H., Deng, W. H., Chattopadhyay, A., Wu, Z. S., Wang, X., & Zhu, H. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445971 · Cited by 71
Ma, Z., Mei, Y., Long, Y., Su, Z., & Gajos, K. Z. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642482 · Cited by 91
A Large-scale Synthetic Pathological Dataset for Deep Learning-enabled Segmentation of Breast Cancer
Ding, K., Zhou, M., Wang, H., Gevaert, O., Metaxas, D., & Zhang, S. (2023) · Scientific Data · doi:10.1038/s41597-023-02125-y · Cited by 71
The poverty of ethical AI: impact sourcing and AI supply chains
Muldoon, J., Cant, C., Graham, M., & Ustek Spilda, F. (2025) · AI & SOCIETY · doi:10.1007/s00146-023-01824-9 · Cited by 88
REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets
Wang, A., Narayanan, A., & Russakovsky, O. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_43 · Cited by 73
Beyond explainability: justifiability and contestability of algorithmic decision systems
Henin, C., & Le Métayer, D. (2022) · AI & SOCIETY · doi:10.1007/s00146-021-01251-8 · Cited by 71
Whose AI Dream? In search of the aspiration in data annotation.
Wang, D., Prabhat, S., & Sambasivan, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502121 · Cited by 70
Palladino, N. (2022) · Telecommunications Policy · doi:10.1016/j.telpol.2022.102479 · Cited by 49
Application and Construction of Deep Learning Networks in Medical Imaging
Torres-Velázquez, M., Chen, W., Li, X., & McMillan, A. B. (2020) · IEEE Transactions on Radiation and Plasma Medical Sciences · doi:10.1109/trpms.2020.3030611 · Cited by 74
Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing
Raji, I. D., Gebru, T., Mitchell, M., Buolamwini, J., Lee, J., & Denton, E. (2020) · arXiv · doi:10.48550/arxiv.2001.00964 · Cited by 69
Self-citatione-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Tasks
Kayser, M., Camburu, O.-M., Salewski, L., Emde, C., Do, V., Akata, Z., & Lukasiewicz, T. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00128 · Cited by 68
Data Preparation for Software Vulnerability Prediction: A Systematic Literature Review
Croft, R., Xie, Y., & Babar, M. A. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3171202 · Cited by 68
Toward Explainable Affective Computing: A Review
Cortiñas-Lorenzo, K., & Lacey, G. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3270027 · Cited by 87
Initial policy considerations for generative artificial intelligence
Lorenz, P., Perset, K., & Berryhill, J. (2023) · OECD artificial intelligence papers · doi:10.1787/fae2d1e6-en · Cited by 123
Rethinking open source generative AI: open washing and the EU AI Act
Liesenfeld, A., & Dingemanse, M. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3659005 · Cited by 123
Ethics and society review: Ethics reflection as a precondition to research funding
Bernstein, M. S., Levi, M., Magnus, D. C., Rajala, B., Satz, D. M., & Waeiss, Q. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2117261118 · Cited by 71
A Silicon Valley love triangle: Hiring algorithms, pseudo-science, and the quest for auditability
Sloane, M., Moss, E., & Chowdhury, R. (2022) · Patterns · doi:10.1016/j.patter.2021.100425 · Cited by 81
Building Trustworthy AI Solutions: A Case for Practical Solutions for Small Businesses
Crockett, K. A., Colyer, E., Gerber, L., & Latham, A. (2021) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2021.3137091 · Cited by 87
Singh, H., Mhasawade, V., & Chunara, R. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000023 · Cited by 67
Evaluation Gaps in Machine Learning Practice
Hutchinson, B., Rostamzadeh, N., Greer, C., Heller, K., & Prabhakaran, V. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533233 · Cited by 97
Stalnaker, T., Wintersgill, N., Chaparro, O., Penta, M. D., Germán, D. M., & Poshyvanyk, D. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3623347 · Cited by 66
Mitigating Bias in Algorithmic Hiring: Evaluating Claims and Practices
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603203 · Cited by 67
Hard choices in artificial intelligence
Dobbe, R., Krendl Gilbert, T., & Mintz, Y. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103555 · Cited by 65
Problems with Cosine as a Measure of Embedding Similarity for High Frequency Words
Zhou, K., Ethayarajh, K., Card, D., & Jurafsky, D. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-short.45 · Cited by 99
Artificial intelligence bias in medical system designs: a systematic review
Kumar, A., Aelgani, V., Vohra, R., Gupta, S. K., Bhagawati, M., Paul, S., Saba, L., Suri, N., Khanna, N. N., Laird, J. R., et al. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16029-x · Cited by 65
Principles to Practices for Responsible AI: Closing the Gap
Schiff, D., Rakova, B., Ayesh, A., Fanti, A., & Lennon, M. (2020) · arXiv · doi:10.48550/arxiv.2006.04707
Medical artificial intelligence for clinicians: the lost cognitive perspective
Tikhomirov, L., Semmler, C., McCradden, M. D., Searston, R. A., Ghassemi, M., & Oakden‐Rayner, L. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00095-5 · Cited by 69
Schramowski, P., Tauchmann, C., & Kersting, K. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533192 · Cited by 86
Groh, M., Harris, C., Daneshjou, R., Badri, O., & Koochek, A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555634 · Cited by 82
Trust criteria for artificial intelligence in health: normative and epistemic considerations
Kostick, K. M., Lang, B. H., Smith, J. N., Hurley, M. E., & Blumenthal‐Barby, J. (2023) · Journal of Medical Ethics · doi:10.1136/jme-2023-109338 · Cited by 70
Artificial Intelligence and Deep Learning in Neuroradiology: Exploring the New Frontier
Kaka, H., Zhang, E., & Khan, N. M. (2020) · Canadian Association of Radiologists Journal · doi:10.1177/0846537120954293 · Cited by 69
Visual Semantic Role Labeling for Video Understanding
Sadhu, A., Gupta, T., Yatskar, M., Nevatia, R., & Kembhavi, A. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00554 · Cited by 63
TROMPA-MER: an open dataset for personalized music emotion recognition
Gómez-Cañón, J., Gutiérrez-Páez, N., Porcaro, L., Porter, A., Cano, E., Herrera-Boyer, P., Gkiokas, A., Santos, P., Hernández-Leo, D., Karreman, C., et al. (2022) · Journal of Intelligence and Information Systems · doi:10.1007/s10844-022-00746-0 · Cited by 44
Artifical Intelligence and Bias: Challenges, Implications, and Remedies
Min, A. (2023) · Journal Of Social Research · doi:10.55324/josr.v2i11.1477 · Cited by 87
Dell, M. (2025) · Journal of Economic Literature · doi:10.1257/jel.20241733 · Cited by 72
You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings
Talat, Z., Névéol, A., Biderman, S., Clinciu, M., Dey, M., Longpre, S., Luccioni, A. S., Masoud, M., Mitchell, M., Radev, D., et al. (2022) · doi:10.18653/v1/2022.bigscience-1.3 · Cited by 77
Kawakami, A., Chowdhary, S., Iqbal, S. T., Liao, Q. V., Olteanu, A., Suh, J., & Saha, K. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610207 · Cited by 79
Heger, A., Marquis, L. B., Vorvoreanu, M., Wallach, H., & Vaughan, J. W. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555760 · Cited by 60
Self-citationFairness in Deep Learning: A Survey on Vision and Language Research
Parraga, O., Móre, M. D., Mattjie, C., Gavenski, N. S., Kupssinskü, L. S., Medronha, A., de Moura, L. V., Simões, G. S., & Barros, R. C. (2023) · ACM Computing Surveys · doi:10.1145/3637549 · Cited by 82
Towards an AI-Enhanced Cyber Threat Intelligence Processing Pipeline
Alevizos, L., & Dekker, M. (2024) · Electronics · doi:10.3390/electronics13112021 · Cited by 83
Reducing AI bias in recruitment and selection: an integrative grounded approach
Soleimani, M., Intezari, A. H., Arrowsmith, J., Pauleen, D. J., & Taşkın, N. (2025) · The International Journal of Human Resource Management · doi:10.1080/09585192.2025.2480617 · Cited by 101
Lekadir, K., Osuala, R., Gallin, C., Lazrak, N., Kushibar, K., Tsakou, G., Aussó, S., Alberich, L. C., Marias, K., Tsiknakis, M., et al. (2021) · arXiv · doi:10.48550/arxiv.2109.09658 · Cited by 58
Facilitating Knowledge Sharing from Domain Experts to Data Scientists for Building NLP Models
Park, S., Wang, A. Y., Kawas, B., Liao, Q. V., Piorkowski, D., & Danilevsky, M. (2021) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3397481.3450637 · Cited by 70
Community-in-the-loop: towards pluralistic value creation in AI, or—why AI needs business ethics
Häußermann, J. J., & Lütge, C. (2021) · AI and Ethics · doi:10.1007/s43681-021-00047-2 · Cited by 64
Making It Possible for the Auditing of AI: A Systematic Review of AI Audits and AI Auditability
Li, Y., & Goel, S. (2024) · Information Systems Frontiers · doi:10.1007/s10796-024-10508-8 · Cited by 65
Software Architecture Challenges for ML Systems
Lewis, G. A., Özkaya, İ., & Xu, X. (2021) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme52107.2021.00071 · Cited by 57
Fair ranking: a critical review, challenges, and future directions
Patro, G. K., Porcaro, L., Mitchell, L., Zhang, Q., Zehlike, M., & Garg, N. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533238 · Cited by 83
Feature Guided Masked Autoencoder for Self-Supervised Learning in Remote Sensing
Wang, Y., Hernández, H. H., Albrecht, C. M., & Zhu, X. X. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3493237 · Cited by 80
Teach Me to Explain: A Review of Datasets for Explainable NLP.Preprint
Wiegreffe, S., & Marasović, A. (2021) · arXiv (Cornell University) · Cited by 57
DynaSent: A Dynamic Benchmark for Sentiment Analysis
Potts, C., Wu, Z., Geiger, A., & Kiela, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.186 · Cited by 56
Li, Y., & Goel, S. (2025) · International Journal of Accounting Information Systems · doi:10.1016/j.accinf.2025.100739 · Cited by 72
AI Documentation: A path to accountability
Königstorfer, F., & Thalmann, S. (2022) · Journal of Responsible Technology · doi:10.1016/j.jrt.2022.100043 · Cited by 55
Behavioral Use Licensing for Responsible AI
Contractor, D., McDuff, D., Haines, J. K., Lee, J., Hines, C., Hecht, B., Vincent, N., & Li, H. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533143 · Cited by 54
Zheng, C., Wang, D., Wang, A. Y., & Ma, X. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517615 · Cited by 54
Kawakami, A., Coston, A., Zhu, H., Heidari, H., & Holstein, K. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642849 · Cited by 54
Risk of Training Diagnostic Algorithms on Data with Demographic Bias
Abbasi-Sureshjani, S., Raumanns, R., Michels, B. E. J., Schouten, G., & Cheplygina, V. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61166-8_20 · Cited by 54
Modeling community standards for metadata as templates makes data FAIR
Musen, M. A., O’Connor, M. J., Schultes, E., Martínez-Romero, M., Hardi, J., & Graybeal, J. B. (2022) · Scientific Data · doi:10.1038/s41597-022-01815-3 · Cited by 60
Assessing and addressing algorithmic bias in practice
Cramer, H. S. M., Garcia-Gathright, J. I., Springer, A., & Reddy, S. (2018) · interactions · doi:10.1145/3278156 · Cited by 53
Teach Me to Explain: A Review of Datasets for Explainable Natural Language ProcessingPreprint
Wiegreffe, S., & Marasović, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.12060 · Cited by 128
Implementing Machine Learning in Interventional Cardiology: The Benefits Are Worth the Trouble
Ali, W. B., Pesaranghader, A., Avram, R., Overtchouk, P., Perrin, N., Laffite, S., Cartier, R., Ibrahim, R., Modine, T., & Hussin, J. (2021) · Frontiers in Cardiovascular Medicine · doi:10.3389/fcvm.2021.711401 · Cited by 56
El–Sappagh, S., Alonso, J. M., Abuhmed, T., Ali, F., & Bugarín, A. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10415-5 · Cited by 58
Transparency of medical artificial intelligence systems
Kim, C., Gadgil, S. U., & Lee, S. (2025) · Nature Reviews Bioengineering · doi:10.1038/s44222-025-00363-w · Cited by 60
Representativeness in Statistics, Politics, and Machine Learning
Chasalow, K., & Levy, K. E. C. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445872 · Cited by 61
Fair Generative Modeling via Weak SupervisionPreprint
Choi, K., Grover, A., Singh, T., Shu, R., & Ermon, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.12008 · Cited by 58
Changing the World by Changing the Data
Rogers, A. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.170 · Cited by 51
Intrinsic Evaluation of Summarization Datasets
Bommasani, R., & Cardie, C. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.649 · Cited by 60
Zeno: An Interactive Framework for Behavioral Evaluation of Machine Learning
Cabrera, Á. A., Fu, E., Bertucci, D., Holstein, K., Talwalkar, A., Hong, J. I., & Perer, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581268 · Cited by 51
La Fors, K., Custers, B., & Keymolen, E. (2019) · Ethics and Information Technology · doi:10.1007/s10676-019-09503-4 · Cited by 50
Bad Seeds: Evaluating Lexical Methods for Bias Measurement
Antoniak, M., & Mimno, D. (2021) · doi:10.18653/v1/2021.acl-long.148 · Cited by 50
An Ethical Framework for Guiding the Development of Affectively-Aware Artificial Intelligence
Ong, D. C. (2021) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii52823.2021.9597441 · Cited by 54
Towards training reproducible deep learning models
Chen, B., Wen, M., Shi, Y., Lin, D., Rajbahadur, G. K., & Jiang, Z. M. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510163 · Cited by 66
Christoforaki, M., & Beyan, O. (2022) · Applied Sciences · doi:10.3390/app12094130 · Cited by 64
A hunt for the Snark: Annotator Diversity in Data Practices
Kapania, S., Taylor, A. S., & Wang, D. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580645 · Cited by 50
AI image generators often give racist and sexist results: can they be fixed?
Ananya, A. (2024) · Nature · doi:10.1038/d41586-024-00674-9 · Cited by 50
Enhancing precision agriculture through cloud based transformative crop recommendation model
Singh, G., & Sharma, S. (2025) · Scientific Reports · doi:10.1038/s41598-025-93417-3 · Cited by 50
Large Language Models for Text Classification: From Zero-Shot Learning to Instruction-Tuning
Chae, Y., & Davidson, T. (2025) · Sociological Methods & Research · doi:10.1177/00491241251325243 · Cited by 72
Biological research and self-driving labs in deep space supported by artificial intelligence
Sanders, L., Scott, R. T., Yang, J. H., Qutub, A. A., Martín, H. G., Berrios, D. C., Hastings, J. J. A., Rask, J. C., Mackintosh, G., Hoarfrost, A., et al. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00618-4 · Cited by 59
Certification Labels for Trustworthy AI: Insights From an Empirical Mixed-Method Study
Scharowski, N., Benk, M., Kühne, S. J., Wettstein, L., & Brühlmann, F. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3593994 · Cited by 61
A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities
Tocchetti, A., Corti, L., Balayn, A., Yurrita, M., Lippmann, P., Brambilla, M., & Yang, J. (2024) · ACM Computing Surveys · doi:10.1145/3665926 · Cited by 77
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
Gehrmann, S., Adewumi, T., Aggarwal, K., Ammanamanchi, P. S., Anuoluwapo, A., Bosselut, A., Chandu, K. R., Clinciu, M., Das, D., Dhole, K. D., et al. (2021) · arXiv · doi:10.48550/arxiv.2102.01672 · Cited by 49
Symphony: Composing Interactive Interfaces for Machine Learning
Bäuerle, A., Cabrera, Á. A., Hohman, F., Maher, M., Koski, D., Suau, X., Barik, T., & Moritz, D. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502102 · Cited by 54
Chen, J., Shao, Z., Zheng, X., Zhang, K., & Yin, J. (2024) · Scientific Reports · doi:10.1038/s41598-024-53318-3 · Cited by 64
Analyzing Dataset Annotation Quality Management in the Wild
Klie, J.-C., de Castilho, R. E., & Gurevych, I. (2024) · Computational Linguistics · doi:10.1162/coli_a_00516 · Cited by 66
Web scraping for research: Legal, ethical, institutional, and scientific considerations
Brown, M. A., Gruen, A., Maldoff, G., Messing, S., Sanderson, Z., & Zimmer, M. T. (2025) · Big Data & Society · doi:10.1177/20539517251381686 · Cited by 48
Dehghani, M., Tay, Y., Gritsenko, A. A., Zhao, Z., Houlsby, N., Diaz, F., Metzler, D., & Vinyals, O. (2021) · arXiv · doi:10.48550/arxiv.2107.07002 · Cited by 48
How Platform-User Power Relations Shape Algorithmic Accountability
Ramesh, D., Kameswaran, V., Wang, D., & Sambasivan, N. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533237 · Cited by 61
Whose side are ethics codes on?: power, responsibility and the social good
Washington, A., & Kuo, R. (2020) · FAT* · doi:10.1145/3351095.3372844 · Cited by 34
Blagec, K., Kraiger, J., Frühwirt, W., & Samwald, M. (2023) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2022.104274 · Cited by 47
Racial Bias within Face Recognition: A Survey
Yücer, Ş., Tektas, F., Moubayed, N. A., & Breckon, T. P. (2024) · ACM Computing Surveys · doi:10.1145/3705295 · Cited by 47
ProtoQA: A Question Answering Dataset for Prototypical Common-Sense Reasoning
Boratko, M., Li, X. L., O’Gorman, T., Das, R., Le, D., & McCallum, A. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.85 · Cited by 56
Metadata
- Authors
- Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K.
- Affiliations
- DAIR Institute (US)
- University of Washington (US)
- Cornell University (US)
- Microsoft Research New York City (United States) (US)
- University of Maryland, College Park (US)
- University of Southern California (US)
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- Volume
- 64
- Issue
- 12
- Pages
- 86-92
- ISSN
- 0001-0782 (ISSN-L)
- arXiv
- 1803.09010
- OpenAlex
- W2795038878
- Cited by
- 2,474
- References
- 37
- Citations by year
- 2026: 795 · 2025: 402 · 2024: 298 · 2023: 379 · 2022: 221 · 2021: 216 · 2020: 91 · 2019: 60 · 2018: 11
- FWCI
- 251.78
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Communications of the ACM (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation
- Alfred P. Sloan Foundation
- Topics
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Physical Sciences › Computer Science › Artificial Intelligence › Semantic Web and Ontologies
- Keywords
- dataset documentation, data provenance, data sharing, data reuse, metadata
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
