Data excellence for AI: why should you care?
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
This forum provides a space to engage with the challenges of designing for intelligent algorithmic experiences. We invite articles that tackle the tensions between research and practice when integrating AI and UX design. We welcome interdisciplinary debate, artful critique, forward-looking research, case studies of AI in practice, and speculative design explorations. --- Juho Kim and Henriette Cramer, Editors
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“The UK government has recently published an Algorithmic Transparency Standard 3 alongside templates designed to help public sector organisations to document the datasets that underlie their ML tools.”
From A Survey of Data Quality Requirements That Matter in ML Development Pipelines (2023) · cites this paper as [3]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Software Engineering for Machine Learning: A Case Study
Amershi, S., Begel, A., Bird, C., DeLine, R., Gall, H., Kamar, E., Nagappan, N., Nushi, B., & Zimmermann, T. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00042 · Cited by 1,135
“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI
Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445518 · Cited by 781
Self-citationMachine learning training to support diversity of opinionPreprint
Christensen, J., & Watson, B. A. (2021) · doi:10.31219/osf.io/m6fdg · Cited by 1
Reducing Annotation Artifacts in Crowdsourcing Datasets for Natural Language Processing
Anonymous, A. (2020) · Figshare · doi:10.6084/m9.figshare.12962480.v1 · Cited by 1
Reducing Annotation Artifacts in Crowdsourcing Datasets for Natural Language Processing
Anonymous, A. (2020) · Figshare · doi:10.6084/m9.figshare.12962480 · Cited by 1
Research works printed but not linked: 2
- Aroyo , L. and Paritosh , P . Uncovering unknown unknowns in machine learning. Google AI blog . Feb. 11, 2021 ; https://ai.googleblog.com/2021/02/uncovering-unknown-unknowns-in-machine.html Aroyo, L. and Paritosh, P. Uncovering unknown unknowns in machine learning. Google AI blog. Feb. 11, 2021; https://ai.googleblog.com/2021/02/uncovering-unknown-unknowns-in-machine.html
- Kapania , S. , Sambasivan , N. , Olson , K. , Highfill , H. , Akrong , D. , Paritosh , P. , and Aroyo , L . Data desiderata: Reliability and fidelity in high-stakes AI . 1st Data Excellence Workshop at HCOMP 2020 . Kapania, S., Sambasivan, N., Olson, K., Highfill, H., Akrong, D., Paritosh, P., and Aroyo, L. Data desiderata: Reliability and fidelity in high-stakes AI. 1st Data Excellence Workshop at HCOMP 2020.
Cited by
Data-centric Artificial Intelligence: A Survey
Zha, D., Bhat, Z. P., Lai, K.-H., Yang, F., Jiang, Z., Zhong, S., & Hu, X. (2025) · ACM Computing Surveys · doi:10.1145/3711118 · Cited by 334
The Principles of Data-Centric AI
Jarrahi, M. H., Memariani, A., & Guha, S. (2023) · Communications of the ACM · doi:10.1145/3571724 · Cited by 204
DataPerf: Benchmarks for Data-Centric AI Development
Mazumder, M., Banbury, C., Yao, X., Karlaš, B., Rojas, W. G., Diamos, S., Diamos, G., He, L., Parrish, A., Kirk, H. R., et al. (2022) · TU/e Research Portal · doi:10.48550/arxiv.2207.10062 · Cited by 138
Self-citationA Survey of Data Quality Requirements That Matter in ML Development Pipelines
Priestley, M., O’donnell, F., & Simperl, E. (2023) · Journal of Data and Information Quality · doi:10.1145/3592616 · Cited by 86
Better, Not Just More: Data-centric machine learning for Earth observation
Roscher, R., Rußwurm, M., Gevaert, C. M., Kampffmeyer, M., dos Santos, J. A., Vakalopoulou, M., Hänsch, R., Hansen, S., Nogueira, K., Prexl, J., et al. (2024) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2024.3470986 · Cited by 65
Balayn, A., Yurrita, M., Yang, J., & Gadiraju, U. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604674 · Cited by 63
Braylan, A., Alonso, O., & Lease, M. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512242 · Cited by 52
Self-citationData-related concepts for artificial intelligence education in K-12
Olari, V., & Romeike, R. (2024) · Computers and Education Open · doi:10.1016/j.caeo.2024.100196 · Cited by 39
Hettiachchi, D., Holcombe-James, I., Livingstone, S., de Silva, A., Lease, M., Salim, F. D., & Sanderson, M. (2023) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v11i1.27546 · Cited by 29
Self-citationMosqueira-Rey, E., Hernández-Pereira, E., Bobes-Bascarán, J., Alonso-Ríos, D., Pérez-Sánchez, A., Fernández-Leal, Á., Moret‐Bonillo, V., Vidal-Ínsua, Y., & Vázquez-Rivera, F. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-09197-2 · Cited by 26
Supporting Co-Adaptive Machine Teaching through Human Concept Learning and Cognitive Theories
Gebreegziabher, S. A., Yang, Y., Glassman, E. L., & Li, T. J.-J. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713708 · Cited by 22
A Bag-of-Prototypes Representation for Dataset-Level Applications
Tu, W., Deng, W., Gedeon, T., & Zheng, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00282 · Cited by 19
Data-related practices for creating Artificial Intelligence systems in K-12
Olari, V., & Romeike, R. (2024) · Workshop in Primary and Secondary Computing Education · doi:10.1145/3677619.3678115 · Cited by 18
Unlocking the Tacit Knowledge of Data Work in Machine Learning
Cha, I., Oh, J., Park, C. Y., Han, J., & Lee, H. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3585616 · Cited by 15
Herrera-Poyatos, D., Herrera-Poyatos, A., Soldado, R. M., de Palacios, P., Esteban, L. G., García‐Iruela, A., Fernández, F. G., & Herrera, F. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650590 · Cited by 14
Data-Centric Digital Agriculture: A PerspectivePreprint
Roscher, R., Roth, L., Stachniss, C., & Walter, A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.03437 · Cited by 14
A two-stage data quality improvement strategy for deep neural networks in fault severity estimation
Yao, Y., Wu, L., Xie, B., Li, L., Wang, Z., & Li, Y. (2023) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2023.110588 · Cited by 13
fastMONAI: A low-code deep learning library for medical image analysis
Kaliyugarasan, S., & Lundervold, A. S. (2023) · Software Impacts · doi:10.1016/j.simpa.2023.100583 · Cited by 8
LMDG: Advancing Lateral Movement Detection Through High-Fidelity Dataset GenerationPreprint
Mabrouk, A., Hatem, M., Mamun, M., & Saad, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.02942 · Cited by 8
Braylan, A., Marabella, M., Alonso, O., & Lease, M. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14388 · Cited by 7
Self-citationFiltering Matters: Experiments in Filtering Training Sets for Machine Translation
Steingrímsson, S., Loftsson, H., & Way, A. (2023) · DSpace repository (University of Tartu) · Cited by 6
Better, Not Just More: Data-centric machine learning for Earth observation
Roscher, R., Rußwurm, M., Gevaert, C. M., Kampffmeyer, M., dos Santos, J. A., Vakalopoulou, M., Hänsch, R., Hansen, S., Nogueira, K., Prexl, J., et al. (2025) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2025.3551556 · Cited by 4
Improving Label Error Detection and Elimination with Uncertainty Quantification
Jakubik, J., Vössing, M., Maskey, M., Wölfle, C., & Satzger, G. (2025) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.16158 · Cited by 4
Peat-DBase v.1: a compiled database of global peat depth measurements
Skye, J., Melton, J. R., Goldblatt, C., Saumier, L., Gallego-Sala, A. V., Garneau, M., Winton, R. S., Bahati, E. B., Benavides, J. C., Fedorchuk, L., et al. (2025) · Earth system science data · doi:10.5194/essd-17-7313-2025 · Cited by 4
Datactive: Data Fault Localization for Object Detection Systems
Yin, Y., Feng, Y., Weng, S., Yao, Y., Liu, J., & Zhao, Z. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3680329 · Cited by 4
Choudhary, S., & Janarthanan, M. (2025) · doi:10.48001/978-81-980647-1-4-13 · Cited by 4
Brokering Ethics: A Retrospective Study on the Use of a Data Ethics Framework
Worth, S., Panagiotidou, G., & Simperl, E. P. B. (2025) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1609/aies.v8i3.36753 · Cited by 3
Xintong, M., Zhu, T., Zhong, B., Wu, W., Naying, L., Yajing, W., Liu, X., Ruixi, L., Abdul, R., Shaoliang, Y., et al. (2026) · Journal of Hazardous Materials · doi:10.1016/j.jhazmat.2026.141233 · Cited by 3
Core-Set Selection for Data-Efficient Land Cover Segmentation
Nogueira, K., Zaytar, A., Ma, W., Roscher, R., Hänsch, R., Robinson, C., Ortiz, A., Fobi, S., Dodhia, R., Ferres, J. L., et al. (2026) · IEEE Access · doi:10.1109/access.2026.3659734 · Cited by 3
Mahalle, P. N., Shinde, G. R., Ingle, Y. S., & Wasatkar, N. N. (2023) · Data-intensive research · doi:10.1007/978-981-99-6353-9_6 · Cited by 3
Striving for data-model efficiency: Identifying data externalities on group performancePreprint
Rolf, E., Packer, B., Beutel, A., & Díaz, F. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.06348 · Cited by 3
Cha, I., Oh, J., Park, C. Y., Han, J., & Lee, H. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.14981 · Cited by 3
Class-Adaptive Sampling Policy for Efficient Continual LearningPreprint
Rezaei, H., & Sabokrou, M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.16485 · Cited by 3
Herrera-Poyatos, D., Herrera-Poyatos, A., Montes, R., de Palacios, P., Esteban, L. G., García‐Iruela, A., Fernández, F. G., & Herrera, F. (2026) · Applied Intelligence · doi:10.1007/s10489-025-06731-8 · Cited by 2
DQM: Data Quality Metrics for AI Components in the Industry
Chaouche, S., Randon, Y., Adjed, F., Boudjani, N., & Khedher, M. I. (2024) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v4i1.31767 · Cited by 2
Editorial: Human-centered AI: Crowd computing
Yang, J., Bozzon, A., Gadiraju, U., & Lease, M. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1161006 · Cited by 2
Self-citationImproving Label Error Detection and Elimination with Uncertainty QuantificationPreprint
Jakubik, J., Vössing, M., Maskey, M., Wölfle, C., & Satzger, G. (2024) · Repository KITopen (Karlsruhe Institute of Technology) · doi:10.48550/arxiv.2405.09602 · Cited by 2
Continual Learning: Less Forgetting, More OOD Generalization via Adaptive Contrastive ReplayPreprint
Rezaei, H., & Sabokrou, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.07110 · Cited by 2
What's the next frontier for Data-centric AI? Data Savvy AgentsPreprint
Seedat, N., Liu, J., & van der Schaar, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.01015 · Cited by 2
Data-Centric AI for Tropical Agricultural Mapping: Challenges, Strategies and Scalable SolutionsPreprint
da Silva, M. P., Correa, S. P. L. P., Oliveira, H., Nunes, I. M., & Santos, J. A. D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.16207 · Cited by 2
Effectively compiling parallel corpora for machine translation in resource-scarce conditions
Steingrímsson, S. (2023) · Opin vísindi (Opin vísindi) · Cited by 2
Binary Classification of Medical Images by Symbolic Regression
Allison, E. (2024) · Advances in intelligent systems and computing · doi:10.1007/978-3-031-47508-5_40 · Cited by 1
Mneme: A Parallel Preprocessing Framework for Large Tabular Datasets
Sofotasios, A., Metaxakis, D., & Hadjidoukas, P. E. (2026) · International Journal of Parallel Programming · doi:10.1007/s10766-026-00817-7 · Cited by 1
Block-Aware Distributed Data Pipelines for Out-of-Core Tabular Machine Learning
Metaxakis, D., Sofotasios, A., Stylios, C. D., & Hadjidoukas, P. E. (2026) · Workshop on Machine Learning and Systems (EuroMLSys) · doi:10.1145/3805621.3807646 · Cited by 1
Leveraging Variational Autoencoder for Improved Construction Progress Prediction Performance
Mostofi, F., Tokdemir, O. B., & Toğan, V. (2024) · Lecture notes in civil engineering · doi:10.1007/978-981-97-4355-1_51 · Cited by 1
Wang, Y., Irion, K., Groth, P., & Harmouch, H. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.06594 · Cited by 1
Management in the Age of AI: Strategic Synergies Across Industries
Som, B. K., & Tiwari, S. (2025) · doi:10.48001/978-81-980647-1-4
Bauscher, E., & Wortmann, T. (2025) · International Journal of Architectural Computing · doi:10.1177/14780771251353797
The Artifice of Intelligence: Libraries and Memory
Schissel, P. (2024) · Carolina Digital Repository (University of North Carolina at Chapel Hill) · doi:10.17615/s843-6b98
Lim, W. X. (2024) · Nottingham ePrints (University of Nottingham)
Wolf, H. (2023) · ePublication (TH Cologne)
Ignatov, A. (2026) · National Interests Priorities and Security · doi:10.24891/ioajhi
Data Representativeness with Hyperdimensional Computing
Burgon, A., Petrick, N., Krainak, D. M., Khan, A., & Samala, R. K. (2026) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-026-01917-z
Data curation strategies for geospatial machine learning: a case study in peatland classification
Saumier, L., Melton, J. R., & Winton, R. S. (2026) · Machine Learning Earth · doi:10.1088/3049-4753/ae8304
Singamaneni, C. R., & Rakesh, S. (2026) · Discover Computing · doi:10.1007/s10791-026-10515-0
Toward Data-Centric Deep Learning for Adaptive Optimization in Large-Scale Systems
Pembroke, A. (2024) · Journal of Computer Science and Software Applications
Metadata
- Authors
- Aroyo, L., Lease, M., Paritosh, P., & Schaekermann, M.
- Affiliations
- Google (United States) (US)
- The University of Texas at Austin (US)
- Amazon (United States) (US)
- Venue
- Interactions
- Published
- 2022-02-25
- Volume
- 29
- Issue
- 2
- Pages
- 66-69
- ISSN
- 1072-5520 (ISSN-L)
- 1558-3449
- OpenAlex
- W4214634229
- Cited by
- 31
- References
- 4
- Citations by year
- 2026: 8 · 2025: 5 · 2024: 7 · 2023: 10 · 2022: 1
- FWCI
- 2.84
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- interactions (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- speculative design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.