“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI
Open accessThe DOI matches its Crossref record; 100 of the 104 references it cites were found in a registry or an index.
Abstract
AI models are increasingly applied in high-stakes domains like health and conservation. Data quality carries an elevated significance in high-stakes AI due to its heightened downstream impact, impacting predictions like cancer detection, wildlife poaching, and loan allocations. Paradoxically, data is the most under-valued and de-glamorised aspect of AI. In this paper, we report on data practices in high-stakes AI, from interviews with 53 AI practitioners in India, East and West African countries, and USA. We define, identify, and present empirical evidence on Data Cascades—compounding events causing negative, downstream effects from data issues—triggered by conventional AI/ML practices that undervalue data quality. Data cascades are pervasive (92% prevalence), invisible, delayed, but often avoidable. We discuss HCI opportunities in designing and incentivizing data excellence as a first-class citizen of AI, resulting in safer and more robust systems for all.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“There has also been an increased focus on analyzing other dimensions of data quality and stewardship [102, 107, 121, 123], with several noteworthy initiatives aiming to document both existing [9, 20, 30, 43], and newly developed [16, 60, 136] resources.”
From Data Governance in the Age of Large-Scale Data-Driven Language Technology (2022) · cites this paper as [123]
In the citing paper
“First, the mainstream ML research focuses predominantly on improvements to the model architecture, training procedure, and (hyper)parameters [100, 123].”
From Data Governance in the Age of Large-Scale Data-Driven Language Technology (2022) · cites this paper as [123]
In the citing paper
“Second, research addressing dataset choices, creation, and curation, is systematically 'under-valued and de-glamorised' [3, 123] 16 .”
From Data Governance in the Age of Large-Scale Data-Driven Language Technology (2022) · cites this paper as [123]
In the citing paper
“Other contextual biases have been detected in image data, with publicly available image corpora such as ImageNet and Open Images coming predominantly from amerocentric and eurocentric contexts [66].”
From A Survey of Data Quality Requirements That Matter in ML Development Pipelines (2023) · cites this paper as [66]
In the citing paper
“Despite calls for more inclusive data-centric AI (e.g., [11, 31, 53, 96, 107]), approaches that enable communities to define and curate datasets remain underexplored.”
From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [96]
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,410
A General Inductive Approach for Analyzing Qualitative Evaluation Data
Thomas, D. R. (2006) · American Journal of Evaluation · doi:10.1177/1098214005283748 · Cited by 10,858 · cited 1×
“Following [119], two members of the research team independently read all units multiple times, and categories (unit of analysis) were initially identified by each researcher, together with a description and examples of each category, until”
MethodThe multimodal brain tumor image segmentation benchmark (BRATS)
Menze, B. H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al. (2015) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2014.2377694 · Cited by 6,781 · cited 1×
“to benchmark publications and non-mainstream tracks in AI/ML conferences [46, 82].”
BackgroundA Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812 · cited 1×
“Data largely determines performance, fairness, robustness, safety, and scalability of AI systems [44, 81].”
BackgroundA Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2019) · arXiv · doi:10.48550/arxiv.1908.09635 · Cited by 5,812 · cited 1×
“Data largely determines performance, fairness, robustness, safety, and scalability of AI systems [44, 81].”
Background(2013) · Encyclopedia of Creativity, Invention, Innovation and Entrepreneurship · doi:10.1007/978-1-4614-3858-8_100431 · Cited by 4,799
Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., Luetge, C., Madelin, R., Pagallo, U., Rossi, F., et al. (2018) · Minds and Machines · doi:10.1007/s11023-018-9482-5 · Cited by 4,774
A Survey of Outlier Detection Methodologies
Hodge, V., & Austin, J. (2004) · Artificial Intelligence Review · doi:10.1007/s10462-004-4304-y · Cited by 3,596
The Unreasonable Effectiveness of Data
Halevy, A., Norvig, P., & Pereira, F. (2009) · IEEE Intelligent Systems · doi:10.1109/mis.2009.36 · Cited by 1,767 · cited 2×
“Data is the critical infrastructure necessary to build Artificial Intelligence (AI) systems [44].”
BackgroundIncreasing value and reducing waste in research design, conduct, and analysis
Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani, R. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62227-8 · Cited by 1,597 · cited 1×
“periments, and testing should also be followed with data [28, 55].”
BackgroundWhy and Where: A Characterization of Data Provenance
Buneman, P., Khanna, S., & Wang-Chiew, T. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44503-x_20 · Cited by 1,334 · cited 1×
“Data documentation, which is a crucial aspect of facilitating collaboration, is well studied in the database and data management community [19, 23].”
BackgroundConcrete Problems in AI Safety
Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016) · arXiv · doi:10.48550/arxiv.1606.06565 · Cited by 1,314 · cited 1×
“The prevalence of data cascades point to the contours of a larger problem: residual conventions and perceptions in AI/ML drawn fromworlds of ‘big data’—of abundant, expendable digital resources and worlds in which one user has one account [108]; of model valourisation [125]; of moving fast to proof-of-concept [8]; and…”
BackgroundMcDonald, N., Schoenebeck, S., & Forte, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359174 · Cited by 1,206 · cited 1×
“Since codes are our process, not product [80], IRR was not used.”
MethodSoftware Engineering for Machine Learning: A Case Study
Amershi, S., Begel, A., Bird, C., DeLine, R., Gall, H., Kamar, E., Nagappan, N., Nushi, B., & Zimmermann, T. (2019) · 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip.2019.00042 · Cited by 1,135 · cited 3×
“Multiple recent studies examine the challenges of production machine learning [6, 100, 101].”
BackgroundPower to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084 · cited 1×
“Machine Learning workflows are fundamentally iterative and exploratory in nature [7, 52, 71, 96].”
Background(2015) · neural information processing systems · Cited by 912
Machine Learning Testing: Survey, Landscapes and Horizons
Zhang, J. M., Harman, M., Ma, L., & Liu, Y. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2962027 · Cited by 897
Deep Learning: A Critical Appraisal
Marcus, G. (2018) · arXiv · doi:10.48550/arxiv.1801.00631 · Cited by 853 · cited 1×
“New AI models are measured against large, curated data sets that lack noise (to report high performances), in contrast to the dynamic nature of the real world [64, 78].”
BackgroundTurkopticon: interrupting worker invisibility in amazon mechanical turk
Irani, L. C., & Silberman, M. S. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2470742 · Cited by 798 · cited 1×
“Learning from HCI scholarship on ways to recognise the human labour in preparing, curating, and nurturing data that powers AI models [34, 117], among crowd workers [34, 57, 79], office clerks [85], and health workers [58] can be helpful.”
BackgroundPennebaker, J. W. (2011) · New Scientist · doi:10.1016/s0262-4079(11)62167-2 · Cited by 780
Beede, E., Baylor, E., Hersch, F., Iurchenko, A., Wilcox, L., Ruamviboonsuk, P., & Vardoulakis, L. M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376718 · Cited by 614 · cited 1×
“’s evaluation of real-world deep learning systems [17] point to the”
BackgroundApplied Machine Learning at Facebook: A Datacenter Infrastructure Perspective
Hazelwood, K., Bird, S., Brooks, D., Chintala, S., Diril, U., Dzhulgakov, D., Fawzy, M., Jia, B., Jia, Y., Kalro, A., et al. (2018) · 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2018.00059 · Cited by 590 · cited 1×
“A well-established, and steadily growing, body of work focuses on understanding and improving data quality to avoid the garbage in, garbage out problem [45, 103].”
BackgroundWrangler: interactive visual specification of data transformation scripts
Kandel, S., Paepcke, A., Hellerstein, J., & Heer, J. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979444 · Cited by 581 · cited 2×
“Interactive data cleaning focuses on making this process easier, because data transformations can be difficult to specify and reuse across multiple tasks [61, 72, 102].”
BackgroundThe cultural work of microwork
Irani, L. (2015) · New Media & Society · doi:10.1177/1461444813511926 · Cited by 568
The WyCash portfolio management system
Cunningham, W. (1993) · ACM SIGPLAN OOPS Messenger · doi:10.1145/157710.157715 · Cited by 545
Data Validation for Machine Learning
Breck, E. J., Zinkevich, M., Polyzotis, N., Whang, S. E., & Roy, S. (2019) · Cited by 510
Human-Computer Interaction (3rd Edition)
Dix, A. J., Finlay, J., Abowd, G. D., & Beale, R. (2003) · Prentice-Hall, Inc eBooks · Cited by 497
Potter's Wheel: An Interactive Data Cleaning System
Raman, V., & Hellerstein, J. L. (2001) · Cited by 492
Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance
Bansal, G., Nushi, B., Kamar, E., Lasecki, W. S., Weld, D. S., & Horvitz, E. (2019) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v7i1.5285 · Cited by 466 · cited 1×
“Currently, there are no standardised metrics for characterising the goodness-of-data [11, 13]; research on metrics is emerging [15, 91] but not yet widely adopted in AI system building.”
BackgroundEnterprise Data Analysis and Visualization: An Interview Study
Kandel, S., Paepcke, A., Hellerstein, J. M., & Heer, J. (2012) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2012.219 · Cited by 439 · cited 1×
“reveal that practitioners consider data wrangling tedious and time-consuming [62].”
BackgroundIBM Watson, heal thyself: How IBM overpromised and underdelivered on AI health care
Strickland, E. (2019) · IEEE Spectrum · doi:10.1109/mspec.2019.8678513 · Cited by 402 · cited 1×
“As an example, poor data practices reduced accuracy in IBM’s cancer treatment AI [115] and led to Google Flu Trends missing the flu peak by 140% [63, 73]).”
MethodAi for social good: unlocking the opportunity for positive impact
Tomašev, N., Cornebise, J., Hutter, F., Mohamed, S., Picciariello, A., Connelly, B., Belgrave, D. C. M., Ezer, D., Haert, F. C. V. D., Mugisha, F., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-15871-z · Cited by 357
Holistic data cleaning: Putting violations into context
Chu, X., Ilyas, I. F., & Papotti, P. (2013) · 2013 IEEE 29th International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2013.6544847 · Cited by 326 · cited 1×
“Data cleaning and wrangling systems address data quality issues by using integrity constraints [27], type inference [36], schema matching [43], outlier detection [51] and more.”
Daniel, F., Kucherbaev, P., Cappiello, C., Benatallah, B., & Allahbakhsh, M. (2019) · ACM Computing Surveys · doi:10.1145/3148148 · Cited by 313
Martin, D., Hanrahan, B. V., O'Neill, J., & Gupta, N. (2014) · Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing · doi:10.1145/2531602.2531663 · Cited by 278 · cited 3×
“[79], and among frontline health workers in India by Ismail and Kumar [58].”
BackgroundThe relevance of software documentation, tools and technologies: a survey
Forward, A., & Lethbridge, T. C. (2002) · Proceedings of the 2002 ACM symposium on Document engineering · doi:10.1145/585058.585065 · Cited by 268 · cited 1×
“Although the behaviour of AI systems is critically determined by data, even more so than code [111]; many of our practitioner strategies mirrored best practices in software engineering [38, 83].”
ResultHow do Data Science Workers Collaborate? Roles, Workflows, and Tools
Zhang, A. X., Muller, M., & Wang, D. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392826 · Cited by 268 · cited 4×
“ship have long examined the practices of collaboration, problem formulation, and sensemaking, by humans behind the datasets, including data collectors and scientists, [69, 86, 127], and are designing computational artefacts for dataset development [53].”
BackgroundInfluentialActiveClean: interactive data cleaning for statistical modeling
Krishnan, S., Wang, J., Wu, E., Franklin, M. J., & Goldberg, K. (2016) · Proceedings of the VLDB Endowment · doi:10.14778/2994509.2994514 · Cited by 264 · cited 4×
“and clean data include ActiveClean and BoostClean [70, 72].”
BackgroundInfluentialHow Data Science Workers Work with Data: Discovery, Capture, Curation, Design, Creation
Muller, M., Lange, I., Wang, D., Piorkowski, D., Tsay, J., Liao, Q. V., Dugan, C., & Erickson, T. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300356 · Cited by 261 · cited 3×
“ship have long examined the practices of collaboration, problem formulation, and sensemaking, by humans behind the datasets, including data collectors and scientists, [69, 86, 127], and are designing computational artefacts for dataset development [53].”
BackgroundScheuerman, M. K., Paul, J. M., & Brubaker, J. R. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359246 · Cited by 252
Clio grows up: from research prototype to industrial tool
Haas, L. M., Hernández, M. A., Ho, H., Popa, L., & Roth, M. (2005) · Proceedings of the 2005 ACM SIGMOD international conference on Management of data · doi:10.1145/1066157.1066252 · Cited by 249 · cited 1×
“Data cleaning and wrangling systems address data quality issues by using integrity constraints [27], type inference [36], schema matching [43], outlier detection [51] and more.”
Truth Is a Lie: Crowd Truth and the Seven Myths of Human Annotation
Aroyo, L., & Welty, C. (2015) · AI Magazine · doi:10.1609/aimag.v36i1.2564 · Cited by 244 · cited 2×
“High-stakes AI requires specialised, subjective decision-making in defning the ground truth, and breadth and number of labels [13].”
BackgroundSelf-citationLipton, Z. C., & Steinhardt, J. (2019) · Queue · doi:10.1145/3317287.3328534 · Cited by 231 · cited 1×
“Critics point to how novel model development and reward systems have reached a point of ridicule: Lipton calls ML scholarship ‘alchemy’ [74], Sculley et al.”
BackgroundRe-imagining Algorithmic Fairness in India and Beyond
Sambasivan, N., Arnesen, E., Hutchinson, B., Doshi, T., & Prabhakaran, V. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445896 · Cited by 228 · cited 2×
“, [31, 98, 107], understanding socio-cultural factors e.”
BackgroundSelf-citationMaking Better Use of the Crowd: How Crowdsourcing Can Advance Machine Learning Research
Vaughan, J. W. (2018) · Journal of Machine Learning Research · Cited by 227
Passi, S., & Jackson, S. J. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274405 · Cited by 225
Revolt: Collaborative Crowdsourcing for Labeling Machine Learning Datasets
Chang, J. C., Amershi, S., & Kamar, E. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3026044 · Cited by 217 · cited 1×
“in making use of crowdsourcing to generate ML data [25, 128], to support creation of better task designs for raters [59], compute interrater reliability, design incentives [50], and improve the quality of crowdsourced data [30], though these areas are less well known in the ML community [122].”
Data Lifecycle Challenges in Production Machine Learning: A Survey
Polyzotis, N., Roy, S., Whang, S. E., & Zinkevich, M. (2018) · ACM SIGMOD Record · doi:10.1145/3299887.3299891 · Cited by 214 · cited 2×
“Multiple recent studies examine the challenges of production machine learning [6, 100, 101].”
BackgroundData Management Challenges in Production Machine Learning
Polyzotis, N., Roy, S., Whang, S. E., & Zinkevich, M. (2017) · Proceedings of the 2017 ACM International Conference on Management of Data · doi:10.1145/3035918.3054782 · Cited by 208 · cited 3×
“Multiple recent studies examine the challenges of production machine learning [6, 100, 101].”
BackgroundLearning from crowdsourced labeled data: a survey
Zhang, J., Wu, X., & Sheng, V. S. (2016) · Artificial Intelligence Review · doi:10.1007/s10462-016-9491-9 · Cited by 203 · cited 1×
“in making use of crowdsourcing to generate ML data [25, 128], to support creation of better task designs for raters [59], compute interrater reliability, design incentives [50], and improve the quality of crowdsourced data [30], though these areas are less well known in the ML community [122].”
BackgroundData scientists in software teams: State of the art and challenges
Kim, M., Zimmermann, T., DeLine, R., & Begel, A. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2754374 · Cited by 199
The Politics of Measurement and Action
Pine, K. H., & Liboiron, M. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702298 · Cited by 194 · cited 3×
“Through their study of public health data, Pine and Liboiron [99] demonstrate how data collection is shaped by values and decisions about “what is counted and what is excluded, and what is considered the best unit of measurement.”
BackgroundInfluentialIntegrating Ethics within Machine Learning Courses
Saltz, J., Skirpan, M., Fiesler, C., Gorelick, M., Yeh, T., Heckman, R., Dewar, N., & Beard, N. (2019) · ACM Transactions on Computing Education · doi:10.1145/3341164 · Cited by 180
Bechmann, A., & Bowker, G. C. (2019) · Big Data & Society · doi:10.1177/2053951718819569 · Cited by 162 · cited 1×
“Critical data studies researchers have demonstrated longstanding interest in the ‘discretionary’ [95] practices shaping data-driven systems and how they are designed and used [6, 16, 33], and the”
BackgroundData-in-Place: Thinking through the Relations Between Data and Community
Taylor, A. S., Lindley, S., Regan, T., Sweeney, D., Vlachokyriakos, V., Grainger, L., & Lingel, J. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702558 · Cited by 152 · cited 3×
“in-place [118]—the situatedness of data within social and physical geographies—i.”
BackgroundArtificial Intelligence for Social Good: A SurveyPreprint
Shi, Z. R., Wang, C., & Fang, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.01818 · Cited by 150 · cited 1×
“For a characterisation of the type of AI [113], refer to table 1.”
BackgroundWagstaff, K. (2012) · arXiv · doi:10.48550/arxiv.1206.4656 · Cited by 148 · cited 3×
“3445518 lionized work of building novel models and algorithms [46, 125].”
BackgroundIncentivizing High Quality Crowdwork
Ho, C.-J., Slivkins, A., Suri, S., & Vaughan, J. W. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2736277.2741102 · Cited by 146 · cited 1×
“in making use of crowdsourcing to generate ML data [25, 128], to support creation of better task designs for raters [59], compute interrater reliability, design incentives [50], and improve the quality of crowdsourced data [30], though these areas are less well known in the ML community [122].”
BackgroundVariolite: Supporting Exploratory Programming by Data Scientists
Kery, M. B., Horvath, A., & Myers, B. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025626 · Cited by 144 · cited 1×
“Prior research in HCI has drawn particular attention to work practices and challenges faced by practitioners in working with data [48, 65, 86, 93, 96].”
BackgroundDataHub: Collaborative Data Science & Dataset Version Management at ScalePreprint
Bhardwaj, A., Bhattacherjee, S., Chavan, A., Deshpande, A., Elmore, A. J., Madden, S. R., & Parameswaran, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.0798 · Cited by 138 · cited 1×
“Data documentation, which is a crucial aspect of facilitating collaboration, is well studied in the database and data management community [19, 23].”
BackgroundInvestigating statistical machine learning as a tool for software development
Patel, K., Fogarty, J. A., Landay, J. A., & Harrison, B. L. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357160 · Cited by 136 · cited 4×
“Prior research in HCI has drawn particular attention to work practices and challenges faced by practitioners in working with data [48, 65, 86, 93, 96].”
BackgroundInfluentialPADS: a domain-specific language for processing ad hoc data
Fisher, K. S., & Gruber, R. E. (2005) · ACM SIGPLAN Notices · doi:10.1145/1064978.1065046 · Cited by 132
Mao, Y., Wang, D., Muller, M., Varshney, K. R., Baldini, I., Dugan, C., & Mojsilović, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3361118 · Cited by 124 · cited 1×
“work and collaboration practices of data scientists [66, 77, 94, 127].”
BackgroundData Vision: Learning to See Through Algorithmic Abstraction
Passi, S., & Jackson, S. (2017) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2998181.2998331 · Cited by 123 · cited 5×
“[94] Samir Passi and Steven J Jackson.”
BackgroundInfluentialReappraising the utility of Google Flu Trends
Kandula, S., & Shaman, J. L. (2019) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007258 · Cited by 123 · cited 1×
“As an example, poor data practices reduced accuracy in IBM’s cancer treatment AI [115] and led to Google Flu Trends missing the flu peak by 140% [63, 73]).”
MethodWinner's Curse? On Pace, Progress, and Empirical Rigor.
Sculley, D., Snoek, J., Wiltschko, A. B., & Rahimi, A. R. O. (2018) · International Conference on Learning Representations · Cited by 117
Feinberg, M. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025837 · Cited by 115 · cited 5×
“Feinberg describes data as a design material re, and 45.3% reported two or more cascades in a given and our role as designers of data, not its appropriators [35].”
BackgroundResultInfluentialHeteromation and its (dis)contents: The invisible division of labor between humans and machines
Ekbia, H., & Nardi, B. (2014) · First Monday · doi:10.5210/fm.v19i6.5331 · Cited by 113 · cited 2×
“, heteromation [34], fauxtomation [117], and “menial” vs.”
BackgroundUnderstanding and Visualizing Data Iteration in Machine Learning
Hohman, F., Wongsuphasawat, K., Kery, M. B., & Patel, K. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376177 · Cited by 110 · cited 3×
“identified common types of data iterations and created a tool to visualise them [52].”
BackgroundMethodThe value of data: considering the context of production in data economies
Vertesi, J., & Dourish, P. (2011) · Proceedings of the ACM 2011 conference on Computer supported cooperative work · doi:10.1145/1958824.1958906 · Cited by 108 · cited 4×
“[123] Janet Vertesi and Paul Dourish.”
BackgroundInfluentialTrials and tribulations of developers of intelligent systems: A field study
Hill, C., Bellamy, R., Erickson, T., & Burnett, M. (2016) · 2016 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vlhcc.2016.7739680 · Cited by 108 · cited 1×
“Prior research in HCI has drawn particular attention to work practices and challenges faced by practitioners in working with data [48, 65, 86, 93, 96].”
BackgroundMaking data science systems work
Passi, S., & Sengers, P. (2020) · Big Data & Society · doi:10.1177/2053951720939605 · Cited by 100 · cited 6×
“Passi and Sengers [95] turn their attention to the”
BackgroundInfluentialSusha, I., Grönlund, Å., & van Tulder, R. J. M. (2018) · Government Information Quarterly · doi:10.1016/j.giq.2018.11.002 · Cited by 92
Machine Learning for the Developing World
De-Arteaga, M., Herlands, W., Neill, D. B., & Dubrawski, A. (2018) · ACM Transactions on Management Information Systems · doi:10.1145/3210548 · Cited by 83 · cited 1×
“, [31, 98, 107], understanding socio-cultural factors e.”
Designing AI for Social Good: Seven Essential Factors
Cowls, J., King, T., Taddeo, M., & Floridi, L. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3388669 · Cited by 83 · cited 1×
“periments, and testing should also be followed with data [28, 55].”
BackgroundEngaging Solidarity in Data Collection Practices for Community Health
Ismail, A., & Kumar, N. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274345 · Cited by 78 · cited 10×
“Some AI practitioners were aware of, and explicitly discussed problematic incentives for their data collectors or domain experts, and shared how they were resource-constrained (echoing Ismail and Kumar [58]).”
BackgroundInfluentialLittmann, M., Selig, K., Cohen-Lavi, L., Frank, Y., Hönigschmid, P., Kataka, E., Mösch, A., Qian, K., Ron, A., Schmid, S., et al. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0139-8 · Cited by 70 · cited 1×
“Finally, high-stakes AI is more often created at the combination of two or more disciplines; for example, AI and diabetic retinopathy, leading to greater collaboration challenges among stakeholders across organizations and domains [75, 121].”
BackgroundThe ambiguity of data science team roles and the need for a data science workforce framework
Saltz, J., & Grady, N. W. (2017) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2017.8258190 · Cited by 62 · cited 1×
“ways in which data science teams are constituted [106].”
BackgroundCrowdsourcing Subjective Tasks: The Case Study of Understanding Toxicity in Online Discussions
Aroyo, L., Dixon, L., Thain, N., Redfield, O., & Rosen, R. (2019) · Companion Proceedings of The 2019 World Wide Web Conference · doi:10.1145/3308560.3317083 · Cited by 62
Self-citationJain, A., Sarma, A. D. N., Parameswaran, A., & Widom, J. (2017) · Proceedings of the VLDB Endowment · doi:10.14778/3067421.3067431 · Cited by 60 · cited 1×
“in making use of crowdsourcing to generate ML data [25, 128], to support creation of better task designs for raters [59], compute interrater reliability, design incentives [50], and improve the quality of crowdsourced data [30], though these areas are less well known in the ML community [122].”
BackgroundDon't forget people in the use of big data for development
Blumenstock, J. (2018) · Nature · doi:10.1038/d41586-018-06215-5 · Cited by 60 · cited 1×
“, [20, 109], and complex partner and government relations e.”
BackgroundAlgorithms of Oppression: How Search Engines Reinforce Racism
Hankwitz, M. (2023) · Leonardo · doi:10.1162/leon_r_02331 · Cited by 59
Towards reliable interactive data cleaning
Krishnan, S., Haas, D., Franklin, M. J. M., & Wu, E. (2016) · Workshop on Human-In-the-Loop Data Analytics (HILDA) · doi:10.1145/2939502.2939511 · Cited by 57 · cited 1×
“Machine Learning workflows are fundamentally iterative and exploratory in nature [7, 52, 71, 96].”
BackgroundBoostClean: Automated Error Detection and Repair for Machine Learning
Krishnan, S., Franklin, M. J. M., Goldberg, K. Y., & Wu, E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.01299 · Cited by 54 · cited 3×
“and clean data include ActiveClean and BoostClean [70, 72].”
Collaborative Practices with Structured Data: Do Tools Support What Users Need?
Koesten, L. M., Kacprzak, E., Tennison, J., & Simperl, E. P. B. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300330 · Cited by 52 · cited 2×
“and sharing inside and outside the organisation or reusing another person’s data with limited interaction with the creator [69].”
BackgroundThe Problem with Metrics is a Fundamental Problem for AIPreprint
Thomas, R. L., & Uminsky, D. (2020) · arXiv.org · doi:10.48550/arxiv.2002.08512 · Cited by 52 · cited 2×
“interventions highlight the importance of catching data errors using mechanisms specific to data validation, instead of using model performance as a proxy for data quality [120].”
MethodToward responsible AI for the next billion users
Sambasivan, N., & Holbrook, J. (2018) · Interactions · doi:10.1145/3298735 · Cited by 51 · cited 1×
“, [20, 109], and complex partner and government relations e.”
BackgroundSelf-citationDudhwala, F., & Larsen, L. B. (2019) · Big Data & Society · doi:10.1177/2053951719858751 · Cited by 43 · cited 1×
“Critical data studies researchers have demonstrated longstanding interest in the ‘discretionary’ [95] practices shaping data-driven systems and how they are designed and used [6, 16, 33], and the”
BackgroundMenzies, T. (2020) · IEEE Software · doi:10.1109/ms.2019.2954841 · Cited by 43 · cited 1×
“Although the behaviour of AI systems is critically determined by data, even more so than code [111]; many of our practitioner strategies mirrored best practices in software engineering [38, 83].”
ResultTalking datasets – Understanding data sensemaking behaviours
Koesten, L. M., Gregory, K., Groth, P., & Simperl, E. P. B. (2020) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2020.102562 · Cited by 36 · cited 1×
“[68], often without adequate application-domain expertise, working with domain experts,”
BackgroundDebugging Machine Learning Pipelines
Lourenço, R., Freire, J., & Shasha, D. (2019) · doi:10.1145/3329486.3329489 · Cited by 27 · cited 1×
“Researchers have created several tools to support the creation of ML ‘pipelines’ and make these workfows manageable [21, 54, 70, 72, 76].”
Examining the challenges in development data pipeline
Pervaiz, F., Vashistha, A., & Anderson, R. (2019) · ACM SIGCAS Conference on Computing and Sustainable Societies (COMPASS) · doi:10.1145/3314344.3332496 · Cited by 25 · cited 1×
“, [31, 98, 107], understanding socio-cultural factors e.”
Making better use of the crowd
Wortman, V. (2017) · Journal of Machine Learning Research · doi:10.5555/3122009.3242050 · Cited by 25
Open Scholarship and Peer Review: a Time for Experimentation
Soergel, D. A. W., Saunders, A., & McCallum, A. (2013) · Cited by 24
Jain, A., Sarma, A. D. N., Parameswaran, A., & Widom, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.06207 · Cited by 9 · cited 1×
“in making use of crowdsourcing to generate ML data [25, 128], to support creation of better task designs for raters [59], compute interrater reliability, design incentives [50], and improve the quality of crowdsourced data [30], though these areas are less well known in the ML community [122].”
BackgroundEvaluation of manually created ground truth for multi-view people localization
Kiss, Á., & Szirányi, T. (2013) · doi:10.1145/2501105.2501106 · Cited by 8 · cited 1×
“searchers have also studied the ways in which data is rarely used as given, and often needs to be created or handcrafted using intricate transformation practices [67, 96].”
BackgroundGlobal Goods Software for the Immunization Cold Chain
Brunette, W., Larson, C., Jain, S., Langford, A., Low, Y. Y., Siew, A., & Anderson, R. (2020) · ACM SIGCAS Conference on Computing and Sustainable Societies (COMPASS) · doi:10.1145/3378393.3402278 · Cited by 5 · cited 1×
“, [22] in AI projects in the Global South.”
BackgroundThe Missing Science of Knowledge Curation
Paritosh, P. K. (2018) · doi:10.1145/3184558.3191551 · Cited by 4 · cited 1×
“appropriately—opening up a whole new space for HCI to act as the compass for AI by answering questions about goodness, fidelity, and validity of data by itself, as HCOMP researchers have pointed out [12, 90].”
BackgroundSelf-citation(2013)
Research works printed but not linked: 16
- 10.15346/hc.v1i1.34 · doi:10.15346/hc.v1i1.34
- Timnit Gebru Jamie Morgenstern Briana Vecchione Jennifer Wortman Vaughan Hanna Wallach Hal Daumé III and Kate Crawford. 2018. Datasheets for datasets. (2018). arXiv:1803.09010 · arXiv
- 10.7551/mitpress/9302.001.0001 · DOI · Paper page
- J Hirschberg. 1998. Every time I fire a linguist, my performance goes up, and other myths of the statistical natural language processing revolution. Invited talk. In Fifteenth National Conference on Artificial Intelligence (AAAI-98).
- Ben Hutchinson, Andrew Smart, Alex Hanna, Emily Denton, Christina Greer, Oddur Kjartansson, Parker Barnes, and Margaret Mitchell. 2020. Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure. (2020). arXiv:2010.13561 · arXiv
- Nick Hynes, D Sculley, and Michael Terry. 2017. The data linter: Lightweight, automated sanity checking for ml data sets. In NIPS MLSys Workshop.
- Zachary C Lipton and Jacob Steinhardt. 2018. Troubling trends in machine learning scholarship. (2018). arXiv:1807.03341 · arXiv
- Hannah Miller and Richard Stirling. 2019. Government AI Readiness Index 2019 — Oxford Insights — Oxford Insights. https://www.oxfordinsights.com/ai-readiness2019. (Accessed on 09/14/2020).
- Naja Holten Møller, Claus Bossen, Kathleen H Pine, Trine Rask Nielsen, and Gina Neff. 2020. Who does the work of data?Interactions 27, 3 (2020), 52–55.
- 10.18574/nyu/9781479833641.001.0001 · DOI · Paper page
- Lawrence A Palinkas, Sarah M Horwitz, Carla A Green, Jennifer P Wisdom, Naihua Duan, and Kimberly Hoagwood. 2015. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administration and policy in mental health and mental health services research 42, 5 (2015), 533–544.
- Praveen Paritosh, Kurt Bollacker, Maria Stone, Lora Aroyo, and Sarah Luger. 2020. Evaluating Evaluation of AI Systems (Meta-Eval 2020). http://eval.how/aaai-2020/. (Accessed on 09/16/2020).
- Praveen Paritosh, Matt Lease, Mike Schaekermann, and Lora Aroyo. 2020. First workshop on Data Excellence (DEW 2020). http://eval.how/dew2020/. (Accessed on 09/16/2020).
- Rashida Richardson, Jason M Schultz, and Kate Crawford. 2019. Dirty data, bad predictions: How civil rights violations impact police data, predictive policing systems, and justice. NYUL Rev. Online 94(2019), 15.
- Nithya Sambasivan, Garen Checkley, Amna Batool, Nova Ahmed, David Nemer, Laura Sanely Gaytán-Lugo, Tara Matthews, Sunny Consolvo, and Elizabeth Churchill. 2018. ” Privacy is not for me, it’s for those rich women”: Performative Privacy Practices on Mobile Phones by Women in South Asia. In Fourteenth Symposium on Usable Privacy and Security ({SOUPS} 2018). 127–142.
- Sarah Myers West, Meredith Whittaker, and Kate Crawford. 2019. Discriminating systems: Gender, race and power in AI. AI Now Institute (2019), 1–33.
Cited by
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Self-supervised learning in medicine and healthcare
Krishnan, R., Rajpurkar, P., & Topol, E. J. (2022) · Nature Biomedical Engineering · doi:10.1038/s41551-022-00914-1 · Cited by 605
Advances, challenges and opportunities in creating data for trustworthy AI
Liang, W., Tadesse, G. A., Ho, D., Fei-Fei, L., Zaharia, M., Zhang, C., & Zou, J. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00516-1 · Cited by 559
“A data-centric focus is often lacking in current AI research 19 .”
BackgroundData-centric Artificial Intelligence: A Survey
Zha, D., Bhat, Z. P., Lai, K.-H., Yang, F., Jiang, Z., Zhong, S., & Hu, X. (2025) · ACM Computing Surveys · doi:10.1145/3711118 · Cited by 334
“Furthermore, undervaluing data quality could trigger data cascades [200], causing negative effects such as decreased accuracy and persistent biases [36].”
Power to the People? Opportunities and Challenges for Participatory AI
Birhane, A., Isaac, W., Prabhakaran, V., Diaz, M., Elish, M. C., Gabriel, I., & Mohamed, S. (2022) · Equity and Access in Algorithms Mechanisms and Optimization · doi:10.1145/3551624.3555290 · Cited by 330
“The recent upswing in participatory tools have varied in approach and point in the machine learning product lifecycle, including: auditing methods for machine learning based interventions [48, 75], public consultation methods such as citizen juries [6, 83] or joint problem formulation [61], information disclosures suc…”
Background
Metadata
- Authors
- Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M.
- Affiliations
- Google (United States) (US)
- Venue
- Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2021-05-06
- Pages
- 1-15
- OpenAlex
- W3118813946
- Semantic Scholar
- 63d7e40da7f0d37308b8e97fca4a14a26a6b52ea
- Cited by
- 781
- References
- 104
- Citations by year
- 2026: 166 · 2025: 151 · 2024: 150 · 2023: 146 · 2022: 122 · 2021: 46
- FWCI
- 104.68
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by-nc-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Information Systems › Data Quality and Management
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- data quality, AI practitioners, data practices, AI safety, health AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.