Making data science systems work
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
How are data science systems made to work? It may seem that whether a system works is a function of its technical design, but it is also accomplished through ongoing forms of discretionary work by many actors. Based on six months of ethnographic fieldwork with a corporate data science team, we describe how actors involved in a corporate project negotiated what work the system should do, how it should work, and how to assess whether it works. These negotiations laid the foundation for how, why, and to what extent the system ultimately worked. We describe three main findings. First, how already-existing technologies are essential reference points to determine how and whether systems work. Second, how the situated resolution of development challenges continually reshapes the understanding of how and whether systems work. Third, how business goals, and especially their negotiated balance with data science imperatives, affect a system’s working. We conclude with takeaways for critical data studies, orienting researchers to focus on the organizational and cultural aspects of data science, the third-party platforms underlying data science systems, and ways to engage with practitioners’ imagination of how systems can and should work.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective
Haraway, D. (1988) · Feminist Studies · doi:10.2307/3178066 · Cited by 18,364
On Justification: Economies of Worth
Boltanski, L., & Thévenot, L. (2006) · doi:10.1515/9781400827145 · Cited by 2,709
'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions
Binns, R., Van Kleek, M., Veale, M., Lyngs, U., Zhao, J., & Shadbolt, N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173951 · Cited by 691 · cited 2×
“One reason for this is that the perspective of people facing the most significant effects from a system’s output is often not included in the system’s design and development process (Binns et al., 2018).”
BackgroundProblem Formulation and Fairness
Passi, S., & Barocas, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287567 · Cited by 283 · cited 3×
“What is normatively better depends as much on people’s normative stance as on the practical judgments that drive their everyday work (Passi and Barocas, 2019).”
BackgroundMethodSelf-citationWorking artefacts: ethnomethods of the prototype
Suchman, L., Trigg, R., & Blomberg, J. (2002) · The British Journal of Sociology · doi:10.1080/00071310220133287 · Cited by 250 · cited 2×
“Whether a data science system works is neither obvious nor given (for a more general account, see Collins, 1985; Rooksby et al., 2009; Suchman et al., 2002); the perceived success and failure of its working depend as much on business goals as on data science imperatives.”
BackgroundPassi, S., & Jackson, S. J. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274405 · Cited by 225
Self-citationDatafication and data fiction: Narrating data and narrating with data
Dourish, P., & Gómez Cruz, E. (2018) · Big Data & Society · doi:10.1177/2053951718784083 · Cited by 220 · cited 1×
“Still, although models and numbers are vital to the process, building data science systems is a sociotechnical endeavor that requires not only technical but also human work (Baumer, 2017; Dourish and Cruz, 2018; Passi and Jackson, 2017, 2018).”
BackgroundToward human-centered algorithm design
Baumer, E. P. (2017) · Big Data & Society · doi:10.1177/2053951717718854 · Cited by 125 · cited 3×
“Still, although models and numbers are vital to the process, building data science systems is a sociotechnical endeavor that requires not only technical but also human work (Baumer, 2017; Dourish and Cruz, 2018; Passi and Jackson, 2017, 2018).”
BackgroundDatatrust: Or, the political quest for numerical evidence and the epistemologies of Big Data
Rieder, G., & Simon, J. (2016) · Big Data & Society · doi:10.1177/2053951716649398 · Cited by 104 · cited 1×
“Data scientists, for instance, often describe a system’s working via performance metrics (Rieder and Simon, 2016).”
Background
Cited by
“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI
Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445518 · Cited by 781
“Passi and Sengers [95] turn their attention to the”
BackgroundInfluentialThe Fallacy of AI Functionality
Raji, I. D., Kumar, I. E., Horowitz, A., & Selbst, A. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533158 · Cited by 251
Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support
Madaio, M., Egede, L., Subramonyam, H., Wortman Vaughan, J., & Wallach, H. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512899 · Cited by 156
Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics
Wong, R. Y., Madaio, M. A., & Merrill, N. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579621 · Cited by 135
Saxena, D., Badillo-Urquiola, K., Wisniewski, P. J., & Guha, S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476089 · Cited by 134
Metadata
- Authors
- Passi, S., & Sengers, P.
- Venue
- Big Data & Society
- Publisher
- SAGE Publications
- Published
- 2020-07
- Volume
- 7
- Issue
- 2
- ISSN
- 2053-9517 (ISSN-L)
- OpenAlex
- W3045417466
- Cited by
- 79
- References
- 9
- FWCI
- 14.21
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Making data science systems work. https://app.cochat.ai/papers/10.1177/2053951720939605
- The paper
- Passi, S., & Sengers, P. (2020). Making data science systems work. Big Data & Society. https://doi.org/10.1177/2053951720939605
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.