CoChat
Sign Up
Cited by 23Information Systems Theories and ImplementationEthics and Social Impacts of AISoftware Engineering Techniques and Practices

From Bias to Repair: Error as a Site of Collaboration and Negotiation in Applied Data Science Work

Open access
Lin, Cindy Kaiying, Jackson, Steven J.·2023·Proceedings of the ACM on Human-Computer Interaction
DOI 10.1145/3579607
Open this paper in CoChat — free accountOpen PDF

Abstract

Managing error has become an increasingly central and contested arena within data science work. While recent scholarship in artificial intelligence and machine learning has focused on limiting and eliminating error, practitioners have long used error as a site of collaboration and learning vis-à-vis labelers, domain experts, and the worlds data scientists seek to model and understand. Drawing from work in CSCW, STS, HCML, and repair studies, as well as from multi-sited ethnographic fieldwork within a government institution and a non-profit organization, we move beyond the notion of error as an edge case or anomaly to make three basic arguments. First, error discloses or calls to attention existing structures of collaboration unseen or underappreciated under 'working' systems. Second, error calls into being new forms and sites of collaboration (including, sometimes, new actors). Third, error redeploys old sites and actors in new ways, often through restructuring relations of hierarchy and expertise which recenter or devalue the position of different actors. We conclude by discussing how an artful living with error can better support the creative strategies of negotiation and adjustment which data scientists and their collaborators engage in when faced with disruption, breakdown, and friction in their work.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3579607

Cites

55 of 55 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 56 references couldn’t be matched to a known paper.

Research works printed but not linked: 38

  1. Mike Ananny. 2022. Seeing Like an Algorithmic Error: What are Algorithmic Mistakes, Why Do They Matter, How Might They Be Public Problems? In The Yale Information Society Project & Yale Journal Of Law And Technology White Paper Series. https://yjolt.org/sites/default/files/0_-_ananny_- _seeing_like_an_algorithmic_error.pdf
  2. Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? . In Proceedings of the 2021 ACM Conference on From Bias to Repair: Error as a Site of Collaboration and Negotiation in Applied Data Science Work 131:27 PACM on Human-Computer Interaction, Vol. 7, No. CSCW1, Article 131, Publication date: April 2023.
  3. Mélanie Bernhardt, Daniel C. Castro, Ryutaro Tanno, Anton Schwaighofer, Kerem C. Tezcan, Miguel Monteiro, Shruthi Bannur et al. 2022. Active label cleaning for improved dataset quality under resource constraints. Nature communications 13, no. 1 (2022), 1-11.
  4. Dan Bouk. 2020. Error, Uncertainty, and the Shifting Ground of Census Data. Harvard Data Science Review, 2(2). https://doi-org.proxy.library.cornell.edu/10.1162/99608f92.962cb309 · DOI
  5. Joy Buolamwini, Sorelle A Friedler, and Christo Wilson. [n.d.]. Gender shades: Intersectional accuracy disparities in commercial gender classification. http: //proceedings.mlr.press/v81/buolamwini18a/buolamwini18a.pdf. Accessed: 2022-1-12.
  6. Meredith Broussard. Forthcoming. More Than a Glitch: Confronting Race, Gender, and Ability Bias in Tech. Cambridge, MA: MIT Press.
  7. Stevie Chancellor. 2022. Towards Practices for Human-Centered Machine Learning. arXiv preprint arXiv:2203.00432 (2022). · arXiv
  8. Aida Mostafazadeh Davani, Mark Díaz, and Vinodkumar Prabhakaran. 2022. Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations. Transactions of the Association for Computational Linguistics, 10:92-110.
  9. John Dewey. 1986. Experience and education. In The educational forum (Vol. 50, No. 3, pp. 241-252).
  10. Catherine D'Ignazio and Lauren F Klein. 2020. Data Feminism. MIT Press, Cambridge, MA.
  11. Anca Dumitrache, Lora Aroyo, and Chris Welty. 2015. CrowdTruth Measures for Language Ambiguity: The Case of Medical Relation Extraction. In In Proc. of LD4IE Workshop, ISWC. http://ceur-ws.org/Vol- 1467/LD4IE2015_Dumitrache.pdf 131:28
  12. Cindy Kaiying Lin & Steven J. Jackson PACM on Human-Computer Interaction, Vol. 7, No. CSCW1, Article 131, Publication date: April 2023.
  13. Matthew Van Gundy, Davide Balzarotti, and Giovanni Vigna. 2007. Catch me, if you can: Evading network signatures with web-based polymorphic worms. In Proceedings of the first USENIX workshop on Offensive Technologies (WOOT '07) . USENIX Association, Berkley, CA, Article 7, 9 pages.
  14. Ian Hacking. 1990. The Taming of Chance. Cambridge University Press.
  15. Jessica Hullman, Sayash Kapoor, Priyanka Nanayakkara, Andrew Gelman, and Arvind Narayanan. 2022. The worst of both worlds: A comparative analysis of errors in learning from data in psychology and machine learning. arXiv preprint arXiv:2203.06498 (2022). · arXiv
  16. Steven J. Jackson and Lara Houston. 2020. The Poetics and Political Economy of Repair. in Janet Wasko and Jeremy Schwartz, eds. Media: A Transdisciplinary Inquiry. Intellect Books, University of Chicago Press: Chicago.
  17. Steven Jackson. 2014. Rethinking Repair, in T. Gillespie, P. Boczkowski, and K. Foot, eds. Media Technologies: Essays on Communication, Materiality and Society. Cambridge, MA: MIT Press.
  18. Ju Yeon Jung, Tom Steinberger, John L. King, and Mark S. Ackerman. 2022. How Domain Experts Work with Data: Situating Data Science in the Practices and Settings of Craftwork. Proc. ACM Hum.-Comput. Interact. 6, From Bias to Repair: Error as a Site of Collaboration and Negotiation in Applied Data Science Work 131:29 PACM on Human-Computer Interaction, Vol. 7, No. CSCW1, Article 131, Publication date: April 2023.
  19. Frederike Kaltheuner, Abeba Birhane, Inioluwa Deborah Raji, Razvan Amironesei, Emily Denton, Alex Hanna, Hilary Nicole, Andrew Smart, Serena Dokuaa Oduro, James Vincent, Alexander Reben, Gemma Milne, Crofton Black, Adam Harvey, Andrew Strait, Tulsi Parida, Aparna Ashok, Fieke Jansen, Corinne Cath, and Aidan Peppin. 2021. Fake AI. Meatspace Press.
  20. Dongyue Li and Hongyang Zhang. 2021. Improved regularization and robustness for fine-tuning in neural networks." In 35th Conference on Neural Information Processing Systems (NeurIPS 2021): 27249-27262.
  21. Zhiyi Ma, Kawin Ethayarajh, Tristan Thrush, Somya Jain, Ledell Wu, Robin Jia, Christopher Potts, Adina Williams, Douwe Kiela. 2021. Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarking. In Advances in Neural Information Processing Systems 34 (NeurIPS 2021). https://proceedings.neurips.cc/paper/2021/hash/55b1927fdafef39c48e5b73b5d61ea60-Abstract.html
  22. McWilliam, N., R. Teeuw, M. Whiteside, and P. Zukowskyj. 2005. Chapter 8: Image Interpretation and Processing GIS, GPS, and remote sensing. In The Expedition Advisory Centre Royal Geographical Society 1 Kensington Gore. https://www.rgs.org/CMSPages/GetFile.aspx?nodeguid=09c5b6e1-87f5-4ba9-9976-e03c383506ff&lang=en-GB
  23. Michael Muller, Melanie Feinberg, Timothy George, Steven J. Jackson, Bonnie E. John, Mary Beth Kery, and Samir Passi. 2019. Human-centered study of data science work practices. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, Glasgow Scotland Uk, 1-8. https: 131:30
  24. Emanuel Moss, Elizabeth Anne Watkins, Ranjit Singh, Madeleine Clare Elish, and Jacob Metcalf. 2021. Assembling accountability: algorithmic impact assessment for the public interest. Available at SSRN 3877437.
  25. Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, Etienne David, Ian Stavness, Wei Guo, Berton Earnshaw, Imran Haque, Sara M Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, Percy Liang. 2021. WILDS: A Benchmark of in-the-Wild Distribution Shifts. Proceedings of the 38th International Conference on Machine Learning, PMLR 139:5637-5664.
  26. Roberta Raileanu, Maxwell Goldstein, Denis Yarats, Ilya Kostrikov, and Rob Fergus. 2021. Automatic data augmentation for generalization in reinforcement learning. In Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS 2021): 5402-5415.
  27. Inioluwa Deborah Raji, I. Elizabeth Kumar, Aaron Horowitz, Andrew D. Selbst. 2022. The Fallacy of AI Functionality. In 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT '22). Association for Computing Machinery, New York, NY, USA, 959-972. https://doi- org.proxy.library.cornell.edu/10.1145/3531146.3533158 · DOI · Paper page
  28. Inioluwa Deborah Raji and Jingying Yang. 2019. About ml: Annotation and benchmarking on understanding and transparency of machine learning lifecycles." arXiv preprint arXiv:1912.06166. · arXiv
  29. Daniela K. Rosner and Morgan G. Ames. 2014. "Designing for Repair? Infrastructures and Materialities of Breakdown." Proceedings of CSCW 2014, ACM Conference on Computer-Supported Cooperative Work and Social Computing. ACM Press, February 2014, 319-331.
  30. Hadi Salman, Greg Yang, Jerry Li, Pengchuan Zhang, Huan Zhang, Ilya Razenshteyn, Sébastien Bubeck. 2019. Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers. 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada: 1-12.
  31. Shreya Shankar, Yoni Halpern, Eric Breck, James Atwood, Jimbo Wilson, D. Sculley. 2017. No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World. In Proceedings of 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA. arXiv:1711.08536v1 · arXiv
  32. Luke Stark and Jevan Hutson. 2022. Physiognomic Artificial Intelligence. forthcoming in Fordham Intellectual Property, Media & Entertainment Law Journal XXXII (2022). https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3927300
  33. Anissa Tanweer, Cecilia R Aragon, Michael Muller, Shion Guha, Samir Passi, Gina Neff, and Marina Kogan. 2022. Interrogating Human-centered Data Science: Taking Stock of Opportunities and Limitations. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (CHI EA '22). Association for Computing Machinery, New York, NY, USA, Article 99, 1-6. https://doi- org.proxy.library.cornell.edu/10.1145/3491101.3503740 · DOI
  34. Anna L. Tsing. (2012). On NonscalabilityThe Living World Is Not Amenable to Precision-Nested Scales. Common knowledge, 18(3), 505-524.
  35. Pablo R. Velasco. 2019. Artificial Intelligibility and Proxy Error. spheres: Journal for Digital Cultures 5: 1-6.
  36. Richmond Y. Wong, Michael A. Madaio, and Nick Merrill. Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics. arXiv preprint arXiv:2202.08792 (2022). · arXiv
  37. Meg Young, Michael Katell, and P.M. Krafft. 2022. Confronting Power and Corporate Capture at the FAccT Conference. In 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT '22). Association for Computing Machinery, New York, NY, USA, 1375-1386. https://doi- org.proxy.library.cornell.edu/10.1145/3531146.3533194 · DOI
  38. Songzhu Zheng, Pengxiang Wu, Aman Goswami, Mayank Goswami, Dimitris Metaxas, Chao Chen. 2020. Error- Bounded Correction of Noisy Labels. In International Conference on Machine Learning, pp. 11447-11457. PMLR, 2020.

Cited by

Sorted by citations

Related

Authors
Lin, C. K., & Jackson, S. J.
Affiliations
  • Pennsylvania State University (US)
  • Cornell University (US)
ORCID
Venue
Proceedings of the ACM on Human-Computer Interaction
Publisher
Association for Computing Machinery (ACM)
Published
2023-04-14
Volume
7
Issue
CSCW1
Pages
1-32
ISSN
2573-0142 (ISSN-L)
DOI
10.1145/3579607
OpenAlex
W4366004045
Cited by
22
References
56
Citations by year
2026: 2 · 2025: 6 · 2024: 14
FWCI
9.49
Type
journal article
Language
en
Open access
Licence
Locations
Proceedings of the ACM on Human-Computer Interaction (publishedVersion, other-oa)
Funders
Cornell Atkinson Center for Sustainability, Cornell University
Topics
  • Social Sciences › Sociology and Political Science › Information Systems Theories and Implementation
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
Keywords
error management, collaboration, negotiation, domain experts, bias, work systems

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review