Cited by 225Ethics and Social Impacts of AIInformation Systems Theories and ImplementationExplainable Artificial Intelligence (XAI)

The DOI matches its Crossref record; 101 of the 101 references it cites were found in a registry or an index.

Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects

Open access

Passi, Samir, Jackson, Steven J.·2018·Proceedings of the ACM on Human-Computer Interaction

DOI 10.1145/3274405

Open this paper in CoChat — free account Open PDF

Abstract

The trustworthiness of data science systems in applied and real-world settings emerges from the resolution of specific tensions through situated, pragmatic, and ongoing forms of work. Drawing on research in CSCW, critical data studies, and history and sociology of science, and six months of immersive ethnographic fieldwork with a corporate data science team, we describe four common tensions in applied data science work: (un)equivocal numbers, (counter)intuitive knowledge, (in)credible data, and (in)scrutable models. We show how organizational actors establish and re-negotiate trust under messy and uncertain analytic conditions through practices of skepticism, assessment, and credibility. Highlighting the collaborative and heterogeneous nature of real-world data science, we show how the management of trust in applied corporate data science settings depends not only on pre-processing and quantification, but also on negotiation and translation. We conclude by discussing the implications of our findings for data science research and practice, both within and beyond CSCW.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1145/3274405

Cites

101 of 101 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The Discovery of Grounded Theory; Strategies for Qualitative Research
Glaser, B. G., Strauss, A. L., & Strutzel, E. (1968) · Nursing Research · doi:10.1097/00006199-196807000-00014 · Cited by 39,449
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Studies in Ethnomethodology.
Wallace, A. F. C., & Garfinkel, H. (1968) · American Sociological Review · doi:10.2307/2092245 · Cited by 12,413
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682
The Mythos of Model Interpretability: In machine learning, the concept of interpretability is both important and slippery.
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“Several data correlations established in large datasets are neither reproducible nor falsifiable [45,61].”
Background
The ethics of algorithms: Mapping the debate
Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016) · Big Data & Society · doi:10.1177/2053951716679679 · Cited by 4,567 · cited 1×
“The increased management of data science systems is a priority [65,96,105], but “the complex decision-making structure” needed to manage them often exceeds “the human and organizational resources available for oversight” [68].”
Background
Studies in Ethnomethodology
Busfield, J., & Garfinkel, H. (1968) · British Journal of Sociology · doi:10.2307/588840 · Cited by 3,828
The Audit Society: Rituals of Verification
Quam, L. (1998) · BMJ · doi:10.1136/bmj.316.7133.787a · Cited by 3,611

Flagged first, then by citations

Show all 100 references

Research works printed but not linked: 19

Geoffrey C. Bowker. 2014. The Theory/Data Thing. Int. J. Commun.8, 1795--1799.
Kathy Charmaz. 2014. Constructing Grounded Theory (Introducing Qualitative Methods series) 2nd Edition. Sage, London.
Harry M. Collins and Robert Evans. 2007. Rethinking Expertise. University of Chicago Press, Chicago.
10.1525/rep.1992.40.1.99p0137h · DOI · Paper page
Lorraine Daston and Peter Galison. 2007. Objectivity. MIT Press Cambridge Massachusetts.
John Dewey. 1939. Theory of Valuation. University of Chicago Press, Chicago.
Tarleton Gillespie. 2014. The Relevance of Algorithms. In Media Technologies: Essays on Communication, Materiality, and Society, Tarleton Gillespie, Pablo J. Boczkowski and Kirsten A. Foot (eds.). MIT Press, Cambridge, 167--194.
Lisa Gitelman. 2006. Raw Data is an Oxymoron. MIT Press, MA.
Ian Hacking. 1990. The Taming of Chance. Cambridge University Press, Cambridge.
Donna Haraway. 2007. Modest_Witness@Second_Millenium. Routledge London.
Edmund Husserl. 1970. The Crisis of European Sciences and Transcendental Philosophy. Northwestern University Press, Evanston.
Bruno Latour. 1987. Science in Action. Harvard University Press, Cambridge, MA.
Zachary Chase Lipton. 2016. The Mythos of Model Interpretability. In2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016).
10.5555/3200092 · DOI
10.5555/2717112 · DOI
Rita Raley. 2013. Dataveillance and Counterveillance. In"Raw Data" Is an Oxymoron, Lisa Gitelman (ed.). MIT Press, Cambridge, Massachusetts, 121--145.
Steven Shapin. 1989. The Invisible Technician. Am. Sci.77, 6, 554--563.
Anselm Strauss and Juliet M. Corbin. 1990. Basics of Qualitative Research: Grounded Theory Techniques and Procedures. Sage, New York.
Alex Tabarrok. 2015. The Rise of Opaque Intelligence. Marginal Revolution. Retrieved February 15, 2018 from http://marginalrevolution.com/marginalrevolution/2015/02/opaque-intelligence.html

Sorted by citations

“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI
Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445518 · Cited by 781
Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI
Wang, D., Weisz, J. D., Muller, M., Ram, P., Geyer, W., Dugan, C., Tausczik, Y., Samulowitz, H., & Gray, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359313 · Cited by 342
How do Data Science Workers Collaborate? Roles, Workflows, and Tools
Zhang, A. X., Muller, M. J., & Wang, D. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392826 · Cited by 304
The Fallacy of AI Functionality
Raji, I. D., Kumar, I. E., Horowitz, A., & Selbst, A. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533158 · Cited by 251
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction
Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., et al. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604673 · Cited by 279
Problem Formulation and Fairness
Passi, S., & Barocas, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287567 · Cited by 283
Self-citation
How to Evaluate Trust in AI-Assisted Decision Making? A Survey of Empirical Methodologies
Vereschak, O., Bailly, G., & Caramiaux, B. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476068 · Cited by 229
Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support
Madaio, M., Egede, L., Subramonyam, H., Wortman Vaughan, J., & Wallach, H. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512899 · Cited by 156
AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications
Chen, P., Wu, L., & Wang, L. (2023) · Applied Sciences · doi:10.3390/app131810258 · Cited by 155
How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study
Piorkowski, D., Park, S., Wang, A. Y., Wang, D., Muller, M., & Portnoy, F. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449205 · Cited by 151
Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision
Miceli, M., Schuessler, M., & Yang, T. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415186 · Cited by 149
A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare
Saxena, D., Badillo-Urquiola, K., Wisniewski, P. J., & Guha, S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476089 · Cited by 134
EXplainable Artificial Intelligence (XAI)—From Theory to Methods and Applications
Ortigossa, E. S., Gonçalves, T., & Nonato, L. G. (2024) · IEEE Access · doi:10.1109/access.2024.3409843 · Cited by 124
The Who in XAI: How AI Background Shapes Perceptions of AI Explanations
Ehsan, U., Passi, S., Liao, Q. V., Chan, L., Lee, I.-H., Muller, M., & Riedl, M. O. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642474 · Cited by 123
Self-citation
Trust in AutoML: exploring information needs for establishing trust in automated machine learning systems
Drozdal, J., Weisz, J., Wang, D., Dass, G., Yao, B., Zhao, C., Muller, M., Ju, L., & Su, H. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377501 · Cited by 119
This Thing Called Fairness: Disciplinary Confusion Realizing a Value in Technology
Mulligan, D. K., Kroll, J. A., Kohli, N., & Wong, R. Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359221 · Cited by 111
Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics
Wong, R. Y., Madaio, M. A., & Merrill, N. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579621 · Cited by 135
Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits
Deng, W. H., Nagireddy, M., Lee, M. S. A., Singh, J., Wu, Z. S., Holstein, K., & Zhu, H. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533113 · Cited by 99
The Making of Data Commodities: Data Analytics as an Embedded Process
Aaltonen, A., Alaimo, C., & Kallinikos, J. (2021) · Journal of Management Information Systems · doi:10.1080/07421222.2021.1912928 · Cited by 95
Documenting Computer Vision Datasets
Miceli, M., Yang, T., Naudts, L., Schuessler, M., Serbanescu, D. A., & Hanna, A. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445880 · Cited by 94
Tactics of Soft Resistance in User Experience Professionals' Values Work
Wong, R. Y. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479499 · Cited by 82
Making data science systems work
Passi, S., & Sengers, P. (2020) · Big Data & Society · doi:10.1177/2053951720939605 · Cited by 89
Self-citation
Designing Ground Truth and the Social Life of Labels
Muller, M., Wolf, C. T., Andres, J., Desmond, M., Joshi, N. N., Ashktorab, Z., Sharma, A., Brimijoin, K., Pan, Q., Duesterwald, E., et al. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445402 · Cited by 83
Governing Artificial Intelligence in an Age of Inequality
Sampath, P. G. (2021) · Global Policy · doi:10.1111/1758-5899.12940 · Cited by 71
Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory
Kaur, H., Adar, E., Gilbert, É., & Lampe, C. A. C. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533135 · Cited by 88
Talking datasets – Understanding data sensemaking behaviours
Koesten, L. M., Gregory, K., Groth, P., & Simperl, E. P. B. (2020) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2020.102562 · Cited by 8
How Experienced Designers of Enterprise Applications Engage AI as a Design Material
Yildirim, N., Kass, A., Tung, T., Upton, C., Costello, D., Giusti, R., Lacin, S., Lovic, S., O'Neill, J. M., Meehan, R. O., et al. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517491 · Cited by 71
An Empirical Survey on Explainable AI Technologies: Recent Trends, Use-Cases, and Categories from Technical and Application Perspectives
Nagahisarchoghaei, M., Nur, N., Cummins, L., Nur, N., Karimi, M. M., Nandanwar, S., Bhattacharyya, S., & Rahimi, S. (2023) · Electronics · doi:10.3390/electronics12051092 · Cited by 83
Generative AI in the Wild: Prospects, Challenges, and Strategies
Sun, Y., Jang, E., Ma, F., & Wang, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642160 · Cited by 64
Understanding Machine Learning Practitioners' Data Documentation Perceptions, Needs, Challenges, and Desiderata
Heger, A., Marquis, L. B., Vorvoreanu, M., Wallach, H., & Vaughan, J. W. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555760 · Cited by 60
AutoAIViz
Weidele, D. K. I., Weisz, J. D., Oduor, E., Muller, M. J., Andrés, J., Gray, A., & Wang, D. (2020) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3377325.3377538 · Cited by 61
Facilitating Knowledge Sharing from Domain Experts to Data Scientists for Building NLP Models
Park, S., Wang, A. Y., Kawas, B., Liao, Q. V., Piorkowski, D., & Danilevsky, M. (2021) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3397481.3450637 · Cited by 70
Investigating Practices and Opportunities for Cross-functional Collaboration around AI Fairness in Industry Practice
Deng, W. H., Yildirim, N., Chang, M., Eslami, M., Holstein, K., & Madaio, M. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594037 · Cited by 79
Making It Possible for the Auditing of AI: A Systematic Review of AI Audits and AI Auditability
Li, Y., & Goel, S. (2024) · Information Systems Frontiers · doi:10.1007/s10796-024-10508-8 · Cited by 65
Privacy Legislation as Business Risks: How GDPR and CCPA are Represented in Technology Companies' Investment Risk Disclosures
Wong, R. Y., Chong, A., & Aspegren, R. C. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579515 · Cited by 65
Forgetting Practices in the Data Sciences
Muller, M. J., & Strohmayer, A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517644 · Cited by 55
Fits and Starts: Enterprise Use of AutoML and the Role of Humans in the Loop
Crisan, A., & Fiore-Gartland, B. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445775 · Cited by 56
Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust
Banovic, N., Yang, Z., Ramesh, A., & Liu, A. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579460 · Cited by 55
A Meta-Summary of Challenges in Building Products with ML Components – Collecting Experiences from 4758+ Practitioners
Nahar, N., Zhang, H., Lewis, G. A., Zhou, S., & Kästner, C. (2023) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain58948.2023.00034 · Cited by 70
Tasks as needs: reframing the paradigm of clinical natural language processing research for real-world decision support
Lederman, A., Lederman, R., & Verspoor, K. M. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac121 · Cited by 52
A hunt for the Snark: Annotator Diversity in Data Practices
Kapania, S., Taylor, A. S., & Wang, D. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580645 · Cited by 50
Model Positionality and Computational Reflexivity: Promoting Reflexivity in Data Science
Cambo, S. A., & Gergle, D. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501998 · Cited by 49
Human-Centered Study of Data Science Work Practices
Muller, M. J., Feinberg, M., George, T., Jackson, S. J., John, B. E., Kery, M. B., & Passi, S. (2019) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3299018 · Cited by 48
Self-citation
Human-Centered AI Design in Reality: A Study of Developer Companies’ Practices
Hartikainen, M., Väänänen, K., Lehtiö, A., Ala-Luopa, S., & Olsson, T. (2022) · Nordic Human-Computer Interaction Conference · doi:10.1145/3546155.3546677 · Cited by 48
No! Re-imagining Data Practices Through the Lens of Critical Refusal
García, P. M., Sutherland, T., Salehi, N., Cifor, M., & Singh, A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3557997 · Cited by 52
Callisto: Capturing the "Why" by Connecting Conversations with Computational Narratives
Wang, A. Y., Wu, Z., Brooks, C. A., & Oney, S. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376740 · Cited by 44
Trusted Decision-Making: Data Governance for Creating Trust in Data Science Decision Outcomes
Brous, P., & Janssen, M. (2020) · Administrative Sciences · doi:10.3390/admsci10040081 · Cited by 43
“Data governance establishes data management processes which manage data quality (Passi and Jackson 2018) and compliance with relevant laws, directives, and policies (Cato et al. 2015).”
BackgroundInfluential
Data Hunches: Incorporating Personal Knowledge into Visualizations
Lin, H., Akbaba, D., Meyer, M., & Lex, A. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3209451 · Cited by 50
How Do UX Practitioners Communicate AI as a Design Material? Artifacts, Conceptions, and Propositions
Feng, K. J. K., Coppock, M. J., & McDonald, D. W. (2023) · Designing Interactive Systems Conference · doi:10.1145/3563657.3596101 · Cited by 52
Seamful XAI: Operationalizing Seamful Design in Explainable AI
Ehsan, U., Liao, Q. V., Passi, S., Riedl, M. O., & Daumé, H. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637396 · Cited by 81
Self-citation
Enhancing Human-Robot Collaboration in Industry 4.0 with AI-driven HRI
Vemuri, N. V. N. (2023) · Power System Technology · doi:10.52783/pst.196 · Cited by 40
Machine learning advice in managerial decision-making: The overlooked role of decision makers’ advice utilization
Sturm, T., Pumplun, L., Gerlach, J. P., Kowalczyk, M., & Buxmann, P. (2023) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2023.101790 · Cited by 48
Slide4N: Creating Presentation Slides from Computational Notebooks with Human-AI Collaboration
Wang, F., Liu, X., Liu, O., Neshati, A., Ma, T., Zhu, M., & Zhao, J. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580753 · Cited by 38
Data Governance as Success Factor for Data Science
Brous, P., Janssen, M., & Krans, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-44999-5_36 · Cited by 37
Untitled work
(2022) · doi:10.1145/3491102.3501868 · Cited by 36
Knowledge Graphs in Practice: Characterizing their Users, Challenges, and Visualization Opportunities
Li, H. X., Appleby, G., Brumar, C. D., Chang, R., & Suh, A. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3326904 · Cited by 57
Trust in AI-assisted Decision Making: Perspectives from Those Behind the System and Those for Whom the Decision is Made
Vereschak, O., Alizadeh, F., Bailly, G., & Caramiaux, B. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642018 · Cited by 50
Using Design Fiction Memos to Analyze UX Professionals’ Values Work Practices
Wong, R. Y. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445709 · Cited by 33
Public Health Calls for/with AI: An Ethnographic Perspective
Ismail, A., Thakkar, D., Madhiwalla, N., & Kumar, N. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610203 · Cited by 38
AutoML in The Wild: Obstacles, Workarounds, and Expectations
Sun, Y., Song, Q., Gui, X., Ma, F., & Wang, T. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581082 · Cited by 31
To Write Code: The Cultural Fabrication of Programming Notation and Practice
Arawjo, I. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376731 · Cited by 32
Towards a Human-Centred Artificial Intelligence Maturity Model
Hartikainen, M., Väänänen, K., & Olsson, T. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3585752 · Cited by 35
Bottom-Up Organizing with Tools from On High: Understanding the Data Practices of Labor Organizers
Khovanskaya, V. D., Sengers, P., & Dombrowski, L. S. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376185 · Cited by 31
Where Do Stories Come From? Examining the Exploration Process in Investigative Data Journalism
Showkat, D., & Baumer, E. P. S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479534 · Cited by 33
"We Have No Idea How Models will Behave in Production until Production": How Engineers Operationalize Machine Learning
Shankar, S., Garcia, R., Hellerstein, J. M., & Parameswaran, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3653697 · Cited by 106
Eliciting Best Practices for Collaboration with Computational Notebooks
Quaranta, L., Calefato, F., & Lanubile, F. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512934 · Cited by 28
Documenting Data Production Processes: A Participatory Approach for Data Work
Miceli, M., Yang, T., Alvarado Garcia, A., Posada, J., Wang, S. M., Pohl, M., & Hanna, A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555623 · Cited by 28
Explanatory Interactive Machine Learning
Pfeuffer, N., Baum, L., Stammer, W., Abdel-Karim, B. M., Schramowski, P., Bucher, A. M., Hügel, C., Rohde, G. G. U., Kersting, K., & Hinz, O. F. (2023) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00806-x · Cited by 38
How Much Automation Does a Data Scientist Want?Preprint
Wang, D., Liao, Q. V., Zhang, Y., Khurana, U., Samulowitz, H., Park, S., Muller, M. J., & Amini, L. D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.03970 · Cited by 57
Model Compression in Practice: Lessons Learned from Practitioners Creating On-device Machine Learning Experiences
Hohman, F., Kery, M. B., Ren, D., & Moritz, D. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642109 · Cited by 33
Beyond Checklist Approaches to Ethics in Design
Wong, R. Y., Boyd, K., Metcalf, J., & Shilton, K. (2020) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3406865.3418590 · Cited by 27
Translation as (Re)mediation: How Ethnic Community-Based Organizations Negotiate Legitimacy
Sum, C. M., Tran, A.-T., Lin, J., Kuo, R., Bennett, C. L., Harrington, C., & Fox, S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581280 · Cited by 29
Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer VisionPreprint
Miceli, M., Schuessler, M., & Yang, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.14886 · Cited by 26
The Affective Growth of Computer Vision
Su, N. M., & Crandall, D. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00917 · Cited by 25
Model Sketching: Centering Concepts in Early-Stage Machine Learning Model Design
Lam, M. S., Ma, Z., Li, A., Freitas, I., Wang, D., Landay, J. A., & Bernstein, M. S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581290 · Cited by 31
An Exploratory Study on Visual Exploration of Model Simulations by Multiple Types of Experts
Boukhelifa, N., Bezerianos, A., Trelea, I.-C., Perrot, N., & Lutton, É. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300874 · Cited by 25
Trust in AutoML: Exploring Information Needs for Establishing Trust in Automated Machine Learning SystemsPreprint
Drozdal, J., Weisz, J. D., Wang, D., Dass, G., Yao, B., Zhao, C., Muller, M. J., Lin, J., & Su, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.06509 · Cited by 25
Studying Up Machine Learning Data: Why Talk About Bias When We Mean Power?Preprint
Miceli, M., Posada, J., & Yang, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.08131 · Cited by 25
How Domain Experts Work with Data: Situating Data Science in the Practices and Settings of Craftwork
Jung, J. Y., Steinberger, T., King, J. L., & Ackerman, M. S. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512905 · Cited by 24
Communicating Uncertainty in Digital Humanities Visualization Research
Panagiotidou, G., Lamqaddam, H., Poblome, J., Brosens, K., Verbert, K., & Moere, A. V. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3209436 · Cited by 24
An Empirical Study on the Usage of Automated Machine Learning Tools
Majidi, F., Openja, M., Khomh, F., & Li, H. (2022) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme55016.2022.00014 · Cited by 24
Who Should Act? Distancing and Vulnerability in Technology Practitioners' Accounts of Ethical Responsibility
Popova, K., Figueras, C., Höök, K., & Lampinen, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637434 · Cited by 25
Acting with Inherently Uncertain Data: Practices of Data-Centric Knowing
Marius Mikalsen, A., & Monteiro, E. (2021) · Journal of the Association for Information Systems · doi:10.17705/1jais.00722 · Cited by 22
Missed Opportunities for Human-Centered AI Research: Understanding Stakeholder Collaboration in Mental Health AI Research
Yoo, D. W., Woo, H., Pendse, S. R., Lu, N. Y., Birnbaum, M. L., Abowd, G. D., & Choudhury, M. D. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637372 · Cited by 23
The New Reality of Reproducibility: The Role of Data Work in Scientific Research
Feinberg, M., Sutherland, W., Nelson, S. B., Jarrahi, M. H., & Rajasekar, A. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392840 · Cited by 24
Scoping Fairness Objectives and Identifying Fairness Metrics for Recommender Systems: The Practitioners’ Perspective
Smith, J. J., Beattie, L., & Cramer, H. S. M. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583204 · Cited by 27
Are Metrics Enough? Guidelines for Communicating and Visualizing Predictive Models to Subject Matter Experts
Suh, A., Appleby, G., Anderson, E. W., Finelli, L. A., Chang, R., & Cashman, D. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3259341 · Cited by 20
Situating Datasets: Making Public Eviction Data Actionable for Housing Justice
Tran, A.-T., Guo, G., Taylor, J., Chan, K. A., Raymond, E. L., & DiSalvo, C. F. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642452 · Cited by 24
Technical, Regulatory, Economic, and Trust Issues Preventing Successful Integration of Sensors into the Mainstream Consumer Wearables Market
Devine, J. K., Schwartz, L. P., & Hursh, S. R. (2022) · Sensors · doi:10.3390/s22072731 · Cited by 22
Characterizing Practices, Limitations, and Opportunities Related to Text Information Extraction Workflows: A Human-in-the-loop Perspective
Rahman, S., & Kandogan, E. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502068 · Cited by 22
Chasing Luck: Data-driven Prediction, Faith, Hunch, and Cultural Norms in Rural Betting Practices
Sultana, S., Mozumder, M. M. H., & Ahmed, S. I. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445047 · Cited by 20
Decoding Human Behavior with Big Data? Critical, Constructive Input from the Decision Sciences
Katsikopoulos, K. V., & Canellas, M. C. (2022) · AI Magazine · doi:10.1002/aaai.12034 · Cited by 22
Workflow analysis of data science code in public GitHub repositories
Ramasamy, D., Sarasua, C., Bacchelli, A., & Bernstein, A. (2022) · Empirical Software Engineering · doi:10.1007/s10664-022-10229-z · Cited by 19
“Who is the right homeless client?”: Values in Algorithmic Homelessness Service Provision and Machine Learning Research
Showkat, D., Smith, A. D. R., Wang, L., & To, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581010 · Cited by 25
Designing a Data Visualisation for Interdisciplinary Scientists. How to Transparently Convey Data Frictions?
Panagiotidou, G., Poblome, J., Aerts, J. A., & Moere, A. V. (2022) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-022-09432-9 · Cited by 19
Unpacking Trust Dynamics in the LLM Supply Chain: An Empirical Exploration to Foster Trustworthy LLM Production & Use
Balayn, A., Yurrita, M., Rancourt, F., Casati, F., & Gadiraju, U. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713787 · Cited by 37
“It’s Like the Value System in the Loop”: Domain Experts’ Values Expectations for NLP Automation
Showkat, D., & Baumer, E. P. S. (2022) · Designing Interactive Systems Conference · doi:10.1145/3532106.3533483 · Cited by 21
Integration of data science with product design towards data-driven design
Liu, A., Lu, S. C.-Y., Tao, F., & Anwer, N. (2024) · CIRP Annals · doi:10.1016/j.cirp.2024.06.003 · Cited by 21
Breaking the vicious circle: A case study on why AI for software analytics and business intelligence does not take off in practice
Figalist, I., Elsner, C., Bosch, J., & Olsson, H. H. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111135 · Cited by 17
Implicit Error, Uncertainty and Confidence in Visualization: An Archaeological Case Study
Panagiotidou, G., Vandam, R., Poblome, J., & Moere, A. V. (2021) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2021.3088339 · Cited by 19

Authors: Passi, S., & Jackson, S. J.
Affiliations: Cornell University (US)
ORCID: Passi, Samir
Venue: Proceedings of the ACM on Human-Computer Interaction
Publisher: Association for Computing Machinery (ACM)
Published: 2018-11
Volume: 2
Issue: CSCW
Pages: 1-28
ISSN: 2573-0142 (ISSN-L)
DOI: 10.1145/3274405
OpenAlex: W4288083705
Cited by: 225
References: 100
Citations by year: 2026: 18 · 2025: 34 · 2024: 34 · 2023: 38 · 2022: 39 · 2021: 32 · 2020: 24 · 2019: 6
FWCI: 31.17
Type: journal article
Language: en
Open access: bronze
doi.org
Licence: www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations: Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Funders: National Science Foundation (CHS-1526155)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
Social Sciences › Business, Management and Accounting › Management Information Systems › Information Systems Theories and Implementation
Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords: applied data science, trust negotiation, credibility, skepticism, model interpretability, data quality, uncertainty, collaboration, critical data studies, sociology of science, computer-supported cooperative work

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks