Trust in Data Science: Collaboration, Translation, and Accountability in Corporate Data Science Projects
Open accessAbstract
The trustworthiness of data science systems in applied and real-world settings emerges from the resolution of specific tensions through situated, pragmatic, and ongoing forms of work. Drawing on research in CSCW, critical data studies, and history and sociology of science, and six months of immersive ethnographic fieldwork with a corporate data science team, we describe four common tensions in applied data science work: (un)equivocal numbers, (counter)intuitive knowledge, (in)credible data, and (in)scrutable models. We show how organizational actors establish and re-negotiate trust under messy and uncertain analytic conditions through practices of skepticism, assessment, and credibility. Highlighting the collaborative and heterogeneous nature of real-world data science, we show how the management of trust in applied corporate data science settings depends not only on pre-processing and quantification, but also on negotiation and translation. We conclude by discussing the implications of our findings for data science research and practice, both within and beyond CSCW.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The trustworthiness of data science outcomes in practice is often affected by tensions arising through ongoing forms of work.
“However, the trustworthiness of data science outcomes in practice is often a ected by tensions arising through ongoing forms of work (Passi and Jackson 2018).”
From Trusted Decision-Making: Data Governance for Creating Trust in Data Science Decision Outcomes (2020) · cites this paper as Passi and Jackson 2018
Data governance establishes data management processes which manage data quality.
“Data governance establishes data management processes which manage data quality (Passi and Jackson 2018)”
From Trusted Decision-Making: Data Governance for Creating Trust in Data Science Decision Outcomes (2020) · cites this paper as Passi and Jackson 2018
The ‘Rationalizing and Reorganizing Data’ section discusses how staff assess the credibility of suspect data, and the implications section calls for documenting data decisions. Neither passage says that data governance establishes management processes to manage data quality; searches for that relationship and for ‘data quality governance management processes’ found no supporting passage in our copy. (LLM, conf=0.70)
Experimental — accuracy not yet measuredA focus on ensuring AI competence could increase end-users' confidence in AI, though it does not necessarily imply trust.
“For example, there is a particular focus on ensuring competence of AI (i.e., that it performs accurately and eciently) [20,41], which could potentially lead to increase in end-users' condence in AI [61] (though without necessarily always implying trust [39]).”
From Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust (2023) · cites this paper as [39]
Processes for data accuracy and accountability require resource allocation for auditing.
“Requires resource allocation for auditing”
From AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications (2023) · cites this paper as [255]
Processes for data accuracy and accountability ensure reliability of data.
“Ensures reliability of data”
From AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications (2023) · cites this paper as [255]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Discovery of Grounded Theory; Strategies for Qualitative Research
Glaser, B. G., Strauss, A. L., & Strutzel, E. (1968) · Nursing Research · doi:10.1097/00006199-196807000-00014 · Cited by 39,449 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
Wallace, A. F. C., & Garfinkel, H. (1968) · American Sociological Review · doi:10.2307/2092245 · Cited by 12,413 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505 · see citing works
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682 · see citing works
The mythos of model interpretability
Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · see citing works
Busfield, J., & Garfinkel, H. (1968) · British Journal of Sociology · doi:10.2307/588840 · Cited by 3,825 · see citing works
The Audit Society: Rituals of Verification
Quam, L. (1998) · BMJ · doi:10.1136/bmj.316.7133.787a · Cited by 3,608 · see citing works
The ethics of algorithms: Mapping the debate
Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016) · Big Data & Society · doi:10.1177/2053951716679679 · Cited by 3,432 · cited 1× · see citing works
“The increased management of data science systems is a priority [65,96,105], but “the complex decision-making structure” needed to manage them often exceeds “the human and organizational resources available for oversight” [68].”
BackgroundGoodwin, C. (1994) · American Anthropologist · doi:10.1525/aa.1994.96.3.02a00100 · Cited by 3,345 · see citing works
Sociomaterial practices: exploring technology at work
Orlikowski, W. J. (2007) · Organization Studies · doi:10.1177/0170840607081138 · Cited by 3,248 · see citing works
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 1× · see citing works
“One limiting factor is the opacity of data science systems [97,100] arising from a variety of reasons: (a) algorithms are often trade-secrets, (b) data science requires specialized knowledge, and (c) novel analytic methods remain conceptually challenging [14].”
BackgroundOn Justification: Economies of Worth
Boltanski, L., & Thévenot, L. (2006) · doi:10.1515/9781400827145 · Cited by 2,709 · see citing works
Strathern, M. (2003) · Routledge eBooks · doi:10.4324/9780203449721 · Cited by 2,665 · see citing works
The Parable of Google Flu: Traps in Big Data Analysis
Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014) · Science · doi:10.1126/science.1248506 · Cited by 2,523 · cited 1× · see citing works
“Several data correlations established in large datasets are neither reproducible nor falsifiable [45,61].”
BackgroundBig Data, new epistemologies and paradigm shifts
Kitchin, R. (2014) · Big Data & Society · doi:10.1177/2053951714528481 · Cited by 2,355 · see citing works
Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life
Cohen, I. B., Shapin, S., & Schaffer, S. (1987) · The American Historical Review · doi:10.2307/1869945 · Cited by 2,150 · see citing works
Ananny, M., & Crawford, K. (2018) · New Media & Society · doi:10.1177/1461444816676645 · Cited by 1,836 · see citing works
Robustness in the Strategy of Scientific Model Building.
Box, G. (1979) · Elsevier eBooks · doi:10.1016/b978-0-12-438150-6.50018-2 · Cited by 1,435 · see citing works
Daston, L., & Galison, P. (1992) · Representations · doi:10.2307/2928741 · Cited by 1,269 · see citing works
Espeland, W. N., & Stevens, M. L. (2008) · European Journal of Sociology · doi:10.1017/s0003975609000150 · Cited by 1,265 · see citing works
Friedman, B., & Nissenbaum, H. (1996) · ACM Transactions on Information Systems · doi:10.1145/230538.230561 · Cited by 1,203 · cited 1× · see citing works
“professional values, but also from the emergent sites of algorithmic re-use as opposed to their contexts of development [29,43,51].”
BackgroundTrust in numbers: The pursuit of objectivity in science and public life
Matthews, J. R. (1996) · Journal of the History of the Behavioral Sciences · doi:10.1002/(sici)1520-6696(199607)32:3 · Cited by 1,004 · see citing works
The Politics of Large Numbers: A History of Statistical Reasoning
Houser, J. A., Desrosières, A., & Naish, C. (1999) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2654208 · Cited by 910 · see citing works
On Justification: Economies of Worth
Creppell, I. (2007) · Perspectives on Politics · doi:10.1017/s1537592707070168 · Cited by 906 · see citing works
The Sense of Dissonance: Accounts of Worth in Economic Life
Stark, D. (2009) · Project Muse (Johns Hopkins University) · Cited by 869 · see citing works
The Data Revolution: Big Data, Open Data, Data Infrastructures and Their Consequences
(2015) · Online Information Review · doi:10.1108/oir-01-2015-0011 · Cited by 785 · see citing works
Studies of the Routine Grounds of Everyday Activities
Garfinkel, H. (1964) · Social Problems · doi:10.2307/798722 · Cited by 776 · see citing works
The ethics of big data: current and foreseeable issues in biomedical contexts
Mittelstadt, B. D., & Floridi, L. (2016) · Science and Engineering Ethics · doi:10.1007/s11948-015-9652-2 · Cited by 714 · see citing works
Model-Agnostic Interpretability of Machine Learning
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · arXiv · doi:10.48550/arxiv.1606.05386 · Cited by 699 · see citing works
'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions
Binns, R., Van Kleek, M., Veale, M., Lyngs, U., Zhao, J., & Shadbolt, N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173951 · Cited by 691 · cited 1× · see citing works
“This is particularly challenging because the systems are not usually designed “with the information needs of those individuals facing significant personal consequences of model outputs” [7:10].”
BackgroundBias in algorithmic filtering and personalization
Bozdag, E. (2013) · Ethics and Information Technology · doi:10.1007/s10676-013-9321-6 · Cited by 649 · see citing works
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · see citing works
What makes Big Data, Big Data? Exploring the ontological characteristics of 26 datasets
Kitchin, R., & McArdle, G. (2016) · Big Data & Society · doi:10.1177/2053951716631130 · Cited by 606 · see citing works
There is a blind spot in AI research
Crawford, K., & Calo, R. (2016) · Nature · doi:10.1038/538311a · Cited by 580 · see citing works
The Black Box Society: The Secret Algorithms That Control Money and Information
(2015) · Choice Reviews Online · doi:10.5860/choice.190706 · Cited by 533 · see citing works
Algorithms and their others: Algorithmic culture in context
Dourish, P. (2016) · Big Data & Society · doi:10.1177/2053951716665128 · Cited by 481 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2019) · CHANCE · doi:10.1080/09332480.2019.1579573 · Cited by 473 · see citing works
Zarsky, T. (2016) · Science, Technology, & Human Values · doi:10.1177/0162243915605575 · Cited by 422 · see citing works
Star, S. L., & Ruhleder, K. (1994) · Proceedings of the 1994 ACM conference on Computer supported cooperative work - CSCW '94 · doi:10.1145/192844.193021 · Cited by 412 · see citing works
Where are human subjects in big data research? the emerging ethics divide
Metcalf, J., & Crawford, K. (2016) · Big Data & Society · doi:10.1177/2053951716650211 · Cited by 394 · see citing works
The Intuitive Appeal of Explainable Machines
Selbst, A. D., & Barocas, S. (2018) · doi:10.2139/ssrn.3126971 · Cited by 392 · see citing works
Ten simple rules for responsible big data research
Zook, M., Barocas, S., boyd, D., Crawford, K., Keller, E., Gangadharan, S. P., Goodman, A. A., Hollander, R. D., Koenig, B. A., Metcalf, J., et al. (2017) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1005399 · Cited by 350 · see citing works
The legal and ethical concerns that arise from using complex predictive analytics in health care
Cohen, I. G., Amarasingham, R., Shah, A., Xie, B., & Lo, B. (2014) · Health Affairs · doi:10.1377/hlthaff.2014.0048 · Cited by 300 · see citing works
What difference does quantity make? On the epistemology of Big Data in biology
Leonelli, S. (2014) · Big Data & Society · doi:10.1177/2053951714534395 · Cited by 293 · see citing works
Leviathan and the Air-Pump: Hobbes, Boyle and the Experimental Life
Hannaway, O., Shapin, S., & Schaffer, S. (1988) · Technology and Culture · doi:10.2307/3105534 · Cited by 291 · see citing works
A Social History of Truth: Civility and Science in Seventeenth-Century England.
Hacking, I., & Shapin, S. (1995) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2077696 · Cited by 270 · see citing works
Just good enough data: Figuring data citizenships through air pollution sensing and data stories
Gabrys, J., Pritchard, H. V., & Barratt, B. (2016) · Big Data & Society · doi:10.1177/2053951716679677 · Cited by 264 · see citing works
Amershi, S., Chickering, M., Drucker, S. M., Lee, B., Simard, P., & Suh, J. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702509 · Cited by 263 · see citing works
Metrology: The Creation of Universality by the Circulation of Particulars
O'Connell, J. (1993) · Social Studies of Science · doi:10.1177/030631293023001005 · Cited by 249 · see citing works
Mackenzie, A. (2017) · The MIT Press eBooks · doi:10.7551/mitpress/10302.001.0001 · Cited by 242 · see citing works
Investigating How Experienced UX Designers Effectively Work with Machine Learning
Yang, Q., Scuito, A., Zimmerman, J., Forlizzi, J., & Steinfeld, A. M. (2018) · Designing Interactive Systems Conference · doi:10.1145/3196709.3196730 · Cited by 230 · see citing works
Cordelia’s Love: Credibility and the Social Studies of Science
Shapin, S. (1995) · Perspectives on Science · doi:10.1162/posc_a_00484 · Cited by 228 · see citing works
Adaptive least squares support vector machines filter for hand tremor canceling in microsurgery
Liu, Z., Wu, Q., Zhang, Y., & Chen, C. L. P. (2011) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-011-0012-5 · Cited by 214 · see citing works
What Counts as Scientific Data? A Relational Framework
Leonelli, S. (2015) · Philosophy of Science · doi:10.1086/684083 · Cited by 213 · see citing works
Willson, M. (2016) · Information Communication & Society · doi:10.1080/1369118x.2016.1200645 · Cited by 205 · see citing works
Student privacy in learning analytics: An information ethics perspective
Rubel, A., & Jones, K. M. L. (2014) · The Information Society · doi:10.1080/01972243.2016.1130502 · Cited by 203 · see citing works
The Sense of Dissonance: Accounts of Worth in Economic Life
Freeland, R. F. (2011) · Contemporary Sociology A Journal of Reviews · doi:10.1177/0094306110396847xx · Cited by 180 · see citing works
Neff, G., Tanweer, A., Fiore-Gartland, B., & Osburn, L. (2017) · Big Data · doi:10.1089/big.2016.0050 · Cited by 173 · see citing works
The Ethics of Big Data: Current and Foreseeable Issues in Biomedical Contexts
Mittelstadt, B., & Floridi, L. (2016) · Law, governance and technology series · doi:10.1007/978-3-319-33525-4_19 · Cited by 161 · see citing works
Models Of and Models For: Theory and Practice in Contemporary Biology
Keller, E. F. (2000) · Philosophy of Science · doi:10.1086/392810 · Cited by 160 · see citing works
Known or knowing publics? Social media data mining and the question of public agency
Kennedy, H., & Moss, G. (2015) · Big Data & Society · doi:10.1177/2053951715611145 · Cited by 148 · see citing works
Kery, M. B., Horvath, A., & Myers, B. A. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025626 · Cited by 146 · see citing works
Studies of the Routine Grounds of Everyday Activities
Garfinkel, H. (1964) · Social Problems · doi:10.1525/sp.1964.11.3.03a00020 · Cited by 137 · see citing works
Machine Learners: Archaeology of a Data Practice
Mackenzie, A. (2017) · Cited by 125 · see citing works
Toward human-centered algorithm design
Baumer, E. P. (2017) · Big Data & Society · doi:10.1177/2053951717718854 · Cited by 125 · see citing works
The Trials and Tribulations of Working with Structured Data
Koesten, L. M., Kacprzak, E., Tennison, J., & Simperl, E. P. B. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025838 · Cited by 122 · see citing works
Passi, S., & Jackson, S. (2017) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2998181.2998331 · Cited by 113 · see citing works
Self-citationLet’s stop trying to be “sexy” – preparing managers for the (big) data-driven business era
Carillo, K. (2017) · Business Process Management Journal · doi:10.1108/bpmj-09-2016-0188 · Cited by 105 · see citing works
Datatrust: Or, the political quest for numerical evidence and the epistemologies of Big Data
Rieder, G., & Simon, J. (2016) · Big Data & Society · doi:10.1177/2053951716649398 · Cited by 104 · see citing works
Society's choices: Social and ethical decision making in biomedicine
Adler, S. R. (1996) · Social Science & Medicine · doi:10.1016/0277-9536(96)82262-3 · Cited by 99 · see citing works
Balka, E., & Wagner, I. (2006) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/1180875.1180912 · Cited by 88 · see citing works
The conundrum of police officer-involved homicides: Counter-data in Los Angeles County
Currie, M., Paris, B. S., Pasquetto, I., & Pierre, J. (2016) · Big Data & Society · doi:10.1177/2053951716663566 · Cited by 86 · see citing works
Data Flakes: An Afterword to “Raw Data” Is an Oxymoron
Bowker, G. C. (2013) · The MIT Press eBooks · doi:10.7551/mitpress/9302.003.0011 · Cited by 84 · see citing works
Autonomy and Solidarity Interviews
Habermas, J., & Dews, P. (1986) · Verso eBooks · Cited by 81 · see citing works
Can we trust Big Data? Applying philosophy of science to software
Symons, J., & Alvarado, R. (2016) · Big Data & Society · doi:10.1177/2053951716664747 · Cited by 69 · see citing works
Kay, M., Patel, S., & Kientz, J. A. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702603 · Cited by 59 · see citing works
Developing a Research Agenda for Human-Centered Data Science
Aragon, C., Hutto, C. J., Echenique, A., Fiore-Gartland, B., Huang, Y. K., Kim, J., Neff, G., Xing, W., & Bayer, J. (2016) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2818052.2855518 · Cited by 53 · see citing works
A Fresh Look at Instrumentation an Introduction
Joerges, B., & Shinn, T. (2001) · Springer eBooks · doi:10.1007/978-94-010-9032-2_1 · Cited by 42 · see citing works
Who Needs Stories if You Can Get the Data? ISPs in the Era of Big Number Crunching
Hildebrandt, M. (2011) · Philosophy & Technology · doi:10.1007/s13347-011-0041-8 · Cited by 40 · see citing works
Knowles, B., Rouncefield, M., Harding, M., Davies, N., Blair, L., Hannon, J. T., Walden, J., & Wang, D. (2015) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2675133.2675154 · Cited by 37 · see citing works
Programs as Black-Box ExplanationsPreprint
Singh, S., Ribeiro, M. T., & Guestrin, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.07579 · Cited by 36 · see citing works
What’s your ML test score? A rubric for ML production systems
Breck, E. J., Cai, S., Nielsen, E., Salib, M., & Sculley, D. (2016) · Cited by 33 · see citing works
Small decisions with big impact on data analytics
Diesner, J. (2015) · Big Data & Society · doi:10.1177/2053951715617185 · Cited by 32 · see citing works
Taylor, A. S., Lindley, S. E., Regan, T. W., & Sweeney, D. (2014) · Big Data & Society · doi:10.1177/2053951714539278 · Cited by 25 · see citing works
Increasing Users' Confidence in Uncertain Data by Aggregating Data from Multiple Sources
Greis, M., Avci, E., Schmidt, A., & Machulla, T. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025998 · Cited by 19 · see citing works
Dasgupta, A., Burrows, S. M., Han, K., & Rasch, P. J. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025882 · Cited by 17 · see citing works
Thoreau, F. (2016) · Big Data & Society · doi:10.1177/2053951716670189 · Cited by 16 · see citing works
alt.metadata.health: Ontological Context for Data Use and Integration
Schuurman, N., & Balka, E. (2008) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-008-9084-5 · Cited by 15 · see citing works
Ten Simple Rules for Responsible Big Data Research
Zook, M., Barocas, S., boyd, D., Crawford, K., Keller, E., Gangadharan, S. P., Goodman, A. A., Hollander, R. D., Koenig, B. A., Metcalf, J., et al. (2022) · doi:10.1002/9781119815075.ch44 · Cited by 14 · see citing works
Will the future of knowledge work automation transform personalized medicine?
Naik, G., & Bhide, S. S. (2014) · Applied & Translational Genomics · doi:10.1016/j.atg.2014.05.003 · Cited by 13 · see citing works
Knowles, B., Harding, M., Blair, L., Davies, N., Hannon, J. T., Rouncefield, M., & Walden, J. (2014) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2531602.2531699 · Cited by 13 · see citing works
Ontic Occlusion and Exposure in Sociotechnical Systems
Knobel, C. P. (2010) · Deep Blue (University of Michigan) · Cited by 11 · see citing works
Veale, M. (2017) · doi:10.31235/osf.io/6cdhe · Cited by 11 · see citing works
Water models and water politics
Jackson, S. (2006) · Annual International Conference on Digital Government Research · doi:10.1145/1146598.1146632 · Cited by 9 · see citing works
Self-citationStudent Privacy in Learning Analytics: An Information Ethics Perspective
Rubel, A. P., & Jones, K. M. L. (2014) · PhilPapers (PhilPapers Foundation) · Cited by 2 · see citing works
Society's Choices — Social and Ethical Decision Making in Biomedicine
Toon, P. D. (1997) · Journal of Medical Ethics · doi:10.1136/jme.23.3.195 · Cited by 2 · see citing works
"From What I see, this makes sense": Seeing meaning in algorithmic resultsPreprint
Passi, S., & Sengers, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.07844 · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
1 of the 100 references couldn’t be matched to a known paper.
Research works printed but not linked: 13
- Kathy Charmaz. 2014. Constructing Grounded Theory (Introducing Qualitative Methods series) 2nd Edition. Sage, London.
- Harry M. Collins and Robert Evans. 2007. Rethinking Expertise. University of Chicago Press, Chicago.
- Lorraine Daston and Peter Galison. 2007. Objectivity. MIT Press, Cambridge, Massachusetts.
- John Dewey. 1939. Theory of Valuation. University of Chicago Press, Chicago.
- Tarleton Gillespie. 2014. The Relevance of Algorithms. In Media Technologies: Essays on Communication, Materiality, and Society, Tarleton Gillespie, Pablo J. Boczkowski and Kirsten A. Foot (eds.). MIT Press, Cambridge, 167-194.
- Lisa Gitelman. 2006. Raw Data is an Oxymoron. MIT Press, MA.
- Ian Hacking. 1990. The Taming of Chance. Cambridge University Press, Cambridge.
- Donna Haraway. 2007. Modest_Witness@Second_Millenium. Routledge, London.
- Edmund Husserl. 1970. The Crisis of European Sciences and Transcendental Philosophy. Northwestern University Press, Evanston.
- Mary Beth Kery, Amber Horvath, and Brad Myers. 2017. Variolite: Supporting Exploratory Programming by Data Scientists. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17), 1265-1276.
- Bruno Latour. 1987. Science in Action. Harvard University Press, Cambridge, MA.
- Samir Passi and Steven J. Jackson. 2017. Data Vision: Learning to See Through Algorithmic Abstraction. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '17), 2436-2447.
- Rita Raley. 2013. Dataveillance and Counterveillance. In"Raw Data" Is an Oxymoron, Lisa Gitelman (ed.). MIT Press, Cambridge, Massachusetts, 121-145.
Cited by
Sorted by citations
“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI
Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445518 · Cited by 781 · see citing works
Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI
Wang, D., Weisz, J. D., Muller, M., Ram, P., Geyer, W., Dugan, C., Tausczik, Y., Samulowitz, H., & Gray, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359313 · Cited by 342 · see citing works
How do Data Science Workers Collaborate? Roles, Workflows, and Tools
Zhang, A. X., Muller, M. J., & Wang, D. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392826 · Cited by 268 · see citing works
The Fallacy of AI Functionality
Raji, I. D., Kumar, I. E., Horowitz, A., & Selbst, A. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533158 · Cited by 251 · see citing works
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction
Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., et al. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604673 · Cited by 279 · see citing works
Problem Formulation and Fairness
Passi, S., & Barocas, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287567 · Cited by 283 · see citing works
Self-citationHow to Evaluate Trust in AI-Assisted Decision Making? A Survey of Empirical Methodologies
Vereschak, O., Bailly, G., & Caramiaux, B. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476068 · Cited by 229 · see citing works
Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support
Madaio, M., Egede, L., Subramonyam, H., Wortman Vaughan, J., & Wallach, H. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512899 · Cited by 156 · see citing works
AI Fairness in Data Management and Analytics: A Review on Challenges, Methodologies and Applications
Chen, P., Wu, L., & Wang, L. (2023) · Applied Sciences · doi:10.3390/app131810258 · Cited by 155 · see citing works
How AI Developers Overcome Communication Challenges in a Multidisciplinary Team: A Case Study
Piorkowski, D., Park, S., Wang, A. Y., Wang, D., Muller, M., & Portnoy, F. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449205 · Cited by 151 · see citing works
Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision
Miceli, M., Schuessler, M., & Yang, T. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3415186 · Cited by 149 · see citing works
Saxena, D., Badillo-Urquiola, K., Wisniewski, P. J., & Guha, S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476089 · Cited by 134 · see citing works
EXplainable Artificial Intelligence (XAI)—From Theory to Methods and Applications
Ortigossa, E. S., Gonçalves, T., & Nonato, L. G. (2024) · IEEE Access · doi:10.1109/access.2024.3409843 · Cited by 124 · see citing works
The Who in XAI: How AI Background Shapes Perceptions of AI Explanations
Ehsan, U., Passi, S., Liao, Q. V., Chan, L., Lee, I.-H., Muller, M., & Riedl, M. O. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642474 · Cited by 123 · see citing works
Self-citationDrozdal, J., Weisz, J., Wang, D., Dass, G., Yao, B., Zhao, C., Muller, M., Ju, L., & Su, H. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377501 · Cited by 119 · see citing works
This Thing Called Fairness: Disciplinary Confusion Realizing a Value in Technology
Mulligan, D. K., Kroll, J. A., Kohli, N., & Wong, R. Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359221 · Cited by 111 · see citing works
Seeing Like a Toolkit: How Toolkits Envision the Work of AI Ethics
Wong, R. Y., Madaio, M. A., & Merrill, N. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579621 · Cited by 103 · see citing works
Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits
Deng, W. H., Nagireddy, M., Lee, M. S. A., Singh, J., Wu, Z. S., Holstein, K., & Zhu, H. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533113 · Cited by 99 · see citing works
The Making of Data Commodities: Data Analytics as an Embedded Process
Aaltonen, A., Alaimo, C., & Kallinikos, J. (2021) · Journal of Management Information Systems · doi:10.1080/07421222.2021.1912928 · Cited by 95 · see citing works
Documenting Computer Vision Datasets
Miceli, M., Yang, T., Naudts, L., Schuessler, M., Serbanescu, D. A., & Hanna, A. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445880 · Cited by 94 · see citing works
Tactics of Soft Resistance in User Experience Professionals' Values Work
Wong, R. Y. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479499 · Cited by 82 · see citing works
Making data science systems work
Passi, S., & Sengers, P. (2020) · Big Data & Society · doi:10.1177/2053951720939605 · Cited by 81 · see citing works
Self-citationDesigning Ground Truth and the Social Life of Labels
Muller, M., Wolf, C. T., Andres, J., Desmond, M., Joshi, N. N., Ashktorab, Z., Sharma, A., Brimijoin, K., Pan, Q., Duesterwald, E., et al. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445402 · Cited by 83 · see citing works
Governing Artificial Intelligence in an Age of Inequality
Sampath, P. G. (2021) · Global Policy · doi:10.1111/1758-5899.12940 · Cited by 71 · see citing works
Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory
Kaur, H., Adar, E., Gilbert, É., & Lampe, C. A. C. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533135 · Cited by 73 · see citing works
Talking datasets – Understanding data sensemaking behaviours
Koesten, L. M., Gregory, K., Groth, P., & Simperl, E. P. B. (2020) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2020.102562 · Cited by 7 · see citing works
How Experienced Designers of Enterprise Applications Engage AI as a Design Material
Yildirim, N., Kass, A., Tung, T., Upton, C., Costello, D., Giusti, R., Lacin, S., Lovic, S., O'Neill, J. M., Meehan, R. O., et al. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517491 · Cited by 71 · see citing works
Nagahisarchoghaei, M., Nur, N., Cummins, L., Nur, N., Karimi, M. M., Nandanwar, S., Bhattacharyya, S., & Rahimi, S. (2023) · Electronics · doi:10.3390/electronics12051092 · Cited by 76 · see citing works
Generative AI in the Wild: Prospects, Challenges, and Strategies
Sun, Y., Jang, E., Ma, F., & Wang, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642160 · Cited by 64 · see citing works
Heger, A., Marquis, L. B., Vorvoreanu, M., Wallach, H., & Vaughan, J. W. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555760 · Cited by 60 · see citing works
Weidele, D. K. I., Weisz, J. D., Oduor, E., Muller, M. J., Andrés, J., Gray, A., & Wang, D. (2020) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3377325.3377538 · Cited by 58 · see citing works
Facilitating Knowledge Sharing from Domain Experts to Data Scientists for Building NLP Models
Park, S., Wang, A. Y., Kawas, B., Liao, Q. V., Piorkowski, D., & Danilevsky, M. (2021) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3397481.3450637 · Cited by 61 · see citing works
Deng, W. H., Yildirim, N., Chang, M., Eslami, M., Holstein, K., & Madaio, M. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594037 · Cited by 57 · see citing works
Making It Possible for the Auditing of AI: A Systematic Review of AI Audits and AI Auditability
Li, Y., & Goel, S. (2024) · Information Systems Frontiers · doi:10.1007/s10796-024-10508-8 · Cited by 65 · see citing works
Wong, R. Y., Chong, A., & Aspegren, R. C. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579515 · Cited by 65 · see citing works
Forgetting Practices in the Data Sciences
Muller, M. J., & Strohmayer, A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517644 · Cited by 55 · see citing works
Fits and Starts: Enterprise Use of AutoML and the Role of Humans in the Loop
Crisan, A., & Fiore-Gartland, B. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445775 · Cited by 56 · see citing works
Banovic, N., Yang, Z., Ramesh, A., & Liu, A. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579460 · Cited by 55 · see citing works
Nahar, N., Zhang, H., Lewis, G. A., Zhou, S., & Kästner, C. (2023) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1109/cain58948.2023.00034 · Cited by 58 · see citing works
Lederman, A., Lederman, R., & Verspoor, K. M. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac121 · Cited by 49 · see citing works
A hunt for the Snark: Annotator Diversity in Data Practices
Kapania, S., Taylor, A. S., & Wang, D. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580645 · Cited by 50 · see citing works
Model Positionality and Computational Reflexivity: Promoting Reflexivity in Data Science
Cambo, S. A., & Gergle, D. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501998 · Cited by 49 · see citing works
Human-Centered Study of Data Science Work Practices
Muller, M. J., Feinberg, M., George, T., Jackson, S. J., John, B. E., Kery, M. B., & Passi, S. (2019) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3299018 · Cited by 48 · see citing works
Self-citationHuman-Centered AI Design in Reality: A Study of Developer Companies’ Practices
Hartikainen, M., Väänänen, K., Lehtiö, A., Ala-Luopa, S., & Olsson, T. (2022) · Nordic Human-Computer Interaction Conference · doi:10.1145/3546155.3546677 · Cited by 48 · see citing works
No! Re-imagining Data Practices Through the Lens of Critical Refusal
García, P. M., Sutherland, T., Salehi, N., Cifor, M., & Singh, A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3557997 · Cited by 49 · see citing works
Callisto: Capturing the "Why" by Connecting Conversations with Computational Narratives
Wang, A. Y., Wu, Z., Brooks, C. A., & Oney, S. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376740 · Cited by 44 · see citing works
Trusted Decision-Making: Data Governance for Creating Trust in Data Science Decision Outcomes
Brous, P., & Janssen, M. (2020) · Administrative Sciences · doi:10.3390/admsci10040081 · Cited by 43 · see citing works
“Data governance establishes data management processes which manage data quality (Passi and Jackson 2018) and compliance with relevant laws, directives, and policies (Cato et al. 2015).”
BackgroundInfluentialData Hunches: Incorporating Personal Knowledge into Visualizations
Lin, H., Akbaba, D., Meyer, M., & Lex, A. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3209451 · Cited by 43 · see citing works
Feng, K. J. K., Coppock, M. J., & McDonald, D. W. (2023) · Designing Interactive Systems Conference · doi:10.1145/3563657.3596101 · Cited by 41 · see citing works
Seamful XAI: Operationalizing Seamful Design in Explainable AI
Ehsan, U., Liao, Q. V., Passi, S., Riedl, M. O., & Daumé, H. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637396 · Cited by 61 · see citing works
Self-citationEnhancing Human-Robot Collaboration in Industry 4.0 with AI-driven HRI
Vemuri, N. V. N. (2023) · Power System Technology · doi:10.52783/pst.196 · Cited by 40 · see citing works
Sturm, T., Pumplun, L., Gerlach, J. P., Kowalczyk, M., & Buxmann, P. (2023) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2023.101790 · Cited by 45 · see citing works
Slide4N: Creating Presentation Slides from Computational Notebooks with Human-AI Collaboration
Wang, F., Liu, X., Liu, O., Neshati, A., Ma, T., Zhu, M., & Zhao, J. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580753 · Cited by 38 · see citing works
Data Governance as Success Factor for Data Science
Brous, P., Janssen, M., & Krans, R. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-44999-5_36 · Cited by 37 · see citing works
(2022) · doi:10.1145/3491102.3501868 · Cited by 36 · see citing works
Li, H. X., Appleby, G., Brumar, C. D., Chang, R., & Suh, A. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3326904 · Cited by 35 · see citing works
Vereschak, O., Alizadeh, F., Bailly, G., & Caramiaux, B. (2024) · International Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642018 · Cited by 58 · see citing works
Using Design Fiction Memos to Analyze UX Professionals’ Values Work Practices
Wong, R. Y. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445709 · Cited by 33 · see citing works
Public Health Calls for/with AI: An Ethnographic Perspective
Ismail, A., Thakkar, D., Madhiwalla, N., & Kumar, N. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610203 · Cited by 36 · see citing works
AutoML in The Wild: Obstacles, Workarounds, and Expectations
Sun, Y., Song, Q., Gui, X., Ma, F., & Wang, T. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581082 · Cited by 31 · see citing works
To Write Code: The Cultural Fabrication of Programming Notation and Practice
Arawjo, I. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376731 · Cited by 32 · see citing works
Towards a Human-Centred Artificial Intelligence Maturity Model
Hartikainen, M., Väänänen, K., & Olsson, T. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3585752 · Cited by 30 · see citing works
Bottom-Up Organizing with Tools from On High: Understanding the Data Practices of Labor Organizers
Khovanskaya, V. D., Sengers, P., & Dombrowski, L. S. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376185 · Cited by 31 · see citing works
Where Do Stories Come From? Examining the Exploration Process in Investigative Data Journalism
Showkat, D., & Baumer, E. P. S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479534 · Cited by 30 · see citing works
Shankar, S., Garcia, R., Hellerstein, J. M., & Parameswaran, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3653697 · Cited by 106 · see citing works
Eliciting Best Practices for Collaboration with Computational Notebooks
Quaranta, L., Calefato, F., & Lanubile, F. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512934 · Cited by 28 · see citing works
Documenting Data Production Processes: A Participatory Approach for Data Work
Miceli, M., Yang, T., Alvarado Garcia, A., Posada, J., Wang, S. M., Pohl, M., & Hanna, A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555623 · Cited by 28 · see citing works
Explanatory Interactive Machine Learning
Pfeuffer, N., Baum, L., Stammer, W., Abdel-Karim, B. M., Schramowski, P., Bucher, A. M., Hügel, C., Rohde, G. G. U., Kersting, K., & Hinz, O. F. (2023) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00806-x · Cited by 30 · see citing works
How Much Automation Does a Data Scientist Want?Preprint
Wang, D., Liao, Q. V., Zhang, Y., Khurana, U., Samulowitz, H., Park, S., Muller, M. J., & Amini, L. D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.03970 · Cited by 45 · see citing works
Hohman, F., Kery, M. B., Ren, D., & Moritz, D. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642109 · Cited by 27 · see citing works
Beyond Checklist Approaches to Ethics in Design
Wong, R. Y., Boyd, K., Metcalf, J., & Shilton, K. (2020) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3406865.3418590 · Cited by 27 · see citing works
Translation as (Re)mediation: How Ethnic Community-Based Organizations Negotiate Legitimacy
Sum, C. M., Tran, A.-T., Lin, J., Kuo, R., Bennett, C. L., Harrington, C., & Fox, S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581280 · Cited by 27 · see citing works
Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer VisionPreprint
Miceli, M., Schuessler, M., & Yang, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.14886 · Cited by 26 · see citing works
The Affective Growth of Computer Vision
Su, N. M., & Crandall, D. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00917 · Cited by 25 · see citing works
Model Sketching: Centering Concepts in Early-Stage Machine Learning Model Design
Lam, M. S., Ma, Z., Li, A., Freitas, I., Wang, D., Landay, J. A., & Bernstein, M. S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581290 · Cited by 25 · see citing works
An Exploratory Study on Visual Exploration of Model Simulations by Multiple Types of Experts
Boukhelifa, N., Bezerianos, A., Trelea, I.-C., Perrot, N., & Lutton, É. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300874 · Cited by 25 · see citing works
Drozdal, J., Weisz, J. D., Wang, D., Dass, G., Yao, B., Zhao, C., Muller, M. J., Lin, J., & Su, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.06509 · Cited by 25 · see citing works
Studying Up Machine Learning Data: Why Talk About Bias When We Mean Power?Preprint
Miceli, M., Posada, J., & Yang, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.08131 · Cited by 25 · see citing works
How Domain Experts Work with Data: Situating Data Science in the Practices and Settings of Craftwork
Jung, J. Y., Steinberger, T., King, J. L., & Ackerman, M. S. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512905 · Cited by 24 · see citing works
Communicating Uncertainty in Digital Humanities Visualization Research
Panagiotidou, G., Lamqaddam, H., Poblome, J., Brosens, K., Verbert, K., & Moere, A. V. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3209436 · Cited by 24 · see citing works
An Empirical Study on the Usage of Automated Machine Learning Tools
Majidi, F., Openja, M., Khomh, F., & Li, H. (2022) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme55016.2022.00014 · Cited by 24 · see citing works
Popova, K., Figueras, C., Höök, K., & Lampinen, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637434 · Cited by 23 · see citing works
Acting with Inherently Uncertain Data: Practices of Data-Centric Knowing
Marius Mikalsen, A., & Monteiro, E. (2021) · Journal of the Association for Information Systems · doi:10.17705/1jais.00722 · Cited by 22 · see citing works
Yoo, D. W., Woo, H., Pendse, S. R., Lu, N. Y., Birnbaum, M. L., Abowd, G. D., & Choudhury, M. D. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637372 · Cited by 22 · see citing works
The New Reality of Reproducibility: The Role of Data Work in Scientific Research
Feinberg, M., Sutherland, W., Nelson, S. B., Jarrahi, M. H., & Rajasekar, A. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392840 · Cited by 20 · see citing works
Smith, J. J., Beattie, L., & Cramer, H. S. M. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583204 · Cited by 22 · see citing works
Suh, A., Appleby, G., Anderson, E. W., Finelli, L. A., Chang, R., & Cashman, D. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3259341 · Cited by 20 · see citing works
Situating Datasets: Making Public Eviction Data Actionable for Housing Justice
Tran, A.-T., Guo, G., Taylor, J., Chan, K. A., Raymond, E. L., & DiSalvo, C. F. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642452 · Cited by 20 · see citing works
Devine, J. K., Schwartz, L. P., & Hursh, S. R. (2022) · Sensors · doi:10.3390/s22072731 · Cited by 20 · see citing works
Rahman, S., & Kandogan, E. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502068 · Cited by 20 · see citing works
Chasing Luck: Data-driven Prediction, Faith, Hunch, and Cultural Norms in Rural Betting Practices
Sultana, S., Mozumder, M. M. H., & Ahmed, S. I. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445047 · Cited by 20 · see citing works
Decoding Human Behavior with Big Data? Critical, Constructive Input from the Decision Sciences
Katsikopoulos, K. V., & Canellas, M. C. (2022) · AI Magazine · doi:10.1002/aaai.12034 · Cited by 21 · see citing works
Workflow analysis of data science code in public GitHub repositories
Ramasamy, D., Sarasua, C., Bacchelli, A., & Bernstein, A. (2022) · Empirical Software Engineering · doi:10.1007/s10664-022-10229-z · Cited by 19 · see citing works
Showkat, D., Smith, A. D. R., Wang, L., & To, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581010 · Cited by 19 · see citing works
Panagiotidou, G., Poblome, J., Aerts, J. A., & Moere, A. V. (2022) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-022-09432-9 · Cited by 19 · see citing works
Balayn, A., Yurrita, M., Rancourt, F., Casati, F., & Gadiraju, U. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713787 · Cited by 22 · see citing works
“It’s Like the Value System in the Loop”: Domain Experts’ Values Expectations for NLP Automation
Showkat, D., & Baumer, E. P. S. (2022) · Designing Interactive Systems Conference · doi:10.1145/3532106.3533483 · Cited by 18 · see citing works
Integration of data science with product design towards data-driven design
Liu, A., Lu, S. C.-Y., Tao, F., & Anwer, N. (2024) · CIRP Annals · doi:10.1016/j.cirp.2024.06.003 · Cited by 20 · see citing works
Figalist, I., Elsner, C., Bosch, J., & Olsson, H. H. (2021) · Journal of Systems and Software · doi:10.1016/j.jss.2021.111135 · Cited by 17 · see citing works
Implicit Error, Uncertainty and Confidence in Visualization: An Archaeological Case Study
Panagiotidou, G., Vandam, R., Poblome, J., & Moere, A. V. (2021) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2021.3088339 · Cited by 16 · see citing works
- Authors
- Passi, S., & Jackson, S. J.
- Affiliations
- Cornell University (US)
- ORCID
- Passi, Samir
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2018-11
- Volume
- 2
- Issue
- CSCW
- Pages
- 1-28
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W4288083705
- Cited by
- 225
- References
- 100
- Citations by year
- 2026: 18 · 2025: 34 · 2024: 34 · 2023: 38 · 2022: 39 · 2021: 32 · 2020: 24 · 2019: 6
- FWCI
- 31.17
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (CHS-1526155)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Sociology and Political Science › Information Systems Theories and Implementation
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- applied data science, trust negotiation, credibility, skepticism, model interpretability, data quality, uncertainty, collaboration, critical data studies, sociology of science, computer-supported cooperative work
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
