Putting Tools in Their Place: The Role of Time and Perspective in Human-AI Collaboration for Qualitative Analysis
Open accessThe DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
Large datasets or 'big data' corpora are typically the domain of quantitative scholars, who work with computational tools to derive numerical and descriptive insights. However, recent work asks how computational tools and other technologies, such as AI, can support qualitative scholars in developing deep and complex insights from large amounts of data. Addressing this question, Jiang et al. found that qualitative scholars are generally opposed to incorporating AI in their practices of data analysis. In this paper, we provide nuance to these earlier findings, showing that the stage of qualitative analysis matters for how scholars believe AI can and should be used. Through interviews with 15 CSCW and HCI qualitative researchers, we explore how AI can be included throughout different stages of qualitative analysis. We find that qualitative scholars are amenable to working with AI in diverse ways, such as for data exploration and coding, as long as it assists rather than automates their analytic work practice. Based on our analysis, we discuss how incorporating AI into qualitative research can shift some analytic practices, and how designing for human-AI collaboration in qualitative analysis necessitates considering tradeoffs in scale, abstraction, and task delegation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Sampling in Interview-Based Qualitative Research: A Theoretical and Practical Guide
Robinson, O. C. (2014) · Qualitative Research in Psychology · doi:10.1080/14780887.2013.801543 · Cited by 3,232
Artificial intelligence and the future of work: Human-AI symbiosis in organizational decision making
Jarrahi, M. H. (2018) · Business Horizons · doi:10.1016/j.bushor.2018.03.007 · Cited by 2,772
Licklider, J. C. R. (1960) · IRE Transactions on Human Factors in Electronics · doi:10.1109/thfe2.1960.4503259 · Cited by 1,938
Cai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359206 · Cited by 594
Human-AI Collaboration in Data Science: Exploring Data Scientists' Perceptions of Automated AI
Wang, D., Weisz, J. D., Muller, M., Ram, P., Geyer, W., Dugan, C., Tausczik, Y., Samulowitz, H., & Gray, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359313 · Cited by 342 · cited 3×
“[48] argue that algorithmic systems should augment rather than automate human qualitative analytic practices.”
BackgroundYang, Q., Steinfeld, A., & Zimmerman, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300468 · Cited by 247
Qualitative HCI Research: Going Behind the Scenes
Blandford, A., Furniss, D., & Makri, S. (2016) · Synthesis Lectures on Human-Centered Informatics · doi:10.1007/978-3-031-02217-3 · Cited by 202
Reddit Rules! Characterizing an Ecosystem of Governance
Fiesler, C., Jiang, J., McCann, J., Frye, K., & Brubaker, J. (2018) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v12i1.15033 · Cited by 165
Self-citationChen, N.-C., Drouhard, M., Kocielnik, R., Suh, J., & Aragon, C. R. (2018) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3185515 · Cited by 152
The Role of Conversational Grounding in Supporting Symbiosis Between People and Digital Assistants
Cho, J., & Rader, E. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392838 · Cited by 49
Jiang, J. A., Wade, K., Fiesler, C., & Brubaker, J. R. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449168 · Cited by 1
Self-citation
Cited by
What is Human-Centered about Human-Centered AI? A Map of the Research Landscape
Capel, T., & Brereton, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580959 · Cited by 228
“Research also investigated how humans contribute to data labelling [202, 208, 217], and how AI and ML could be incorporated into the research and design process [60, 91, 100, 109, 155].”
BackgroundAI and its implications for research in higher education: a critical dialogue
Butson, R., & Spronken-Smith, R. (2024) · Higher Education Research & Development · doi:10.1080/07294360.2023.2280200 · Cited by 130
PaTAT: Human-AI Collaborative Qualitative Coding with Explainable Interactive Rule Synthesis
Gebreegziabher, S. A., Zhang, Z., Tang, X., Meng, Y., Glassman, E. L., & Li, T. J.-J. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581352 · Cited by 72
Balancing AI and authenticity: EFL students' experiences with ChatGPT in academic writing
Werdiningsih, I., Marzuki, & Rusdin, D. (2024) · Cogent Arts & Humanities · doi:10.1080/23311983.2024.2392388 · Cited by 65
Glinka, K., & Müller-Birn, C. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610054 · Cited by 26
“, [3, 18, 37, 51]), showing us that there is a reasonable overlap despite the differences in contexts.”
Background
Metadata
- Authors
- Feuston, J. L., & Brubaker, J. R.
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2021-10-13
- Volume
- 5
- Issue
- CSCW2
- Pages
- 1-25
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W3207098447
- Cited by
- 86
- References
- 12
- FWCI
- 19.51
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.