CoChat

Critical-Reflective Human-AI Collaboration: Exploring Computational Tools for Art Historical Image Retrieval

Open access
Glinka, Katrin, Müller-Birn, Claudia · 2023 · Proceedings of the ACM on Human-Computer Interaction

The DOI matches its Crossref record; 73 of the 74 references it cites were found in a registry or an index, and 1 are not checked yet.

Abstract

Just as other disciplines, the humanities explore how computational research approaches and tools can meaningfully contribute to scholarly knowledge production. Building on related work from the areas of CSCW and HCI, we approach the design of computational tools through the analytical lens of 'human-AI collaboration.' Such work investigates how human competencies and computational capabilities can be effectively and meaningfully combined. However, there is no generalizable concept of what constitutes 'meaningful' human-AI collaboration. In terms of genuinely human competencies, we consider criticality and reflection as guiding principles of scholarly knowledge production and as deeply embedded in the methodologies and practices of the humanities. Although (designing for) reflection is a recurring topic in CSCW and HCI discourses, it has not been centered in work on human-AI collaboration. We posit that integrating both concepts is a viable approach to supporting 'meaningful' human-AI collaboration in the humanities and other qualitative, interpretivist, and hermeneutic research areas. Our research, thus, is guided by the question of how critical reflection can be enabled in human-AI collaboration. We address this question with a use case that centers on computer vision (CV) tools for art historical image retrieval. Specifically, we conducted a qualitative interview study with art historians to explore a) what potentials and affordances art historians ascribe to human-AI collaboration and CV in particular, and b) in what ways art historians conceptualize critical reflection in the context of human-AI collaboration. We extended the interviews with a think-aloud software exploration. We observed and recorded participants' interaction with a ready-to-use CV tool in a possible research scenario. We found that critical reflection, indeed, constitutes a core prerequisite for 'meaningful' human-AI collaboration in humanities research contexts. However, we observed that critical reflection was not fully realized during interaction with the CV tool. We interpret this divergence as supporting our hypothesis that computational tools need to be intentionally designed in such a way that they actively scaffold and support critical reflection during interaction. Based on our findings, we suggest four empirically grounded design implications for 'critical-reflective human-AI collaboration': supporting reflection on the basis of transparency, foregrounding epistemic presumptions, emphasizing the situatedness of data, and strengthening interpretability through contextualized explanations.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3610054

Cites

73 of 74 references verified0 citation errors · 0 not checkable

Statistics

Figures

Image 1
CC-BY-SA · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 74 references

Research works printed but not linked: 15

  1. Philip E. Agre. 1997. Toward a critical technical practice: Lessons learned in trying to reform AI. In Social Science, Technical Systems and Cooperative Work: Beyond the Great Divide (Computers, Cognition, and Work). Lawrence Erlbaum Associates, Mahwah, N.J, 131 -- 158.
  2. Caroline Bassett, David Michael Berry, M. Beatrice Fazi, Jack Pay, and Ben Roberts. 2017. Critical Digital Humanities and Machine-Learning. In 12th Annual International Conference of the Alliance of Digital Humanities Organizations, DH 2017, August 8--11, 2017, Conference Abstracts. Alliance of Digital Humanities Organizations, Montréal, Canada, 36 -- 40. https://dh2017.adho.org/abstracts/509/509.pdf
  3. 10.1145/3449287 · doi:10.1145/3449287 · Paper page
  4. 10.1111/j.1755--5825.1978.tb02059.x · doi:10.1111/j.1755--5825.1978.tb02059.x
  5. 10.1007/978--3-030--85616--8_36 · doi:10.1007/978--3-030--85616--8_36
  6. Nan Z Da. 2019. The computational case against computational literary studies. Critical inquiry 45, 3 (2019), 601--639.
  7. John Dewey. 1997. How we think. Dover Publications, Mineola.
  8. 10.1609/aimag.v24i3.1720 · doi:10.1609/aimag.v24i3.1720
  9. Catherine D'Ignazio and Lauren F. Klein. 2020. Data Feminism. MIT Press, Cambridge, MA, USA.
  10. Vilém Flusser. 2005. Thought and reflection. Flusser Studies 1, 3 (2005), 16 -- 35.
  11. Matthew G Kirschenbaum. 2016. What is digital humanities and what's it doing in English departments? In Defining Digital Humanities. Routledge, New York, NY, 211--220.
  12. Clayton Lewis. 1982. Using the" thinking-aloud" method in cognitive interface design. IBM TJ Watson Research Center, Yorktown Heights, NY.
  13. Fabian Offert and Peter Bell. 2023. imgs.ai. A Deep Visual Search Engine for Digital Art History. International Journal for Digital Art History. forthcoming.
  14. 10.1016/0950--7051(95)98369-H · doi:10.1016/0950--7051(95)98369-h
  15. Nikolai Ufer, Max Simon, Sabine Lang, and Björn Ommer. 2021. Large-scale interactive retrieval in art collections using multi-style feature aggregation. PLoS ONE 16, 11 (2021), 38 pages.

Cited by

Metadata

Authors
Glinka, K., & Müller-Birn, C.
Affiliations
Freie Universität Berlin (DE)
Venue
Proceedings of the ACM on Human-Computer Interaction
Published
2023-09-28
Volume
7
Issue
CSCW2
Pages
1-33
ISSN
2573-0142 (ISSN-L)
OpenAlex
W4387329456
Cited by
26
References
67
Citations by year
2026: 9 · 2025: 15 · 2024: 1 · 2023: 1
FWCI
71.17
Type
journal article
Language
en
Open access
Locations
  • Proceedings of the ACM on Human-Computer Interaction (publishedVersion, cc-by-sa)
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
  • arXiv (Cornell University)
Funders
Freie Universität Berlin
Topics
  • Social Sciences › Arts and Humanities › Literature and Literary Theory › Digital Humanities and Scholarship
  • Social Sciences › Arts and Humanities › Visual Arts and Performing Arts › Art, Technology, and Culture
  • Social Sciences › Business, Management and Accounting › Management Information Systems › Information Systems Theories and Implementation
Keywords
human-AI collaboration, critical reflection, art historians, transparency, interpretability, think-aloud

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.