CoChat

Effect of Explanation Conceptualisations on Trust in AI-assisted Credibility Assessment

Open access
Pareek, Saumya, van Berkel, Niels, Velloso, Eduardo, Goncalves, Jorge · 2024 · Proceedings of the ACM on Human-Computer Interaction

The DOI matches its Crossref record; 103 of the 103 references it cites were found in a registry or an index.

Abstract

As misinformation increasingly proliferates on social media platforms, it has become crucial to explore how to best convey automated news credibility assessments to end-users, and foster trust in fact-checking AIs. In this paper, we investigate how model-agnostic, natural language explanations influence trust and reliance on a fact-checking AI. We construct explanations from four Conceptualisation Validations (CVs) - namely consensual, expert, internal (logical), and empirical - which are foundational units of evidence that humans utilise to validate and accept new information. Our results show that providing explanations significantly enhances trust in AI, even in a fact-checking context where influencing pre-existing beliefs is often challenging, with different CVs causing varying degrees of reliance. We find consensual explanations to be the least influential, with expert, internal, and empirical explanations exerting twice as much influence. However, we also find that users could not discern whether the AI directed them towards the truth, highlighting the dual nature of explanations to both guide and potentially mislead. Further, we uncover the presence of automation bias and aversion during collaborative fact-checking, indicating how users' previously established trust in AI can moderate their reliance on AI judgements. We also observe the manifestation of a 'boomerang'/backfire effect often seen in traditional corrections to misinformation, with individuals who perceive AI as biased or untrustworthy doubling down and reinforcing their existing (in)correct beliefs when challenged by the AI. We conclude by presenting nuanced insights into the dynamics of user behaviour during AI-based fact-checking, offering important lessons for social media platforms.

Overview

Integrity signals

Registry

Crossref · 10.1145/3686922

Cites

103 of 103 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 103 references

Research works printed but not linked: 13

  1. 10.1145/3449287 · doi:10.1145/3449287 · Paper page
  2. Santanu Chakrabarti, Lucile Stengel, and Sapna Solanki. 2018. Duty, identity, credibility: Fake news and the ordinary citizen in India. BBC World Service Audiences Research (2018).
  3. 10.1007/s10869-011--9231--8 · doi:10.1007/s10869-011--9231--8
  4. 10.1111/j.1083--6101.2007.00388.x · doi:10.1111/j.1083--6101.2007.00388.x
  5. N.A. Karlova and K.E. Fisher. 2013. A social diffusion model of misinformation and disinformation for understanding human information behaviour. Information Research, Vol. 18 (Jan. 2013).
  6. 10.1111/j.1460--2466.2010.01488.x · doi:10.1111/j.1460--2466.2010.01488.x
  7. 10.1186/s41235-020-00251--4 · doi:10.1186/s41235-020-00251--4
  8. Harikumar Pallathadka, Malik Jawarneh, Domenic Sanchez, Guna Sajja, Sanjeev Gour, and Mohd Naved. 2021. The Impact of Machine Learning on Management, Healthcare, and Agriculture. (July 2021).
  9. 10.1111/j.1559--1816.2004.tb02547.x · doi:10.1111/j.1559--1816.2004.tb02547.x
  10. 10.37016/mr-2020--81 · doi:10.37016/mr-2020--81
  11. Ron Sellers. 2013. How Sliders Bias Survey Data. MRA's Alert, Vol. 53, 3 (2013), 56--57. https://greymatterresearch.com/wp-content/uploads/2019/09/Alert-Sliders-2013.pdf
  12. 10.1111/j.1540--5907.2006.00214.x · doi:10.1111/j.1540--5907.2006.00214.x
  13. 10.1007/978--3-030--85610--6_26 · doi:10.1007/978--3-030--85610--6_26

Cited by

See all 43

Metadata

Authors
Pareek, S., van Berkel, N., Velloso, E., & Goncalves, J.
Affiliations
  • The University of Melbourne (AU)
  • Aalborg University, Aalborg, Denmark
  • The University of Melbourne, Melbourne, Victoria, Australia
Venue
Proceedings of the ACM on Human-Computer Interaction
Published
2024-11-07
Volume
8
Issue
CSCW2
Pages
1-31
ISSN
2573-0142 (ISSN-L)
OpenAlex
W4404172855
Cited by
42
References
100
Citations by year
2026: 21 · 2025: 20 · 2024: 1
FWCI
30.03
Type
journal article
Language
en
Open access
Locations
  • Proceedings of the ACM on Human-Computer Interaction (publishedVersion, cc-by)
  • VBN Forskningsportal (Aalborg Universitet) (publishedVersion, cc-by)
Topics
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
Keywords
trust in AI, fact-checking, natural language explanations, misinformation, social media, automation bias, backfire effect

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.