Transparency and trust in artificial intelligence systems
The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index.
Abstract
Assistive technology featuring artificial intelligence (AI) to support human decision-making has become ubiquitous. Assistive AI achieves accuracy comparable to or even surpassing that of human experts. However, often the adoption of assistive AI systems is limited by a lack of trust of humans into an AI’s prediction. This is why the AI research community has been focusing on rendering AI decisions more transparent by providing explanations of an AIs decision. To what extent these explanations really help to foster trust into an AI system remains an open question. In this paper, we report the results of a behavioural experiment in which subjects were able to draw on the support of an ML-based decision support tool for text classification. We experimentally varied the information subjects received and show that transparency can actually have a negative impact on trust. We discuss implications for decision makers employing assistive AI technology.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Schmidt et al. (2020) found that highlighting elements of the algorithm ' s confidence in certain predictions decrease the trust of its users in the algorithm.”
From Explaining Why the Computer Says No: Algorithmic Transparency Affects the Perceived Trustworthiness of Automated Decision‐Making (2023) · cites this paper as Schmidt et al. (2020)
In the citing paper
“It should be noted that most of these studies that tested the effect of algorithmic transparency on trust employed samples with limited resemblance to the overall population (e.g. Kizilcec 2016) or are about decisions that have a little direct impact on citizen lives (e.g. Schmidt et al. 2020).”
From Explaining Why the Computer Says No: Algorithmic Transparency Affects the Perceived Trustworthiness of Automated Decision‐Making (2023) · cites this paper as Schmidt et al. 2020
In the citing paper
“This lack of transparency is echoed by other scholars who argue that the inherent lack of transparency in the process of decision making by AI, makes organisational agents hesitant about using AI for important decisions (Schmidt et al., 2020).”
From Unlocking the value of artificial intelligence in human resource management through AI capability framework (2023) · cites this paper as Schmidt et al., 2020
Citations
Cites
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 1×
“These models can achieve impressive predictive performance but their inner workings are usually neither easy to access nor to interpret (Ren et al., 2015).”
BackgroundFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812 · cited 1×
“These models can achieve impressive predictive performance but their inner workings are usually neither easy to access nor to interpret (Ren et al., 2015).”
BackgroundPerceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483 · cited 1×
“While the original model focussed on perceived usefulness and ease of use (Davis, 1989; Venkatesh & Davis, 2000), numerous extensions and variations have considered additional drivers, factors, moderators, and outcomes within the broader area of technology acceptance (Venkatesh et al., 2016).”
MethodA Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · cited 2×
“…space, for instance, one can inspect the top ranking features by highlighting words in text classification (Horn et al., 2017; Ribiero et al., 2016) or mask the lowest ranking pixels in image classification (Alber et al., 2018; Lundberg & Lee, 2017; Montavon et al., 2018; Ribiero et al., 2016).”
BackgroundA Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies
Venkatesh, V., & Davis, F. D. (2000) · Management Science · doi:10.1287/mnsc.46.2.186.11926 · Cited by 23,066 · cited 1×
“While the original model focussed on perceived usefulness and ease of use (Davis, 1989; Venkatesh & Davis, 2000), numerous extensions and variations have considered additional drivers, factors, moderators, and outcomes within the broader area of technology acceptance (Venkatesh et al., 2016).”
MethodVisualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · cited 1×
“The second group of feature scoring methods are explanations that are tailored to specific model classes, for instance, explanations for neural networks (Alber et al., 2018; Montavon et al., 2018; Simonyan et al., 2013; Zeiler & Fergus, 2014) or linear models (Haufe et al., 2014).”
MethodDermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362 · cited 1×
“…intelligence (AI) and machine learning (ML) have become increasingly important for aiding human decision making in various fields, including medical diagnosis, political predictions, policing, predictive maintenance, and many more (Esteva et al., 2017; Nickerson & Rogers, 2014; Susto et al., 2015).”
BackgroundAn integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736 · cited 2×
“…refers to trust as the ‘willingness of a party to be vulnerable to the actions of another party based on the expectation that the other will perform a particular action important to the trustor, irrespective of the ability to monitor or control that other party’ (Mayer et al., 1995, p. 712).”
BackgroundTrust and TAM in Online Shopping: An Integrated Model1
Gefen, D., Karahanna, E., & Straub, D. W. (2003) · MIS Quarterly · doi:10.2307/30036519 · Cited by 8,635 · cited 2×
“Among these, trust has emerged as one key driver of technology acceptance (Gefen et al., 2003).”
BackgroundLearning Word Vectors for Sentiment Analysis
Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., & Potts, C. (2011) · Cited by 6,960
Research works printed but not linked: 20
- Ajzen I., 2014, Health Psychology Review, 7199, 1
- Dietvorst B.J., 2014, Journal of Experimental Psychology, 114
- Gefen D., 2003, Information Systems Research, 25, 1
- Grzymek V., 2019, Bertelsmann Stiftung, 1
- Koene A., 2019, European Parliamentary Research Service, 1
- Lipton Z.C., 2016, Proceedings of the ICML Workshop on Human Interpretability in Machine Learning, 96
- Lundberg S.M., 2017, 31st Conference on Neural Information Processing Systems, 1
- Pacaci G., 2019, ICCS 2019 Proceedings, 334
- Polonski V., 2016, Would you let an algorithm choose the next U.S. president?
- Polonski V., 2018, Humans don’t trust AI predictions - Here’s how to fix it
- Quach K., 2018, Don’t try and beat AI, merge with it says chess champ Garry Kasparov
- Ren S., 2015, NIPS 2015 Proceedings, 1
- Rohde N., 2018, Bertelsmann Stiftung, 1
- Schmidt P., 2019, AAAI 2019 Proceedings, 1
- Siau K., 2018, Cutter Business Technology Journal, 31, 47
- Söllner M., 2012, ICIS 2012 Proceedings, 1
- Szegedy C., 2017, AAAI 2017 Proceedings, 4278
- Yong E., 2018, A popular algorithm is no better at predicting crimes than random people
- Zhang C., 2017, ICLR 2017 Proceedings, 1
- Zhao R., 2019, ECIS 2019 Proceedings, 1
Cited by
Trust in AI and Its Role in the Acceptance of AI Technologies
Choung, H., David, P., & Ross, A. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2050543 · Cited by 963
“We expected that both dimensions of trust would influence technology adoption and use (Schmidt et al., 2020), especially for technologies that are associated with risk and uncertainty.”
BackgroundChowdhury, S., Dey, P., Joel-Edgar, S., Bhattacharya, S., Rodriguez-Espindola, O., Abadie, A., & Truong, L. (2023) · Human Resource Management Review · doi:10.1016/j.hrmr.2022.100899 · Cited by 927
Trust in AI: progress, challenges, and future directions
Afroogh, S., Akbari, A., Malone, E., Kargar, M., & Alambeigi, H. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-04044-8 · Cited by 507
“For example, researchers have shown through a behavioral experiment that giving excessive transparency will confuse the user and negatively impact trust (Schmidt et al., 2020a; 2020b).”
Schepman, A., & Rodway, P. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2085400 · Cited by 503
“A further aspect of trust is linked to explainability (see e.g., EU, 2020; Schmidt et al., 2020; Stanton & Jensen, 2021).”
BackgroundLambert, S. I., Madi, M., Sopka, S., Lenes, A., Stange, H., Buszello, C.-P., & Stephan, A. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00852-5 · Cited by 404
Metadata
- Authors
- Schmidt, P., Biessmann, F., & Teubner, T.
- Affiliations
- Amazon (Germany) (DE)
- Berliner Hochschule für Technik (DE)
- Einstein Center Digital Future (DE)
- Corresponding author
- Biessmann, Felix
- Venue
- Journal of Decision Systems
- Publisher
- Informa UK Limited
- Published
- 2020-09-10
- Volume
- 29
- Issue
- 4
- Pages
- 260-278
- ISSN
- 1246-0125 (ISSN-L)
- 2116-7052
- OpenAlex
- W3085537935
- Cited by
- 440
- References
- 51
- Citations by year
- 2026: 98 · 2025: 109 · 2024: 81 · 2023: 86 · 2022: 48 · 2021: 17 · 2020: 1
- FWCI
- 14.94
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Decision System (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- trust in AI, explainable AI, assistive AI, decision support, text classification, human-AI decision-making, behavioural experiments
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Transparency and trust in artificial intelligence systems. https://app.cochat.ai/papers/10.1080/12460125.2020.1819094
- The paper
- Schmidt, P., Biessmann, F., & Teubner, T. (2020). Transparency and trust in artificial intelligence systems. Journal of Decision Systems. https://doi.org/10.1080/12460125.2020.1819094
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.