Psychological Factors Influencing Appropriate Reliance on AI-enabled Clinical Decision Support Systems: Experimental Web-Based Study Among Dermatologists
Open accessPublished version of a preprint
The DOI matches its Crossref record; 75 of the 78 references it cites were found in a registry or an index; 16 of the 84 sources its claims cite were checked for support.
Abstract
Background: Artificial intelligence (AI)–enabled decision support systems are critical tools in medical practice; however, their reliability is not absolute, necessitating human oversight for final decision-making. Human reliance on such systems can vary, influenced by factors such as individual psychological factors and physician experience. Objective: This study aimed to explore the psychological factors influencing subjective trust and reliance on medical AI’s advice, specifically examining relative AI reliance and relative self-reliance to assess the appropriateness of reliance. Methods: A survey was conducted with 223 dermatologists, which included lesion image classification tasks and validated questionnaires assessing subjective trust, propensity to trust technology, affinity for technology interaction, control beliefs, need for cognition, as well as queries on medical experience and decision confidence. Results: A 2-tailed t test revealed that participants’ accuracy improved significantly with AI support (t222=−3.3; P<.001; Cohen d=4.5), but only by an average of 1% (1/100). Reliance on AI was stronger for correct advice than for incorrect advice (t222=4.2; P<.001; Cohen d=0.1). Notably, participants demonstrated a mean relative AI reliance of 10.04% (139/1384) and a relative self-reliance of 85.6% (487/569), indicating a high level of self-reliance but a low level of AI reliance. Propensity to trust technology influenced AI reliance, mediated by trust (indirect effect=0.024, 95% CI 0.008-0.042; P<.001), and medical experience negatively predicted AI reliance (indirect effect=–0.001, 95% CI –0.002 to −0.001; P<.001). Conclusions: The findings highlight the need to design AI support systems in a way that assists less experienced users with a high propensity to trust technology to identify potential AI errors, while encouraging experienced physicians to actively engage with system recommendations and potentially reassess initial decisions.
Overview
CoChat Summary
## TL;DR
This preregistered web-based experiment with 223 dermatologists found that participants relied more on correct than incorrect AI advice, but overall showed low AI reliance and high self-reliance. Trust in the system and medical experience were the main predictors of reliance: higher trust increased reliance, while greater medical experience reduced trust and AI reliance.
## Key findings
- Among 223 dermatologists, first-response accuracy was 63% (15/24) and final accuracy with AI support was 65% (15/24), an average increase of 1% (SD 4.5%).
- Reliance was much more self-directed than AI-directed: RAIR was 10.04% (139/1384) and RSR was 85.6% (487/569).
- Participants relied less strongly on incorrect AI advice than on correct advice; the difference in WoA was significant (t\_222 = 4.2; P < .001).
- Trust predicted reliance (β = 0.05; P < .001), explaining 5.8% of the variance in reliance (R² = 0.1); trust was positively related to RAIR and negatively related to RSR.
- Propensity to trust technology had a significant indirect effect on reliance through trust (indirect effect = 0.024; 95% CI 0.008-0.042), while medical experience showed a significant negative indirect effect on reliance through trust (indirect effect = −0.001; 95% CI −0.002 to −0.001).
- Affinity for technology interaction, control beliefs in dealing with technology, and need for cognition were not significant predictors of trust or reliance.
## Methods
- Web-based preregistered repeated-measures experiment conducted from June to September 2023 with 223 professional dermatologists (out of 256 responses) in Germany; mean experience was 15.5 years (SD 11.2; range 0-50).
- Participants classified 24 skin lesion image cases as benign or malignant in a 2-step task: an initial unaided judgment, then a revised judgment after seeing AI advice said to be 80% accurate; 5 of 24 AI recommendations were deliberately incorrect and randomized.
- Main analyses included a t test for reliance differences between correct vs incorrect AI advice, regression and moderation analyses for trust/reliance, and PROCESS-based mediation analyses for psychological factors and medical experience; the planned structural equation model was dropped because model fit was unacceptable.
## Limitations
- The paper states that the AI only provided the final classification, so participants could not engage with explanations or other background information about the model’s validity.
- Confidence was measured with a proxy based on slider position and dichotomized, which the paper notes may be less sensitive than directly asking participants about confidence.
- AI-misclassified cases were randomly selected, which the paper says may have introduced uncontrolled variation in case difficulty.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



Verification
Claims the paper makes
9 Valid · 0 Not valid · 40 Not checked · 78 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Extensive research has shown that the accuracy of CNNs is comparable to that of dermatologists [ 32 - 34 ].”
What the sources say
- Supportedcites [32] source
The review's section 'The application of convolutional deep learning in dermatology' summarises multiple head-to-head studies (Esteva et al., Brinker et al., Tschandl et al., Haenssle et al.), stating that 'the studies all demonstrate that, on average, dermatologists sit below the ROC curve of the machine learning algorithm' (Figure 4 legend) and that in Tschandl's challenge the algorithms averaged 19/30 versus 17/30 for dermatologists — i.e. at least comparable, often superior, accuracy. Its reference list also titles Brinker's clinical-image study as a CNN that 'performed on par with 145 dermatologists'. The paper further notes that these assessments 'may be biased against clinicians' and that 'all of the published studies indicate superiority of machine learning algorithms over dermatologists' warrants scrutiny of study design (Key biases section), which supports the conservative 'comparable' framing rather than contradicting it. The scope in our copy is restricted to skin lesion/melanoma image classification (and onychomycosis), not dermatology as a whole. (LLM, conf=0.85)
- Partly supportedcites [33] sourceExperimental — accuracy not yet measured
Our copy does contain a large evidence base on this point: the review pooled 81 non-randomised studies comparing CNN/deep learning algorithms with expert clinicians, several of them in dermatology (Esteva 2017, Fujisawa 2018, Haenssle 2018, Han 2018, Marchetti 2017, Yu 2018 — Table 3, Results, Non-randomised studies), and it records that in 77 of 81 studies the abstract made a claim about AI-versus-clinician performance, with AI described as comparable in 25 (32%), comparable or better in 13 (17%) and superior in 23 (30%). However, the source does not endorse equivalence: its Conclusions state that 'many arguably exaggerated claims exist about equivalence with or superiority over clinicians', 72% (58/81) of studies were high risk of bias by PROBAST, only nine were prospective, and the Discussion (Principal findings, secondly) says comparisons against human clinicians are 'difficult to evaluate given the artificial in silico context'. So the citing sentence's assertion that extensive research 'has shown' CNN accuracy is comparable to dermatologists states as established fact what this review reports only as authors' claims it explicitly cautions against, and it narrows a cross-speciality finding to dermatology, where our copy tabulates study characteristics but reports no comparative accuracy result. (LLM, conf=0.72)
- Supportedcites [34] source
The Introduction (sec. 1) of our copy states that 'Studies have demonstrated that AI/ML algorithms can perform as well as, or better than, consultant dermatologists in diagnosing skin cancers [17]', and elsewhere that 'AI achieved higher diagnostic accuracy than dermatologists, outperforming them in 7 out of 11 cases [8]', against dermatologist visual-examination accuracy of about 60% (89% with dermoscopy). The review's evidence base is deep-learning/CNN work (Decision Support section reports CNN/hybrid models at 96% and 99.179% accuracy on ISIC datasets), and the Conclusion repeats that 'AI algorithms have also outperformed dermatologists in various studies'. The cited review therefore supports the paraphrase that CNN/AI accuracy is comparable to (indeed at least matching) dermatologists; the only soft point is that the source frames the head-to-head comparison as 'AI/ML algorithms' rather than naming CNNs in that sentence, and describes 'several studies' rather than 'extensive research'. (LLM, conf=0.80)
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“As clinicians gain experience, they develop a deeper understanding of clinical contexts and decision-making processes, which may influence both trust in and reliance on CDSSs [ 8 , 15 - 17 ].”
What the sources say
- Partly supportedcites [8] sourceExperimental — accuracy not yet measured
In Section 5.2 Results, the paper does tie decision-maker experience to how AI/XAI support is taken up: 'less experienced decision-makers may generally have a larger benefit from the introduction of this type of support, while more experienced decision-makers may also be negatively impacted by such systems, possibly due to lower acquaintance with their interpretation' — offered to explain why the student/resident-heavy ECG study gained more than the specialist-only MRI study. What our copy does not state is the mechanism the claim attributes to it (that growing experience yields a deeper understanding of clinical contexts and decision-making processes), nor anything about trust in and reliance on CDSSs as such; the discussion is about reliance patterns and technology-dominance metrics under AI advice and explanations, and the experience point itself is flagged as 'highlighted by recent results in the literature' [14,74] rather than as a finding of this paper. So only the general 'experience modulates uptake of decision support' half of the claim is carried by this source. (LLM, conf=0.72)
- Not checked yetcites [15] source
- Partly supportedcites [16] sourceExperimental — accuracy not yet measured
Our copy supports only the second half of the attributed idea: the paper's own results show that physicians' level of task expertise shapes their trust in and use of AI-CDSS advice (Results, 'Non-Task Experts Preferred Explainable Advice': annotated advice raised advice-quality ratings — a scale including trustworthiness — only among non-task experts; Discussion: only non-task experts benefited from annotations). But the paper never says that clinicians develop a deeper understanding of clinical contexts and decision-making as they accumulate experience: expertise is operationalised by discipline/formal training in image review (radiologists vs IM/EM physicians, Participants section), with years of professional experience reported only as demographics (Table 4), and the expertise-vs-reliance link in the Introduction is credited to other work (refs 7, 19). The 'deeper understanding of clinical contexts with experience' premise is therefore not attributable to this source. (LLM, conf=0.68)
- Partly supportedcites [17] sourceExperimental — accuracy not yet measured
Our copy supports only the general link between user expertise and trust/reliance on algorithmic decision aid: the Abstract (p. 1) lists 'a user's domain expertise, mental models of an AI system, trust in recommendations' as factors for Human-AI team success, and §6.1 Baseline Expertise (p. 22) shows benefits from the AI-Assistant varied with users' baseline expertise, with low-expertise users lacking 'the knowledge or understanding of when they should accept or reject an AI recommendation'. In the clinical domain the paper only relays someone else's result — §3.1.3 (p. 5) reports Gaube et al.'s finding that radiologists, compared with less task-expert physicians, rated AI-labelled diagnostic advice as lower quality. What the source does not support is the specific claim attributed to it: that clinicians' accumulating experience gives them a deeper understanding of clinical contexts and decision-making, and that this shapes trust in and reliance on clinical decision support systems — the study itself is a blood-vessel (Stall Catchers) labeling task with a simulated AI-Assistant, not clinicians using a CDSS, and no passage describes expertise development in clinical contexts. (LLM, conf=0.70)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In addition to these psychological factors and medical experience, confidence in the initial decision is a key factor influencing decision-making processes in high-stakes environments, such as medical diagnosis [ 23 - 25 ].”
What the sources say
- Not checked yetcites [23] source
- Partly supportedcites [24] sourceExperimental — accuracy not yet measured
The Results (Influence of Participants’ Decisional Certainty on Trust) report that greater certainty in an initial assessment was associated with less willingness to follow a decision aid’s advice. But the study concerned laypersons deciding the urgency of care for a single vignette, not medical diagnosis; the Limitations say that deviation from the vignette’s gold standard was not consequential and that the findings do not establish effects for other urgency decisions. The claim extends the finding to high-stakes decision-making beyond what this study tested. (LLM, conf=0.86)
- Not checked yetcites [25] source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“When humans are confident in their own decisions, they are less likely to change it after receiving advice [ 23 , 24 ].”
What the sources say
- Not checked yetcites [23] source
- Partly supportedcites [24] sourceExperimental — accuracy not yet measured
The Results section ('Influence of Participants’ Decisional Certainty on Trust') reports that participants were less likely to follow contrary advice as certainty in their initial assessment increased. The finding concerns laypersons using a medical symptom checker, rather than humans receiving advice generally; even among participants with maximum certainty, more than half followed the advice. (LLM, conf=0.82)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Our research identified medical experience as a significant factor influencing self-reported trust and reliance behavior, which is consistent with previous research [ 58 - 60 ].”
What the sources say
- Not checked yetcites [58] source
- Not checked yetcites [59] source
- Partly supportedcites [60] sourceExperimental — accuracy not yet measured
The INTRODUCTION lists 'nurse clinical experience' among factors 'believed to influence nurse acceptance or rejection of CDS suggestions,' but hedges it as background and credits it to other works (refs 7 Campion et al., 11 Dowding et al., 12 Cho et al.), so the paper does not establish experience as a statistically significant factor in its own data. The METHODS (Analysis) and RESULTS show the only inferential test was t-tests of the trust/disagreement survey items 7–10 against rejection of four specific suggestions (p=.04, DISCUSSION and RESULTS), with years of nursing experience reported descriptively only (mean 5.0, SD 10.3, RESULTS). Thus the trust-to-reliance link is supported, but the claim's specific assertion that medical/clinical experience significantly influences self-reported trust and reliance is not this paper's finding, and its population is 21 nurses in a simulated CAVE2 environment rather than clinicians' medical experience. (LLM, conf=0.75)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
28 more claims
Create a free account to see all 127 claims and to verify the 68 references still open.
- Probably not a substantial claim. 59
- It may state two separate claims as one. 49
- The quote only partly states this claim. 3
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“explaining 24.1% (54/224) of the variance”
“R 2 =0.2; F 1,222 =13.9; P <.001”
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“explaining 21% (47/224) of the variance”
“R 2 =0.2; F 1,222 =10.2; P =.002”
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“Participants failing both checks were excluded from further analysis”
“Participants who failed either of these checks were excluded from the analysis”
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“P <.001; Cohen d =4.5) but small difference in accuracy between the first unaided response, which had 63% (15/24; SD 8.7%) accuracy, and the final response, which had 65% (15/24; SD 9%) accuracy. On average, the accuracy increased by 1% (SD 4.5%)”
Reported 4.5 · recomputed 0.222222
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“predicting 25% (56/224) of the variance”
“F 3,219 =6.3; P <.001”
3 more findings
Create a free account to see every finding of this check.
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
(2013) · Cited by 44,563
Venkatesh, V., Thong, J. Y. L., & Xu, X. (2012) · MIS Quarterly · doi:10.2307/41410412 · Cited by 15,756 · cited 1×
“Research ranges from system acceptance and adoption [39,40] to specific investigations of trust in systems [41-44] to the calibration of trust and the appropriateness of reliance [8,9,45-49].”
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605
Cacioppo, J. T., & Petty, R. E. (1982) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.42.1.116 · Cited by 5,359
Garg, A. X., Adhikari, N. K. J., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., Sam, J., & Haynes, R. B. (2005) · JAMA · doi:10.1001/jama.293.10.1223 · Cited by 2,936 · cited 1×
“CDSSs play a central role in managing large amounts of data [29] and ensuring the retrieval of relevant information.”
Explainability for artificial intelligence in healthcare: a multidisciplinary perspective
the Precise4Q consortium, Amann, J., Blasimme, A., Vayena, E., Frey, D., & Madai, V. I. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01332-6 · Cited by 2,203 · cited 1×
“Meanwhile, clinicians contribute their nuanced understanding of patient history, context, and ethical considerations to ensure that decisions align with individual patient needs [36,37].”
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061 · cited 1×
“As clinicians gain experience, they develop a deeper understanding of clinical contexts and decision-making processes, which may influence both trust in and reliance on CDSSs [8,15-17].”
The role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276 · cited 1×
“The propensity to trust technology complements these by reflecting a baseline level of trust that users have in new systems, making it a foundational factor for reported trust and behavioral reliance on system advice [13,14].”
Nagendran, M., Chen, Y., Lovejoy, C. A., Gordon, A. C., Komorowski, M., Harvey, H., Topol, E. J., Ioannidis, J. P. A., Collins, G. S., & Maruthappu, M. (2020) · BMJ · doi:10.1136/bmj.m689 · Cited by 1,152
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124
Human-computer collaboration for skin cancer recognition
Tschandl, P., Rinner, C., Apalla, Z., Argenziano, G., Codella, N., Halpern, A., Janda, M., Lallas, A., Longo, C., Malvehy, J., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0942-0 · Cited by 1,024 · cited 9×
“Our findings extend prior research [10,11,23] by addressing the appropriateness of reliance on these systems.”
InfluentialArtificial intelligence, bias and clinical safety
Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2018-008370 · Cited by 956
Clinical decision support in the era of artificial intelligence
Shortliffe, E. H., & Sepúlveda, M. J. (2018) · JAMA · doi:10.1001/jama.2018.17163 · Cited by 920 · cited 1×
“Meanwhile, clinicians contribute their nuanced understanding of patient history, context, and ethical considerations to ensure that decisions align with individual patient needs [36,37].”
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895 · cited 2×
“Thus, it is important to correctly calibrate trust and thereby adjust expectations about the trustworthiness of systems [26,50,51].”
Franke, T., Attig, C., & Wessel, D. (2019) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2018.1456150 · Cited by 828 · cited 2×
“Given the favorable disposition toward technology inherent in many of these factors, we intend to investigate affinity for technology interaction, defined as the inclination to actively engage in extensive technological interaction, as a fundamental personal asset for adapting to technology [61].”
Artificial Intelligence and Human Trust in Healthcare: Focus on Clinicians
Asan, O., Bayrak, A. E., & Choudhury, A. (2020) · Journal of Medical Internet Research · doi:10.2196/15154 · Cited by 825 · cited 6×
“The importance of technological experience has been highlighted in several studies as a key element for the acceptance and reliance on systems [10,18].”
InfluentialDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757 · cited 2×
“Previous research showed that high trust leads to blindly following the advice received [1].”
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754
Receiving other people’s advice: Influence and benefit
Yaniv, I. (2004) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2003.08.002 · Cited by 652 · cited 1×
“The WoA is a commonly used metric to quantify the incorporation of advice [67] and is defined as follows: WoA = (response 2 – response 1 )/(advice – response 1 ) (1) WoA was cut off at −1 and 1 for the purpose of this study, assuming a maximum magnitude of 100%.”
Antoniadi, A. M., Du, Y., Guendouz, Y., Wei, L., Mazo, C., Becker, B. A., & Mooney, C. (2021) · Applied Sciences · doi:10.3390/app11115088 · Cited by 629 · cited 1×
“However, the integration of machine learning and AI, such as CNNs, has significantly enhanced their capabilities [30].”
Trustworthy artificial intelligence
Thiebes, S., Lins, S., & Sunyaev, A. (2021) · Electronic Markets · doi:10.1007/s12525-020-00441-4 · Cited by 595 · cited 1×
“Research ranges from system acceptance and adoption [39,40] to specific investigations of trust in systems [41-44] to the calibration of trust and the appropriateness of reliance [8,9,45-49].”
Merritt, S. M., & Ilgen, D. R. (2008) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872008x288574 · Cited by 527 · cited 1×
“The propensity to trust is defined as the tendency to trust others [55] and has previously been positively associated with self-reported trust [56].”
Trust, Confidence, and Expertise in a Judge-Advisor System
Sniezek, J. A., & Van Swol, L. M. (2001) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.2000.2926 · Cited by 469 · cited 1×
“Sniezek and Van Swol [52] define confidence as the belief “that a specific statement, opinion, or decision is the best possible.””
The Role of Explanations on Trust and Reliance in Clinical Decision Support Systems
Bussone, A., Stumpf, S., & O'Sullivan, D. (2015) · 2015 International Conference on Healthcare Informatics · doi:10.1109/ichi.2015.26 · Cited by 441 · cited 1×
“Issues such as amplification of biases present in the training data, failure to generalize effectively beyond specific populations, or errors in classification can affect reliability [2-7].”
Evaluating XAI: A comparison of rule-based and example-based explanations
van der Waa, J., Nieuwburg, E., Cremers, A., & Neerincx, M. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2020.103404 · Cited by 393
The Very Efficient Assessment of Need for Cognition: Developing a Six-Item Version
Lins de Holanda Coelho, G., H. P. Hanel, P., & J. Wolf, L. (2020) · Assessment · doi:10.1177/1073191118793208 · Cited by 358 · cited 2×
“The need for cognition is a personality trait that reflects an inclination to engage in mentally taxing tasks and to derive pleasure from doing so [63,64].”
What is AI? Applications of artificial intelligence to dermatology
Du‐Harpur, X., Watt, F. M., Luscombe, N. M., & Lynch, M. D. (2020) · British Journal of Dermatology · doi:10.1111/bjd.18880 · Cited by 355 · cited 1×
“Extensive research has shown that the accuracy of CNNs is comparable to that of dermatologists [32-34].”
Fan, W., Liu, J., Zhu, S., & Pardalos, P. M. (2020) · Annals of Operations Research · doi:10.1007/s10479-018-2818-y · Cited by 346 · cited 1×
“However, the operationalization varies widely, encompassing perception toward automation [18], experience with computers [60], innovativeness in IT [54], past IT experience, and ability to control a new technology [59].”
Trusting Automation: Designing for Responsivity and Resilience
Chiou, E. K., & Lee, J. D. (2023) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208211009995 · Cited by 301 · cited 3×
“This is in line with previous research [13,28,46].”
Adaptive trust calibration for human-AI collaboration
Okamura, K., & Yamada, S. (2020) · PLOS ONE · doi:10.1371/journal.pone.0229132 · Cited by 239 · cited 1×
“Thus, it is important to correctly calibrate trust and thereby adjust expectations about the trustworthiness of systems [26,50,51].”
Ferrario, A., Loi, M., & Viganò, E. (2020) · Philosophy & Technology · doi:10.1007/s13347-019-00378-3 · Cited by 230 · cited 1×
“Research ranges from system acceptance and adoption [39,40] to specific investigations of trust in systems [41-44] to the calibration of trust and the appropriateness of reliance [8,9,45-49].”
Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations
Schemmer, M., Kuehl, N., Benz, C., Bartos, A., & Satzger, G. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584066 · Cited by 208 · cited 2×
“In both cases, a value close to 1 indicates a more appropriate level of trust, according to Schemmer et al [48].”
Factors influencing decision support system acceptance
Shibl, R., Lawley, M., & Debuse, J. (2013) · Decision Support Systems · doi:10.1016/j.dss.2012.09.018 · Cited by 193 · cited 1×
“Research ranges from system acceptance and adoption [39,40] to specific investigations of trust in systems [41-44] to the calibration of trust and the appropriateness of reliance [8,9,45-49].”
Dermatologist-like explainable AI enhances trust and confidence in diagnosing melanoma
Chanda, T., Hauser, K., Hobelsberger, S., Bucher, T.-C., Garcia, C. N., Wies, C., Kittler, H., Tschandl, P., Navarrete-Dechent, C., Podlipnik, S., et al. (2024) · Nature Communications · doi:10.1038/s41467-023-43095-4 · Cited by 182 · cited 2×
“Research has shown that explanations generally increase user trust in AI systems [79,80], promoting transparency and trust.”
A Deep Learning Approach Based on Explainable Artificial Intelligence for Skin Lesion Classification
Nigar, N., Umar, M., Shahzad, M. K., Islam, S., & Abalo, D. (2022) · IEEE Access · doi:10.1109/access.2022.3217217 · Cited by 179 · cited 1×
“By providing a contextual background to the AI’s decisions, explainable AI supports physicians in continually reassessing their own decisions, fostering a collaborative, human-in-the-loop dynamic that is essential for hybrid intelligence in medical decision-making [75-78].”
Jeng, D. J.-F., & Tzeng, G.-H. (2012) · Computers & Industrial Engineering · doi:10.1016/j.cie.2011.12.016 · Cited by 173 · cited 1×
“Medical experience is widely recognized as a critical factor influencing the collaborative performance of human-AI teams in medical contexts [10,18,23,57].”
The effects of personality and locus of control on trust in humans versus artificial intelligence
Sharan, N. N., & Romano, D. M. (2020) · Heliyon · doi:10.1016/j.heliyon.2020.e04572 · Cited by 160 · cited 3×
“This aspect gains importance in line with the observations by Sharan and Romano [20], who emphasize that a sense of control over technology can enhance trust by reducing participants’ anxiety when interacting with the system.”
Ma, S., Lei, Y., Wang, X., Zheng, C., Shi, C., Yin, M., & Ma, X. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581058 · Cited by 143 · cited 1×
“Previous studies [23,24,47] have found a similar relationship between confidence and following advice.”
Inkpen, K., Chappidi, S., Mallari, K., Nushi, B., Ramesh, D., Michelucci, P., Mandava, V., Vepřek, L. H., & Quinn, G. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3534561 · Cited by 132 · cited 1×
“As clinicians gain experience, they develop a deeper understanding of clinical contexts and decision-making processes, which may influence both trust in and reliance on CDSSs [8,15-17].”
Thalpage, N. S. (2023) · Journal of Digital Art & Humanities · doi:10.33847/2712-8148.4.1_4 · Cited by 132 · cited 1×
“Research has shown that explanations generally increase user trust in AI systems [79,80], promoting transparency and trust.”
Brennan, M., Puri, S., Ozrazgat-Baslanti, T., Feng, Z., Ruppert, M., Hashemighouchani, H., Momcilovic, P., Li, X., Wang, D. Z., & Bihorac, A. (2019) · Surgery · doi:10.1016/j.surg.2019.01.002 · Cited by 131 · cited 2×
“Finally, the physicians’ need for cognition did not have a significant influence on trust or reliance, as suggested by previous research by Brennan et al [65].”
Molina, M. D., & Sundar, S. S. (2024) · New Media & Society · doi:10.1177/14614448221103534 · Cited by 124
Riedl, R. (2022) · Electronic Markets · doi:10.1007/s12525-022-00594-4 · Cited by 123 · cited 2×
“These findings can inform the development of systems that are designed not only for acceptance, but also for effective and appropriate use, thus promoting beneficial human-AI collaboration [10,12].”
Knop, M., Weber, S., Mueller, M., & Niehaves, B. (2022) · JMIR Human Factors · doi:10.2196/28639 · Cited by 103 · cited 6×
“Our findings extend prior research [10,11,23] by addressing the appropriateness of reliance on these systems.”
InfluentialDreiseitl, S., & Binder, M. (2005) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2004.07.007 · Cited by 102 · cited 2×
“This approach follows the methodology described by Dreiseitl and Binder [27] and is supported by qualitative findings by Küper and Krämer [28], which suggests that slider positions within the outermost 20% (0-20 or 80-100) reflect high confidence, while positions in the middle range indicate…”
Winkler, J. K., Blum, A., Kommoss, K., Enk, A., Toberer, F., Rosenberger, A., & Haenssle, H. A. (2023) · JAMA Dermatology · doi:10.1001/jamadermatol.2023.0905 · Cited by 96 · cited 1×
“To explore this, we conducted a subgroup analysis based on clinical expertise development, using a 5-year threshold informed by prior research [72], to distinguish between “early-career” and “established” dermatologists.”
Human-AI interaction in skin cancer diagnosis: a systematic review and meta-analysis
Krakowski, I., Kim, J., Cai, Z. R., Daneshjou, R., Lapins, J., Eriksson, H., Lykou, A., & Linos, E. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01031-w · Cited by 87 · cited 2×
“This approach highlights the complementary strengths of human expertise and AI capabilities, and how their integration can address each other’s limitations and improve diagnostic outcomes beyond what either could achieve independently [38].”
You Complete Me: Human-AI Teams and Complementary Expertise
Zhang, Q., Lee, M. L., & Carter, S. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517791 · Cited by 85 · cited 1×
“While complementary expertise is the basis for collaborative human-AI decision-making [74], participants with high medical experience appear to need additional cues to build trust in the AI and make it easier to identify when it might be beneficial to follow AI suggestions.”
Trust in AI: why we should be designing for APPROPRIATE reliance
Benda, N. C., Novak, L. L., Reale, C., & Ancker, J. S. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocab238 · Cited by 76 · cited 2×
“Research ranges from system acceptance and adoption [39,40] to specific investigations of trust in systems [41-44] to the calibration of trust and the appropriateness of reliance [8,9,45-49].”
Do Humans Trust Advice More if it Comes from AI?: An Analysis of Human-AI Interactions
Vodrahalli, K., Daneshjou, R., Gerstenberg, T., & Zou, J. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534150 · Cited by 63 · cited 5×
“Our research identified medical experience as a significant factor influencing self-reported trust and reliance behavior, which is consistent with previous research [58-60].”
InfluentialAdaptive Cognitive Mechanisms to Maintain Calibrated Trust and Reliance in Automation
Lebiere, C., Blaha, L. M., Fallon, C. K., & Jefferson, B. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.652776 · Cited by 62 · cited 1×
“Thus, it is important to correctly calibrate trust and thereby adjust expectations about the trustworthiness of systems [26,50,51].”
Aquino, Y. S. J., Carter, S. M., Houssami, N., Braunack-Mayer, A., Win, K. T., Degeling, C., Wang, L., & Rogers, W. A. (2025) · Journal of Medical Ethics · doi:10.1136/jme-2022-108850 · Cited by 62
Felmingham, C. M., Adler, N. R., Ge, Z., Morton, R. L., Janda, M., & Mar, V. J. (2021) · American Journal of Clinical Dermatology · doi:10.1007/s40257-020-00574-4 · Cited by 57 · cited 8×
“Our findings extend prior research [10,11,23] by addressing the appropriateness of reliance on these systems.”
InfluentialMachine Learning and Knowledge Extraction
Holzinger, A., Kieseberg, P., Tjoa, A. M., & Weippl, E. R. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-57321-8 · Cited by 54
Cabitza, F., Campagner, A., Angius, R., Natali, C., & Reverberi, C. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581095 · Cited by 51 · cited 5×
“Research ranges from system acceptance and adoption [39,40] to specific investigations of trust in systems [41-44] to the calibration of trust and the appropriateness of reliance [8,9,45-49].”
InfluentialPsychological Traits and Appropriate Reliance: Factors Shaping Trust in AI
Küper, A., & Krämer, N. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2024.2348216 · Cited by 49 · cited 8×
“Comparing our results with the study by Küper and Krämer [28], in which a general audience was asked to perform art period classification tasks, our research shows notable differences.”
InfluentialSelf-citationImproving trust and confidence in medical skin lesion diagnosis through explainable deep learning
Metta, C., Beretta, A., Guidotti, R., Yin, Y., Gallinari, P., Rinzivillo, S., & Giannotti, F. (2023) · International Journal of Data Science and Analytics · doi:10.1007/s41060-023-00401-z · Cited by 45
Doyen, S., & Dadario, N. B. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.765406 · Cited by 40
Felmingham, C. M., MacNamara, S., Cranwell, W. C., Williams, N., Wada, M., Adler, N. R., Ge, Z., Sharfe, A., Bowling, A., Haskett, M., et al. (2022) · BMJ Open · doi:10.1136/bmjopen-2021-050203 · Cited by 37 · cited 1×
“Among various AI systems, CNNs have been extensively used for automated image recognition, particularly in the diagnosis of skin cancer and melanoma [31].”
Determinants of Laypersons’ Trust in Medical Decision Aids: Randomized Controlled Trial
Kopka, M., Schmieding, M. L., Rieger, T., Roesler, E., Balzer, F., & Feufel, M. A. (2022) · JMIR Human Factors · doi:10.2196/35219 · Cited by 37 · cited 3×
“Previous studies [23,24,47] have found a similar relationship between confidence and following advice.”
Should I Follow AI-based Advice? Measuring Appropriate Reliance in Human-AI Decision-Making
Schemmer, M., Hemmer, P., Kühl, N., Benz, C., & Satzger, G. (2022) · arXiv · doi:10.48550/arxiv.2204.06916 · Cited by 33 · cited 6×
“Research ranges from system acceptance and adoption [39,40] to specific investigations of trust in systems [41-44] to the calibration of trust and the appropriateness of reliance [8,9,45-49].”
InfluentialCalibrating Human-AI Collaboration: Impact of Risk, Ambiguity and Transparency on Algorithmic Bias
Schmidt, P., & Biessmann, F. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-57321-8_24 · Cited by 26
Cognitive bias and decision aid use under stressful conditions
Phillips-Wren, G., Jefferson, T., & McKniff, S. (2019) · Journal of Decision Systems · doi:10.1080/12460125.2019.1643695 · Cited by 22 · cited 1×
“In addition to these psychological factors and medical experience, confidence in the initial decision is a key factor influencing decision-making processes in high-stakes environments, such as medical diagnosis [23-25].”
Development and validation of the trust in AI scale (TAIS)
Wischnewski, M., Doebler, P., & Krämer, N. C. (2026) · Journal of Psychology and AI · doi:10.1080/29974100.2026.2715475 · Cited by 19 · cited 1×
“To assess trust, we used the global trust scale developed by Wischnewski et al [68].”
Self-citationSOUSA, V. E. C., LOPEZ, K. D., FEBRETTI, A., STIFTER, J., YAO, Y., JOHNSON, A., WILKIE, D. J., & KEENAN, G. M. (2015) · CIN: Computers, Informatics, Nursing · doi:10.1097/cin.0000000000000185 · Cited by 18 · cited 3×
“Our research identified medical experience as a significant factor influencing self-reported trust and reliance behavior, which is consistent with previous research [58-60].”
Faulhaber, A. K., Ni, I., & Schmidt, L. (2021) · Mensch und Computer (MuC) · doi:10.1145/3473856.3473886 · Cited by 17
The Role of Personality in Trust in Public Policy Automation
Waggoner, P., & Kennedy, R. (2022) · Journal of Behavioral Data Science · doi:10.35566/v2n1/p4/ · Cited by 7
AI in Dermatology: A Systematic Review on Skin Cancer Detection
Andromeda, S., & Dwijaksara, N. L. B. (2024) · TIERS Information Technology Journal · doi:10.38043/tiers.v5i1.5444 · Cited by 4 · cited 2×
“AI systems excel at processing large datasets, identifying subtle patterns, and providing consistent and rapid analysis, which is particularly valuable for diagnostic support [34].”
Who should do as AI say? Only non-task expert physicians benefit from correct explainable AI advice
Gaube, S., Suresh, H., Raue, M., Lermer, E., Koch, T., Hudecek, M., Ackery, A. D., Grover, S. C., Coughlin, J. F., Frey, D., et al. (2022) · doi:10.21203/rs.3.rs-1687219/v1 · Cited by 3
The impact of subliminal stimuli on interpersonal trust and team trust
Cai, J., Tang, X., Zhang, J., & Sun, X. (2022) · PsyCh Journal · doi:10.1002/pchj.619 · Cited by 2 · cited 1×
“This is based on previous research that identified trust in the system as a mediating factor [53] that mediates the influence of personal differences on system adoption [54].”
Human factors influencing reliance on AIPreprint
Küper, A. (2023) · OSF Preprints (OSF Preprints) · doi:10.17605/osf.io/87tpu · Cited by 2
Self-citationBias in AI and Machine Learning: The Impact of COVID-19 in African Healthcare Communities
Galvin, A., & Hogan, A. (2024) · AIJR Proceedings · doi:10.21467/proceedings.157.10 · Cited by 1 · cited 1×
“Issues such as amplification of biases present in the training data, failure to generalize effectively beyond specific populations, or errors in classification can affect reliability [2-7].”
Artificial Intelligence in Skin Cancer: A Literature Review from Diagnosis to Prevention and Beyond
Said, K. K., Rhaimi, C. B., & Alatresh, S. A. (2024) · Journal for Research in Applied Sciences and Biotechnology · doi:10.55544/jrasb.3.5.26 · Cited by 1
Research works printed but not linked: 5
- Agrawal, V, arXiv
- HandrichMAlexa, you freak me out – identifying drivers of innovation resistance and adoption of intelligent personal assistantsProceedings of Second International Conference on Intelligent System 20212021ICIS 2021December 12-15, 2021Austin, TX
- Beier, G, 1999, Report Psychologie, 9, 684
- JessupSASchneiderTRAlarconGMRyanTJCapiolaAThe measurement of the propensity to trust automationProceedings of the 11th International Conference on Virtual, Augmented and Mixed Reality. Applications and Case Studies2019VAMR 2019July 26-31, 2019Orlando, FL
- Papenmeier, A, arXiv
Cited by
Exploring automation bias in human–AI collaboration: a review and implications for explainable AI
Romeo, G., & Conti, D. (2026) · AI & SOCIETY · doi:10.1007/s00146-025-02422-7 · Cited by 209
“…biases, which can distort perceptions of reliability (Naiseh et al. 2023), Across the studies, trust consistently emerged as a central determinant of overreliance: Küper et al. (2025) reported that trust accounted for up to 24.1% of the variance of reliant behavior, whether appropriate or not.”
InfluentialImpact of AI recommendation correctness on diagnostic accuracy in clinical decision-making
Kücking, F., Busch, D., Przysucha, M., Kutza, J.-O., Hannemann, N., Hüsers, J., Babitsch, B., & Hübner, U. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.106223 · Cited by 42
Jackson, N. J., Brown, K. E., Rn, M., Murrow, M., Cauley, M. R., Collins, B. X., Novak, L. L., Benda, N. C., & Ancker, J. S. (2026) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocag002 · Cited by 19
“The attitudes measured differed widely with most studies reporting some measure of trust in AI 40,56,57,71 or users’ confidence in their own ability (sometimes relative to AI), 35,40,55,68,72 with several studies inferring attitudes toward AI by comparing acceptance of AI advice to human advice.”
InfluentialExploring the impact of explainable AI and cognitive capabilities on users’ decisions
Cau, F. M., & Spano, L. D. (2025) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-025-09438-0 · Cited by 20
“…engagement with AI assistance) to promote more deliberate, critical thinking, and offer potential improvements in accuracy and appropriate reliance on AI (Buçinca et al. 2021; He et al. 2023a,b; Sal-imzadeh et al. 2024; Agudo et al. 2024; Morrison et al. 2024; Cao et al. 2024b; Küper et al. 2025).”
Dong, A., Li, P., Chen, Y., Gibson, S., Zhao, L., & He, M. (2026) · Sustainability · doi:10.3390/su18115313 · Cited by 9
Metadata
- Authors
- Küper, A., Lodde, G. C., Livingstone, E., Schadendorf, D., & Krämer, N.
- Affiliations
- University of Duisburg-Essen (DE)
- Essen University Hospital (DE)
- Corresponding author
- Küper, Alisa
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2025-04-04
- Volume
- 27
- Pages
- e58660
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC12008695
- OpenAlex
- W4408250977
- Cited by
- 31
- References
- 78
- Citations by year
- 2026: 27 · 2025: 4
- FWCI
- 7.41
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Universitätsbibliographie, Universität Duisburg-Essen (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Deutsche Forschungsgemeinschaft
- Universität Duisburg-Essen
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- dermatologists, clinical experience, self-reliance, control beliefs, need for cognition
- MeSH terms
- Dermatologists, Adult, Artificial Intelligence, Female, Humans, Male, Middle Aged, Surveys and Questionnaires, Decision Support Systems, Clinical, Internet, Trust
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.