The Role of Domain Expertise in User Trust and the Impact of First Impressions with Intelligent Systems
Open accessAbstract
Domain-specific intelligent systems are meant to help system users in their decision-making process. Many systems aim to simultaneously support different users with varying levels of domain expertise, but prior domain knowledge can affect user trust and confidence in detecting system errors. While it is also known that user trust can be influenced by first impressions with intelligent systems, our research explores the relationship between ordering bias and domain expertise when encountering errors in intelligent systems. In this paper, we present a controlled user study to explore the role of domain knowledge in establishing trust and susceptibility to the influence of first impressions on user trust. Participants reviewed an explainable image classifier with a constant accuracy and two different orders of observing system errors (observing errors in the beginning of usage vs. in the end). Our findings indicate that encountering errors early-on can cause negative first impressions for domain experts, negatively impacting their trust over the course of interactions. However, encountering correct outputs early helps more knowledgable users to dynamically adjust their trust based on their observations of system performance. In contrast, novice users suffer from over-reliance due to their lack of proper knowledge to detect errors.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Nourani et al. (2020) showed that novices were prone to over-reliance due to limited error-detection skills.
“Nourani et al. (2020) showed that novices were prone to over-reliance due to limited error- detection skills.”
From Investigating appropriate reliance on AI-Based decision support systems: the role of expertise, trust, and self-confidence (2025) · cites this paper as Nourani et al., 2020
Individuals' level of experience affected trustworthiness perceptions, and one study found experts were generally less willing to rely on automation than lay users.
“In addition, individuals' level of experience afected the trustworthiness perceptions [103], and one study even found that experts were generally less willing to rely on automation than lay users [113].”
From Measuring and Understanding Trust Calibrations for Automated Systems: A Survey of the State-Of-The-Art and Future Directions (2023) · cites this paper as [103]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
1 retracted
Retracted: A Semantic Machine Learning Approach for Cyber Security Monitoring
Goyal, Y., & Sharma, A. (2019) · International Conference on Computing Methodologies and Communication (ICCMC) · doi:10.1109/iccmc.2019.8819796 · Cited by 4 · see citing works
Retracted"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · see citing works
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · see citing works
Dunning, D. A. (2011) · Advances in experimental social psychology · doi:10.1016/b978-0-12-385522-0.00005-6 · Cited by 970 · see citing works
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895 · see citing works
Cai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359206 · Cited by 594 · see citing works
Kaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., & Wortman Vaughan, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376219 · Cited by 564 · see citing works
Understanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527 · see citing works
Metrics for explainable AI: challenges and prospects
Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2018) · arXiv · doi:10.48550/arxiv.1812.04608 · Cited by 518 · see citing works
The Role of Explanations on Trust and Reliance in Clinical Decision Support Systems
Bussone, A., Stumpf, S., & O'Sullivan, D. (2015) · 2015 International Conference on Healthcare Informatics · doi:10.1109/ichi.2015.26 · Cited by 441 · see citing works
Algorithm Aversion: People Erroneously Avoid Algorithms after Seeing Them ErrPreprint
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2466040 · Cited by 356 · see citing works
Hoffman, R. R., Johnson, M., Bradshaw, J. M., & Underbrink, A. J. (2013) · IEEE Intelligent Systems · doi:10.1109/mis.2013.24 · Cited by 238 · see citing works
Hohman, F., Park, H., Robinson, C., & Chau, D. H. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2934659 · Cited by 217 · see citing works
Zebrowitz, L. A. (2017) · Current Directions in Psychological Science · doi:10.1177/0963721416683996 · Cited by 214 · see citing works
How model accuracy and explanation fidelity influence user trustPreprint
Papenmeier, A., Englebienne, G., & Seifert, C. (2019) · International Joint Conference on Artificial Intelligence · doi:10.48550/arxiv.1907.12652 · Cited by 145 · see citing works
Impact of robot failures and feedback on real-time trust
Desai, M., Kaniarasu, P., Medvedev, M., Steinfeld, A. M., & Yanco, H. A. (2013) · doi:10.5555/2447556.2447663 · Cited by 144 · see citing works
Optimized Scoring Systems: Toward Trust in Machine Learning for Healthcare and Criminal Justice
Rudin, C. D., & Ustun, B. (2018) · INFORMS Journal on Applied Analytics · doi:10.1287/inte.2018.0957 · Cited by 114 · see citing works
The World is not Enough: Trust in Cognitive Agents
de Visser, E. J., Krueger, F. W., McKnight, P. E., Scheid, S., Smith, M. A. B., Chalk, S., & Parasuraman, R. (2012) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181312561062 · Cited by 107 · see citing works
Nourani, M., Kabir, S., Mohseni, S., & Ragan, E. D. (2019) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v7i1.5284 · Cited by 104 · see citing works
Self-citationNumber of Alternatives per Choice Point and Stability of Likert-Type Scales
Oaster, T. R. (1989) · Perceptual and Motor Skills · doi:10.2466/pms.1989.68.2.549 · Cited by 103 · see citing works
User Trust Dynamics: An Investigation Driven by Differences in System Performance
Yu, K., Berkovsky, S., Taib, R., Conway, D., Zhou, J., & Chen, F. (2017) · Proceedings of the 22nd International Conference on Intelligent User Interfaces · doi:10.1145/3025171.3025219 · Cited by 90 · see citing works
Explaining Explanation For “Explainable Ai”
Hoffman, R. R., Klein, G. L., & Mueller, S. T. (2018) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931218621047 · Cited by 90 · see citing works
Merritt, S. M., Lee, D., Unnerstall, J. L., & Huber, K. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814561675 · Cited by 83 · see citing works
Honeycutt, D. R., Nourani, M., & Ragan, E. D. (2020) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v8i1.7464 · Cited by 77 · see citing works
Self-citationAutomation use and automation bias
Mosier, K. L., & Skitka, L. J. (1999) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193129904300346 · Cited by 69 · see citing works
A Survey of Evaluation Methods and Measures for Interpretable Machine LearningPreprint
Mohseni, S., Zarei, N., & Ragan, E. D. (2018) · arXiv (Cornell University) · Cited by 69 · see citing works
Self-citationMachine Learning Forecasts of Risk to Inform Sentencing Decisions
Berk, R. A., & Hyatt, J. M. (2015) · Federal Sentencing Reporter · doi:10.1525/fsr.2015.27.4.222 · Cited by 67 · see citing works
Dasgupta, A., Lee, J., Wilson, R., Lafrance, R. A., Cramer, N., Cook, K. A., & Payne, S. (2016) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2016.2598544 · Cited by 59 · see citing works
Vaidyanathan, P., Pelz, J. B., Alm, C. O., Shi, P., & Haake, A. R. (2014) · ACM Symposium on Eye Tracking Research and Applications (ETRA) · doi:10.1145/2578153.2578207 · Cited by 37 · see citing works
Investigating the Importance of First Impressions and Explainable AI with Interactive Video Analysis
Nourani, M., Honeycutt, D. R., Block, J. E., Roy, C., Rahman, T., Ragan, E. D., & Gogate, V. (2020) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3334480.3382967 · Cited by 36 · see citing works
Self-citationThe World is not Enough: Trust in Cognitive Agents: (572172013-055)
de Visser, E. J., Krueger, F., McKnight, P., Scheid, S., Smith, M., Chalk, S., & Parasuraman, R. (2012) · PsycEXTRA Dataset · doi:10.1037/e572172013-055 · Cited by 34 · see citing works
Ignorant experts and erudite novices: Exploring the Dunning-Kruger effect in wine consumers
Aqueveque, C. (2017) · Food Quality and Preference · doi:10.1016/j.foodqual.2017.12.007 · Cited by 31 · see citing works
Petrak, B., Weitz, K., Aslan, I., & André, E. (2019) · IEEE RO-MAN · doi:10.1109/ro-man46459.2019.8956463 · Cited by 29 · see citing works
Nourani, M., Roy, C., Rahman, T., Ragan, E. D., Ruozzi, N., & Gogate, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.02335 · Cited by 18 · see citing works
Self-citationMatters Order: The Role of Information Order on Implicit Impression Formation
Fourakis, E., & Cone, J. (2019) · Social Psychological and Personality Science · doi:10.1177/1948550619843930 · Cited by 16 · see citing works
Automation bias and system design: a case study in a medical application
Alberdi, E., Ayton, P., Povyakalo, A. A., & Strigini, L. (2005) · IEE and MOD HFI DTC Symposium on People and Systems - Who are we Designing for? · doi:10.1049/ic:20050451 · Cited by 9 · see citing works
Medical Diagnosis using Machine Learning
Kiruba, J., Visalakshi, R., Vaishnavi, A., Ahalya, R., & Keerthi, R. (2019) · Indian Journal of Public Health Research & Development · doi:10.5958/0976-5506.2019.00898.2 · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [16]16. Hoff, K. A., and Bashir, M. 2015. Trust in automation: Integrating empirical evidence on factors that influence trust. Human factors 57(3):407-434.
Cited by
Sorted by citations
Chen, H., Gomez, C., Huang, C.-M., & Unberath, M. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00699-2 · Cited by 300 · see citing works
The role of explainable AI in the context of the AI Act
Panigutti, C., Hamon, R., Hupont, I., Llorca, D. F., Yela, D. F., Junklewitz, H., Scalzo, S., Mazzini, G., Sánchez, I., Garrido, J. S., et al. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594069 · Cited by 171 · see citing works
Wischnewski, M., Krämer, N., & Müller, E. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581197 · Cited by 146 · see citing works
“Income prediction [40, 145], number pattern discovery [26, 27], pothole detection [104], classifcation of plants/animals/images [11, 75, 103, 110, 137], meal preparation [12], online dating Others [109, 141], hotel reviews [71], clearing tables [18], fnd housing [43, 124], navigation through buildings [101, 112], long…”
BackgroundGómez, C., Cho, S. M., Ke, S., Huang, C., & Unberath, M. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1521066 · Cited by 163 · see citing works
Dikmen, M., & Burns, C. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102792 · Cited by 116 · see citing works
Anchoring Bias Affects Mental Model Formation and User Reliance in Explainable AI Systems
Nourani, M., Roy, C., Block, J. E., Honeycutt, D. R., Rahman, T., Ragan, E., & Gogate, V. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450639 · Cited by 107 · see citing works
Self-citationInteractive Model Cards: A Human-Centered Approach to Model Documentation
Crisan, A., Drouhard, M., Vig, J., & Rajani, N. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533108 · Cited by 101 · see citing works
Human-AI Collaboration: The Effect of AI Delegation on Human Task Performance and Task Satisfaction
Hemmer, P., Westphal, M., Schemmer, M., Vetter, S., Vössing, M., & Satzger, G. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584052 · Cited by 111 · see citing works
You Complete Me: Human-AI Teams and Complementary Expertise
Zhang, Q., Lee, M. L., & Carter, S. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517791 · Cited by 85 · see citing works
Lee, M. H., & Chew, C. J. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610218 · Cited by 78 · see citing works
Honeycutt, D. R., Nourani, M., & Ragan, E. D. (2020) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v8i1.7464 · Cited by 77 · see citing works
Self-citationSecond Chance for a First Impression? Trust Development in Intelligent System Interaction
Tolmeijer, S., Gadiraju, U., Ghantasala, R., Gupta, A., & Bernstein, A. (2021) · Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3450613.3456817 · Cited by 67 · see citing works
Levy, A., Agrawal, M., Satyanarayan, A., & Sontag, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445522 · Cited by 73 · see citing works
The effects of explanations on automation bias
Vered, M., Livni, T., Howe, P. D. L., Miller, T., & Sonenberg, L. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2023.103952 · Cited by 71 · see citing works
The Impact of Imperfect XAI on Human-AI Decision-Making
Morrison, K., Spitzer, P., Turri, V., Feng, M., Kühl, N., & Perer, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641022 · Cited by 69 · see citing works
Impacts of Personal Characteristics on User Trust in Conversational Recommender Systems
Cai, W., Jin, Y., & Chen, L. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517471 · Cited by 59 · see citing works
Mitigating knowledge imbalance in AI-advised decision-making through collaborative user involvement
Gómez, C., Unberath, M., & Huang, C. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102977 · Cited by 55 · see citing works
Kahr, P. K., Rooks, G., Willemsen, M. C., & Snijders, C. C. P. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3686164 · Cited by 49 · see citing works
Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistant
He, G., Aishwarya, N., & Gadiraju, U. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712133 · Cited by 53 · see citing works
How Time Pressure in Different Phases of Decision-Making Influences Human-AI Collaboration
Cao, S., Gomez, C., & Huang, C.-M. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610068 · Cited by 48 · see citing works
The Effects of Depth of Knowledge of a Virtual Agent
Yang, F., Duque, K., & Mousas, C. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3456148 · Cited by 44 · see citing works
A Missing Piece in the Puzzle: Considering the Role of Task Complexity in Human-AI Decision Making
Salimzadeh, S., He, G., & Gadiraju, U. (2023) · Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3565472.3592959 · Cited by 42 · see citing works
Zheng, C., Wu, Y., Shi, C., Ma, S., Luo, J., & Ma, X. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581131 · Cited by 41 · see citing works
Salimzadeh, S., He, G., & Gadiraju, U. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641905 · Cited by 41 · see citing works
“Such factors can be broadly classified into three primary categories: human-related factors [35, 95, 96], attributes of the AI systems [94, 98], and characteristics of the decision-making tasks [16, 56, 126].”
BackgroundKahr, P. K., Rooks, G., Willemsen, M. C., & Snijders, C. C. P. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584058 · Cited by 43 · see citing works
He, G., Buijsman, S., & Gadiraju, U. (2023) · Proc. ACM Hum. Comput. Interact. · doi:10.1145/3610067 · Cited by 59 · see citing works
Does More Advice Help? The Effects of Second Opinions in AI-Assisted Decision Making
Lu, Z., Wang, D., & Yin, M. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3653708 · Cited by 40 · see citing works
Wang, X., & Yin, M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581366 · Cited by 38 · see citing works
The Role of Human Knowledge in Explainable AI
Tocchetti, A., & Brambilla, M. (2022) · Data · doi:10.3390/data7070093 · Cited by 39 · see citing works
Visser, R. W., Peters, T. M., Scharlau, I., & Hammer, B. (2025) · Cognitive Systems Research · doi:10.1016/j.cogsys.2025.101357 · Cited by 47 · see citing works
When Algorithms Err : Differential Impact of Early vs. Late Errors on Users’ Reliance on Algorithms
Kim, A., Yang, M., & Zhang, J. (2020) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3557889 · Cited by 40 · see citing works
Painting the Black Box White: Experimental Findings from Applying XAI to an ECG Reading Setting
Cabitza, F., Campagner, A., Natali, C., Parimbelli, E., Ronzio, L., & Cameli, M. (2023) · Machine Learning and Knowledge Extraction · doi:10.3390/make5010017 · Cited by 34 · see citing works
Trust Development and Repair in AI-Assisted Decision-Making during Complementary Expertise
Pareek, S., Velloso, E., & Gonçalves, J. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658924 · Cited by 34 · see citing works
Linder, R., Mohseni, S., Yang, F., Pentyala, S., Ragan, E. D., & Hu, X. (2021) · Applied AI Letters · doi:10.1002/ail2.49 · Cited by 30 · see citing works
Self-citationNourani, M., Roy, C., Block, J. E., Honeycutt, D. R., Rahman, T., Ragan, E. D., & Gogate, V. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3531066 · Cited by 30 · see citing works
Self-citationUnderstanding Choice Independence and Error Types in Human-AI Collaboration
Erlei, A., Sharma, A., & Gadiraju, U. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641946 · Cited by 26 · see citing works
Bergomi, L., Nicora, G., Orlowska, M. A., Podrecca, C., Bellazzi, R., Fregosi, C., Salinaro, F., Bonzano, M., Crescenzi, G., Speciale, F., et al. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03045-0 · Cited by 27 · see citing works
The Trust Recovery Journey. The Effect of Timing of Errors on the Willingness to Follow AI Advice.
Kahr, P. K., Rooks, G., Snijders, C., & Willemsen, M. C. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645167 · Cited by 21 · see citing works
Gómez, C., Yin, J., Huang, C., & Unberath, M. (2024) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1427463 · Cited by 23 · see citing works
Mok, L., Nanda, S., & Anderson, A. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610100 · Cited by 19 · see citing works
Incorporating experts’ judgment into machine learning models
Park, H., Megahed, A., Yin, P., Ong, Y. J., Mahajan, P., & Pei, G. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120118 · Cited by 17 · see citing works
Lyons, J. B., Mator, J. D., Orr, T., Alarcon, G. M., & Barrera, K. (2024) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/15553434231225909 · Cited by 16 · see citing works
To Err Is AI! Debugging as an Intervention to Facilitate Appropriate Reliance on AI Systems
He, G., Bharos, A., & Gadiraju, U. (2024) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3648188.3675130 · Cited by 17 · see citing works
Horstmann, A. C., Gratch, J. S., & Krämer, N. C. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102683 · Cited by 16 · see citing works
Investigating Trust in Human-AI Collaboration for a Speech-Based Data Analytics Task
Tutul, A. A., Nirjhar, E., & Chaspari, T. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2328910 · Cited by 19 · see citing works
Dogru, E. Ö., & Krämer, N. C. (2025) · Journal of Decision Systems · doi:10.1080/12460125.2025.2593251 · Cited by 15 · see citing works
“Nourani et al. (2020) showed that novices were prone to over-reliance due to limited error-Figure detection skills.”
Preferences for AI Explanations Based on Cognitive Style and Socio-Cultural Factors
Kopecka, H., Such, J. M., & Flucke, G. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637386 · Cited by 15 · see citing works
Guided By AI: Navigating Trust, Bias, and Data Exploration in AI‐Guided Visual Analytics
Ha, S., Monadjemi, S., & Ottley, A. (2024) · Computer Graphics Forum · doi:10.1111/cgf.15108 · Cited by 14 · see citing works
Kpodo, J., & Nejadhashemi, A. P. (2025) · Artificial Intelligence in Agriculture · doi:10.1016/j.aiia.2025.04.001 · Cited by 19 · see citing works
Ooge, J., Dereu, L., & Verbert, K. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584046 · Cited by 17 · see citing works
Ford, C., & Keane, M. T. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-37731-0_15 · Cited by 13 · see citing works
Kahr, P. K., Rooks, G., Snijders, C. C. P., & Willemsen, M. (2025) · International Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713099 · Cited by 19 · see citing works
Subgoal-Based Explanations for Unreliable Intelligent Decision Support Systems
Das, D., Kim, B., & Chernova, S. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584055 · Cited by 13 · see citing works
Salimzadeh, S., & Gadiraju, U. (2024) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3627043.3659567 · Cited by 11 · see citing works
Towards Understanding AI Delegation: The Role of Self-Efficacy and Visual Processing Ability
Westphal, M., Hemmer, P., Vössing, M., Schemmer, M., Vetter, S., & Satzger, G. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3696423 · Cited by 10 · see citing works
Shamszare, H., Chaudhry, Z. S., Berenji, M., & Choudhury, A. (2025) · IEEE Access · doi:10.1109/access.2025.3586555 · Cited by 11 · see citing works
Wang, J. (2024) · Journal of Technical Writing and Communication · doi:10.1177/00472816241260033 · Cited by 10 · see citing works
Continuous Error Timing in Automation: The Peak-End Effect on Human-Automation Trust
Wang, K., Lü, J., Ruan, S., & Qi, Y. (2023) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2023.2223954 · Cited by 9 · see citing works
Strategic Adversarial Attacks in AI-assisted Decision Making to Reduce Human Trust and Reliance
Lu, Z., Li, Z., Chiang, C.-W., & Yin, M. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/337 · Cited by 9 · see citing works
Micro-entries: Encouraging Deeper Evaluation of Mental Models Over Time for Interactive Data Systems
Block, J. E., & Ragan, E. D. (2020) · doi:10.1109/beliv51497.2020.00012 · Cited by 9 · see citing works
Self-citationKaufman, R. P., Broukhim, A., Kirsh, D., & Weibel, N. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713088 · Cited by 7 · see citing works
Bubbleu: Exploring Augmented Reality Game Design with Uncertain AI-based Interaction
Kim, M., Lee, K., Balan, R. K., & Lee, Y. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581270 · Cited by 10 · see citing works
Trust in Prediction Models: a Mixed-Methods Pilot Study on the Impact of Domain Expertise
Ooge, J., & Verbert, K. (2021) · IEEE Workshop on TRust and EXpertise in Visual Analytics (TREX) · doi:10.1109/trex53765.2021.00007 · Cited by 10 · see citing works
Raees, M., Khan, V.-J., Lykourentzou, I., & Papangelis, K. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791467 · Cited by 5 · see citing works
Passi, S., Dhanorkar, S., & Vorvoreanu, M. (2025) · Springer eBooks · doi:10.1007/978-981-95-3658-0_98 · Cited by 6 · see citing works
Towards an understanding and explanation for mixed-initiative artificial scientific text detection
Weng, L., Liu, S., Zhu, H., Sun, J., Kam-Kwai, W., Han, D., Zhu, M., & Chen, W. (2024) · Information Visualization · doi:10.1177/14738716241240156 · Cited by 6 · see citing works
AI-initiated second opinions: a framework for advanced caries treatment planning
Dascalu, T., Ramezanzade, S., Bakhshandeh, A., Bjørndal, L., & Ibragimov, B. (2024) · BMC Oral Health · doi:10.1186/s12903-024-04551-9 · Cited by 6 · see citing works
Visually Explaining Uncertain Price Predictions in Agrifood: A User-Centred Case-Study
Ooge, J., & Verbert, K. (2022) · Agriculture · doi:10.3390/agriculture12071024 · Cited by 6 · see citing works
Trust Junk and Evil Knobs: Calibrating Trust in AI Visualization
Wall, E., Matzen, L. E., El‐Assady, M., Masters, P., Hosseinpour, H., Endert, A., Borgo, R., Chau, P., Perer, A., Schupp, H. T., et al. (2024) · IEEE Pacific Visualization Conference (PacificVis) · doi:10.1109/pacificvis60374.2024.00012 · Cited by 7 · see citing works
Park, J., Min, B., Son, K., Song, J. Y., Ma, X., & Kim, J. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789107 · Cited by 5 · see citing works
In AI We Trust? Exploring the Role of Explainable GenAI and Expertise in Education
Safarov, C., Gadzinski, G., & Schlögl, S. (2025) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208251401773 · Cited by 5 · see citing works
Human Evaluation of Large Language Models: A Review and Protocol Selection Framework
Brunyé, T. T. (2026) · AI · doi:10.3390/ai7050174 · Cited by 4 · see citing works
Albannai, N. A. A. (2025) · The International Journal of Management Education · doi:10.1016/j.ijme.2025.101254 · Cited by 4 · see citing works
Imperfections of XAI: Phenomena Influencing AI-Assisted Decision-Making
Spitzer, P., Morrison, K., Turri, V., Feng, M., Perer, A., & Kühl, N. (2025) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3750052 · Cited by 4 · see citing works
Degen, H., Budnik, C. J., Chitre, K., & Lintereur, A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-90963-5_31 · Cited by 4 · see citing works
Chen, Y., Verbert, K., Gerling, K., Abeele, M.-E. V., & Abeele, V. V. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712139 · Cited by 5 · see citing works
Ghaffar, F., Islam, Y. Z., Furtado, N. M., & Burns, C. M. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.111132 · Cited by 3 · see citing works
Leveraging Large Language Models to Enhance Domain Expert Inclusion in Data Science Workflows
Shih, J. Y., Mohanty, V., Katsis, Y., & Subramonyam, H. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3651115 · Cited by 3 · see citing works
Flora, M. L., Gallo, B. T., Potvin, C. K., Clark, A. J., & Wilson, K. (2024) · Weather and Forecasting · doi:10.1175/waf-d-24-0038.1 · Cited by 3 · see citing works
Anik, A. I., & Bunt, A. (2024) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc60511.2024.00024 · Cited by 3 · see citing works
An Interpretable Algorithm for Uveal Melanoma Subtyping from Whole Slide Cytology ImagesPreprint
Chen, H., Liu, T. Y. A., Gómez, C., Corrêa, Z. M., & Unberath, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.06246 · Cited by 3 · see citing works
RelEYEance: Gaze-based Assessment of Users' AI-reliance at Run-time
Wu, Z., Wang, Y., Langer, M., & Feit, A. M. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3725841 · Cited by 2 · see citing works
Evaluation and incident prevention in an enterprise AI assistant
Maharaj, A. V., Arbour, D. T., Lee, D., Bhattacharya, U., Rao, A. B., Zane, A. V., Feller, A., Qian, K., Rahman, S., & Li, Y. (2025) · AI Magazine · doi:10.1002/aaai.70028 · Cited by 2 · see citing works
Wang, Y., Zhao, J. L., Ba, Y., Mancenido, M. V., Chiou, E. K., & Maciejewski, R. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2564272 · Cited by 2 · see citing works
Making informed trust judgements: the role of metacognition in ChatGPT interactions
Colville, S., & Ostern, N. (2026) · Behaviour & Information Technology · doi:10.1080/0144929x.2026.2651114 · Cited by 1 · see citing works
Yamada, K. (2026) · AI and Ethics · doi:10.1007/s43681-026-01295-w · Cited by 2 · see citing works
Divergent impacts of explainable AI for dermatological diagnosis on clinicians versus lay people
Xu, X., Hu, H.-Y., Zhang, H., Wang, W., Wang, R., Soenksen, L., Badri, O., Jafry, S., Burger, E., Nwandu, L., et al. (2026) · Nature Medicine · doi:10.1038/s41591-026-04553-w · Cited by 2 · see citing works
Huang, Y., Shen, C., Li, W., Wang, X., Jin, B., & Cai, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72111-3_45 · Cited by 2 · see citing works
Algorithmic Rulemaking vs. Algorithmic GuidancePreprint
Henderson, P., & Krass, M. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4784350 · Cited by 2 · see citing works
Anik, A. I., & Bunt, A. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789087 · Cited by 1 · see citing works
Beyond Anthropomorphism: a Spectrum of Interface Metaphors for LLMs
So, J., Cheng, C., & Murthy, S. K. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798591 · Cited by 1 · see citing works
Jessup, S. A., Alarcon, G. M., Willis, S. M., & Lee, M. A. (2024) · Applied Ergonomics · doi:10.1016/j.apergo.2024.104363 · Cited by 1 · see citing works
Chapter 8: Conclusion – strategies and policy decisions in the new digital-era governance
Tan, E., & Crompvoets, J. (2022) · Wageningen Academic Publishers eBooks · doi:10.3920/978-90-8686-930-5_8
Influence of Errors on the Evaluation of Text Classification Systems
Bracamonte, V., Hidano, S., & Kiyomoto, S. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-45725-8_8
Recognizing Selected Features in the News Text
Kaczmarek, A., Tobolewski, F., Ganzha, M., Paprzycki, M., & Wróblewska, A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-86193-2_17
Vazquez, E., & Brugh, C. S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93864-1_10
Natali, C., Marconi, L., Fregosi, C., & Cabitza, F. (2025) · IFIP advances in information and communication technology · doi:10.1007/978-3-031-95334-7_11
Liebherr, M., Babiker, A., Alshakhsi, S. A. A., Al‐Thani, D., Montag, C., & Ali, R. (2025) · Advances in Human-Computer Interaction · doi:10.1155/ahci/5590506
Know Thyself: Exploring Self-Awareness of Execution Skill in Darts
Culmer, J., Sheppard, M., & Archibald, C. (2025) · Annual Conference on Human-Computer Interaction and Sports (SportsHCI) · doi:10.1145/3749385.3749408
Behzad, T., Gurney, N., Wang, N., & Pynadath, D. V. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-13187-4_8
- Authors
- Nourani, M., King, J., & Ragan, E.
- Affiliations
- University of Florida (US)
- Texas A&M University (US)
- Venue
- Proceedings of the AAAI Conference on Human Computation and Crowdsourcing
- Published
- 2020-10-01
- Volume
- 8
- Pages
- 112-121
- ISSN
- 2769-1330 (ISSN-L)
- 2769-1349
- OpenAlex
- W3070321535
- Semantic Scholar
- 23c9685bbecaa187ea4d0d1f8aed8ca46f9bb996
- Cited by
- 116
- References
- 36
- Citations by year
- 2026: 25 · 2025: 23 · 2024: 26 · 2023: 21 · 2022: 11 · 2021: 8 · 2020: 2
- FWCI
- 14.82
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the AAAI Conference on Human Computation and Crowdsourcing (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1900767)
- Defense Advanced Research Projects Agency (N66001, N66001-17-2-4032)
- Topics
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- domain knowledge, user trust, first impressions, intelligent systems, error detection, trust dynamics, novice users, expert users, trust in AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
