The DOI matches its Crossref record; 114 of the 116 references it cites were found in a registry or an index.
Manipulating and Measuring Model Interpretability
Open accessAbstract
With machine learning models being increasingly used to aid decision making even in high-stakes domains, there has been a growing interest in developing interpretable models. Although many supposedly interpretable models have been proposed, there have been relatively few experimental studies investigating whether these models achieve their intended effects, such as making people more closely follow a model’s predictions when it is beneficial for them to do so or enabling them to detect when a model has made a mistake. We present a sequence of pre-registered experiments (N = 3, 800) in which we showed participants functionally identical models that varied only in two factors commonly thought to make machine learning models more or less interpretable: the number of features and the transparency of the model (i.e., whether the model internals are clear or black box). Predictably, participants who saw a clear model with few features could better simulate the model’s predictions. However, we did not find that participants more closely followed its predictions. Furthermore, showing participants a clear model meant that they were less able to detect and correct for the model’s sizable mistakes, seemingly due to information overload. These counterintuitive findings emphasize the importance of testing over intuition when developing interpretable models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Some researchers measure subjective trust using questionnaires, while others measure objective trust related behavior such as reliance.”
From Exploring the effects of human-centered AI explanations on trust and reliance (2023) · cites this paper as Poursabzi-Sangdeh et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While WOA has been used in the past by researchers in XAI as an alternative trust measurement (Logg et al., 2019; Mucha et al., 2021; Poursabzi-Sangdeh et al., 2021), it has never been explicitly referred to as reliance and thus clearly differentiated from trust to the best of our knowledge.”
From Exploring the effects of human-centered AI explanations on trust and reliance (2023) · cites this paper as Poursabzi-Sangdeh et al., 2021
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, across many domains, empirical studies of explanations have produced mixed results [4,11,49,53,58,68,82,87], with some suggesting that explanations increase decisionmakers' tendency to rely on AI predictions even when the AI system is wrong [4,68,82,87]-a phenomenon often referred to as overreliance [10,78].”
From Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations (2023) · cites this paper as [68]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“On the contrary, several studies found that showing feature-based explanations increases people's tendency to over-rely on the model when it is wrong, compared to showing only the AI predictions [4,68,82,87].”
From Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations (2023) · cites this paper as [68]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Such explanations often do not match the end-users' mental models [56], making the explanations ineective [2,40,65] and at times even harmful [13,28,30,47,51].”
From Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust (2023) · cites this paper as [40]
Experimental — accuracy not yet measured
The cited source says what the claim attributes to it.
In the citing paper
“Work in the human–computer interaction community has identified that increased transparency can hamper users’ ability to detect sizable model errors and correct for them, “seemingly due to information overload””
From The false hope of current approaches to explainable artificial intelligence in health care (2021) · cites this paper as [15]
The cited source says what the claim attributes to it.
In the citing paper
“and evidence suggests models that use explainability techniques can hamper people's ability to detect when a model makes serious mistakes”
From The false hope of current approaches to explainable artificial intelligence in health care (2021) · cites this paper as [15]
The cited source says what the claim attributes to it.
In the citing paper
“However, many empirical studies on human-AI decision-making did not observe complementary performance [6,16,37,52,58,61,68,82,84,87].”
From Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations (2023) · cites this paper as [68]
The cited source says what the claim attributes to it.
In the citing paper
“naive application of existing XAI methods may be ineective [2,40,65] to broader audience of end-users, if not harmful [28,47,49,51].”
From Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust (2023) · cites this paper as [40]
The cited source says what the claim attributes to it.
In the citing paper
“and that providing too much detail may reduce user performance due to information overload [44].”
From Explainable AI in hospital clinical decision support systems: A scoping review of healthcare professionals’ perspectives (2026) · cites this paper as [44]
The cited source says what the claim attributes to it.
In the citing paper
“a study by Poursabzi-Sangdeh et al. [48] found that more 'clear' models can actually lead to information overload and hinder people's ability to detect AI errors.”
From Improving Human-AI Collaboration With Descriptions of AI Behavior (2023) · cites this paper as Poursabzi-Sangdeh et al. [48]
The cited source says what the claim attributes to it.
In the citing paper
“However, they choose to use the task of real estate valuation, and how it relates to a decision and an outcome of interest and the utility of explanations in achieving that goal was not established.”
From Explainable machine learning for public policy: Use cases, gaps, and research directions (2023) · cites this paper as Poursabzi-Sangdeh et al. (2021)
The cited source says what the claim attributes to it.
In the citing paper
“Algorithmic transparency similarly reduces people's ability to detect and correct model errors”
From The flaws of policies requiring human oversight of government algorithms (2022) · cites this paper as Poursabzi-Sangdeh et al., 2021
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“The complexity of tasks plays a signifcant role in determining the degree of reliance on AI advice, consistent with the fndings of [9, 105].”
From Dealing with Uncertainty: Understanding the Impact of Prognostic Versus Diagnostic Tasks on Trust and Reliance in Human-AI Decision Making (2024) · cites this paper as [105]
Experimental — accuracy not yet measuredIn the citing paper
“Many studies have found that cognitive biases (particularly anchoring bias and automation bias more broadly) can lead users to misunderstand model behavior and place unwarranted trust in AI systems [20, 34, 52, 55].”
From Interactive Model Cards: A Human-Centered Approach to Model Documentation (2022) · cites this paper as [55]
In the citing paper
“Previous works show that the availability of AI's explanations, regardless of their correctness, is likely to increase people's trust in AI [6,54,81,100].”
From Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making (2023) · cites this paper as [81]
In the citing paper
“Many studies have investigated the role of AI in assisting humans in making individual decisions [9,16,36, 37, 50, 51, 55, 58, 69, 83, 84, 89], e.g., predicting whether a person will recidivate in the near future.”
From Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation (2022) · cites this paper as [69]
In the citing paper
“However, recent research has shown that the recommendation-driven explainable/interpretable AI has little effect on decision making [1, 6-8, 28, 32], although this is not always the case [22,23,41].”
From Explainable AI is Dead, Long Live Explainable AI!: Hypothesis-driven Decision Support using Evaluative AI (2023) · cites this paper as [32]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Fitting Linear Mixed-Effects Models Using lme4
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015) · Journal of Statistical Software · doi:10.18637/jss.v067.i01 · Cited by 86,835
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 21,030
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Gauriat, C.-M., Pencolé, Y., Ribot, P., et al. (2024) · HAL · doi:10.4230/oasics.dx.2024.27 · Cited by 12,623
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Simple Heuristics That Make Us Smart
Gigerenzer, G., & Todd, P. M. (1999) · Cited by 3,563
Reasoning the fast and frugal way: Models of bounded rationality.
Gigerenzer, G., & Goldstein, D. G. (1996) · Psychological Review · doi:10.1037/0033-295x.103.4.650 · Cited by 3,504
Self-citationAlgorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172
Towards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121
Algorithms of Oppression: How Search Engines Reinforce Racism
Noble, S. U. (2020) · doi:10.2307/j.ctt1pwt9w5 · Cited by 2,709
“Clinical versus Actuarial Judgment.”
Dawes, R. M., Faust, D., & Meehl, P. E. (1989) · Science · doi:10.1126/science.2648573 · Cited by 2,298
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159
Inside the Turk: Understanding Mechanical Turk as a Participant Pool
Paolacci, G., & Chandler, J. (2014) · Current Directions in Psychological Science · doi:10.1177/0963721414531598 · Cited by 2,136
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
The robust beauty of improper linear models in decision making.
Dawes, R. M. (1979) · American Psychologist · doi:10.1037/0003-066x.34.7.571 · Cited by 2,046
Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission
Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., & Elhadad, N. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2788613 · Cited by 1,685
Casler, K., Bickel, L., & Hackett, E. (2013) · Computers in Human Behavior · doi:10.1016/j.chb.2013.05.009 · Cited by 1,662 · cited 1×
“tudies show that data from high reputation Mechanical Turk workers are comparable to data from commercial panels and university pools when assessing factors such as attentiveness, honesty, and effort [11, 13, 16, 53, 59]. (a) Clear, two-feature condition (CLEAR-2). (b) Black-box, two-feature condition (BB-2). (c) Clea…”
BackgroundClinical versus mechanical prediction: a meta-analysis
Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000) · Psychological Assessment · doi:10.1037/1040-3590.12.1.19 · Cited by 1,640
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514
The algorithmic imaginary: exploring the ordinary affects of Facebook algorithms
Bucher, T. (2016) · Information Communication & Society · doi:10.1080/1369118x.2016.1154086 · Cited by 1,420
Machine learning for medical diagnosis: history, state of the art and perspective
Kononenko, I. (2001) · Artificial Intelligence in Medicine · doi:10.1016/s0933-3657(01)00077-x · Cited by 1,306
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2018) · Management Science · doi:10.1287/mnsc.2016.2643 · Cited by 1,230
Bennett, J. R., & Lanning, S. (2007) · Cited by 1,140
Management Misinformation Systems
Ackoff, R. L. (1967) · Management Science · doi:10.1287/mnsc.14.4.b147 · Cited by 1,102
Counterfactual explanations without opening the black box: automated decisions and the GDPR
Wachter, S., Mittelstadt, B., & Russell, C. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3063289 · Cited by 1,061
Statistical Classification Methods in Consumer Credit Scoring: A Review
Hand, D. J., & Henley, W. E. (1997) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.1997.00078.x · Cited by 1,042
Human Decisions and Machine Predictions
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018) · The Quarterly Journal of Economics · doi:10.1093/qje/qjx032 · Cited by 1,040
WHY SUMMARIES OF RESEARCH ON PSYCHOLOGICAL THEORIES ARE OFTEN UNINTERPRETABLE
Meehl, P. E. (1990) · Psychological Reports · doi:10.2466/pr0.1990.66.1.195 · Cited by 934
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925
Designing Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906
Models and Managers: The Concept of a Decision Calculus
Little, J. D. C. (2004) · Management Science · doi:10.1287/mnsc.1040.0267 · Cited by 812
The cost structure of sensemaking
Russell, D. M., Stefik, M., Pirolli, P. L. T., & Card, S. K. (1993) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/169059.169209 · Cited by 793
Contextual cueing of visual attention
Chun, M. M. (2000) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01476-5 · Cited by 732
'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions
Binns, R., Van Kleek, M., Veale, M., Lyngs, U., Zhao, J., & Shadbolt, N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173951 · Cited by 691
Interpretable Decision Sets: A Joint Framework for Description and Prediction
Lakkaraju, H., Bach, S. H., & Leskovec, J. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939874 · Cited by 687
Receiving other people’s advice: Influence and benefit
Yaniv, I. (2004) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2003.08.002 · Cited by 652
Effects of Quality and Quantity of Information on Decision Effectiveness
Keller, K. L., & Staelin, R. (1987) · Journal of Consumer Research · doi:10.1086/209106 · Cited by 634
Accurate intelligible models with pairwise interactions
Lou, Y., Caruana, R., Gehrke, J., & Hooker, G. (2013) · Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2487575.2487579 · Cited by 564 · cited 1×
“simple models, such as point systems that can be memorized [35, 69] or generative additive models in which the impact of each feature on the model’s prediction should be relatively easy to understand [12, 50, 51]. The second is to provide post-hoc explanations for (potentially complex) models. One thread of research i…”
BackgroundKaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., & Wortman Vaughan, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376219 · Cited by 564
Self-citationWhy and why not explanations improve the intelligibility of context-aware intelligent systems
Lim, B. Y., Dey, A. K., & Avrahami, D. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1519023 · Cited by 552
Intelligible models for classification and regression
Lou, Y., Caruana, R., & Gehrke, J. (2012) · Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2339530.2339556 · Cited by 547
Generalizing from Survey Experiments Conducted on Mechanical Turk: A Replication Approach
Coppock, A. (2019) · Political Science Research and Methods · doi:10.1017/psrm.2018.10 · Cited by 515
Actionable Recourse in Linear Classification
Ustun, B., Spangher, A., & Liu, Y. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287566 · Cited by 501
Perspectives on Information Overload
Jacoby, J. (1984) · Journal of Consumer Research · doi:10.1086/208981 · Cited by 495
Veale, M., Van Kleek, M., & Binns, R. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174014 · Cited by 490 · cited 4×
“[106] Michael Veale, Max Van Kleek, and Reuben Binns.”
BackgroundInfluentialDatabase Models and Managerial Intuition: 50% Model + 50% Manager
Blattberg, R. C., & Hoch, S. J. (1990) · Management Science · doi:10.1287/mnsc.36.8.887 · Cited by 473
Intelligibility and Accountability: Human Considerations in Context-Aware Systems
Bellotti, V., & Edwards, K. (2001) · Human–Computer Interaction · doi:10.1207/s15327051hci16234_05 · Cited by 471
Huysmans, J., Dejaeger, K., Mues, C., Vanthienen, J., & Baesens, B. (2011) · Decision Support Systems · doi:10.1016/j.dss.2010.12.003 · Cited by 425
Too much, too little, or just right? Ways explanations impact end users' mental models
Kulesza, T., Stumpf, S., Burnett, M., Yang, S., Kwan, I., & Wong, W.-K. (2013) · 2013 IEEE Symposium on Visual Languages and Human Centric Computing · doi:10.1109/vlhcc.2013.6645235 · Cited by 388
Explanations as Mechanisms for Supporting Algorithmic Transparency
Rader, E., Cotter, K., & Cho, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173677 · Cited by 388
The Perceived Utility of Human and Automated Aids in a Visual Detection Task
Dzindolet, M. T., Pierce, L. G., Beck, H. P., & Dawe, L. A. (2002) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/0018720024494856 · Cited by 386
Understanding User Beliefs About Algorithmic Curation in the Facebook News Feed
Rader, E., & Gray, R. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702174 · Cited by 386 · cited 1×
“ent [33], judicial sentencing and bail [3, 14, 35, 37], and hiring [43]. In addition, these models influence decisions about what news people see on social media platforms (e.g., Twitter and Facebook, [2, 10, 48, 61]), provide recommendations for what movies to watch (e.g., Netflix, [7]) or what music to listen to on on…”
BackgroundAn Experimental Comparison of Different Approaches to Determining Weights in Additive Utility Models
Schoemaker, P. J. H., & Waid, C. C. (1982) · Management Science · doi:10.1287/mnsc.28.2.182 · Cited by 381
Explaining Recommendations: Design and Evaluation
Tintarev, N., & Masthoff, J. (2015) · Springer eBooks · doi:10.1007/978-1-4899-7637-6_10 · Cited by 366
The relative influence of advice from human experts and statistical methods on forecast adjustments
Önkal, D., Goodwin, P., Thomson, M. E., Gonul, M. S., & Pollock, A. C. (2009) · Journal of Behavioral Decision Making · doi:10.1002/bdm.637 · Cited by 331
MTurk Character Misrepresentation: Assessment and Solutions
Sharpe Wessling, K., Huber, J., & Netzer, O. (2017) · Journal of Consumer Research · doi:10.1093/jcr/ucx053 · Cited by 330
InterpretML: A Unified Framework for Machine Learning Interpretability
Nori, H., Jenkins, S., Koch, P., & Caruana, R. (2019) · arXiv · doi:10.48550/arxiv.1909.09223 · Cited by 322
Supersparse linear integer models for optimized medical scoring systems
Ustun, B., & Rudin, C. D. (2015) · Machine Learning · doi:10.1007/s10994-015-5528-6 · Cited by 316
Mehrotra, R., McInerney, J., Bouchard, H., Lalmas, M., & Díaz, F. (2018) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3269206.3272027 · Cited by 301
Explaining Decision-Making Algorithms through UI: Strategies to Help Non-Expert Stakeholders
Cheng, H.-F., Wang, R., Zhang, Z., O'Connell, F., Gray, T., Harper, F. M., & Zhu, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300789 · Cited by 292
Effects of task difficulty on use of advice
Gino, F., & Moore, D. A. (2006) · Journal of Behavioral Decision Making · doi:10.1002/bdm.539 · Cited by 288
Explaining models: an empirical study of how explanations impact fairness judgment
Dodge, J., Liao, Q. V., Zhang, Y., Bellamy, R. K. E., & Dugan, C. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302310 · Cited by 286
Kocielnik, R., Amershi, S., & Bennett, P. N. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300641 · Cited by 277
Faithful and Customizable Explanations of Black Box Models
Lakkaraju, H., Kamar, E. S., Caruana, R., & Leskovec, J. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3314229 · Cited by 268
psiTurk: An open-source framework for conducting replicable behavioral experiments online
Gureckis, T. M., Martin, J. B., McDonnell, J. V., Rich, A. S., Markant, D. B., Coenen, A., Halpern, D., Hamrick, J. B., & Chan, P. (2015) · Behavior Research Methods · doi:10.3758/s13428-015-0642-8 · Cited by 263
The challenge of crafting intelligible intelligence
Weld, D. S., & Bansal, G. (2019) · Communications of the ACM · doi:10.1145/3282486 · Cited by 253
Hohman, F., Head, A., Caruana, R., DeLine, R. A., & Drucker, S. M. (2019) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300809 · Cited by 252
Promberger, M., & Baron, J. (2006) · Journal of Behavioral Decision Making · doi:10.1002/bdm.542 · Cited by 242
Miller, T., Howe, P., & Sonenberg, L. (2017) · arXiv · doi:10.48550/arxiv.1712.00547 · Cited by 218
Efficient Search for Diverse Coherent Explanations
Russell, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287569 · Cited by 218
Toward establishing trust in adaptive agents
Glass, A., McGuinness, D. L., & Wolverton, M. J. (2008) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/1378773.1378804 · Cited by 217
Interacting meaningfully with machine learning systems: Three experiments
Stumpf, S., Rajaram, V., Li, L., Wong, W.-K., Burnett, M., Dietterich, T., Sullivan, E., & Herlocker, J. (2009) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2009.03.004 · Cited by 184
Communicating Algorithmic Process in Online Behavioral Advertising
Eslami, M., Krishna Kumaran, S. R., Sandvig, C., & Karahalios, K. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174006 · Cited by 171
Goldstein, D. G., & Gigerenzer, G. (2009) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2009.05.010 · Cited by 170
Self-citation(2015) · International Conference on Artificial Intelligence and Statistics · Cited by 162
Persuasiveness of expert systems
Dijkstra, J. J., Liebrand, W. B. G., & Timminga, E. (1998) · Behaviour & Information Technology · doi:10.1080/014492998119526 · Cited by 161
Åstebro, T. B., & Elhedhli, S. (2006) · Management Science · doi:10.1287/mnsc.1050.0468 · Cited by 148
Introduction to this Special Issue on Sensemaking
Pirolli, P. L. T., & Russell, D. M. (2011) · Human-Computer Interaction · doi:10.1080/07370024.2011.556557 · Cited by 139
Towards Algorithmic Experience
Alvarado, O., & Wærn, A. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173860 · Cited by 139
Tan, S., Caruana, R., Hooker, G., & Lou, Y. (2018) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3278721.3278725 · Cited by 138
Interpretable to Whom? A Role-based Model for Analyzing Interpretable Machine Learning Systems
Tomsett, R., Braines, D., Harborne, D., Preece, A. D., & Chakraborty, S. (2018) · ORCA Online Research @Cardiff (Cardiff University) · doi:10.48550/arxiv.1806.07552 · Cited by 127
User agreement with incorrect expert system advice
Dijkstra, J. J. (1999) · Behaviour and Information Technology · doi:10.1080/014492999118832 · Cited by 124
Liem, C. C. S., Langer, M., Demetriou, A. M., Hiemstra, A. M. F., Wicaksana, A. S., Born, M. P., & König, C. J. (2018) · The Springer series on challenges in machine learning · doi:10.1007/978-3-319-98131-4_9 · Cited by 116
Optimized Scoring Systems: Toward Trust in Machine Learning for Healthcare and Criminal Justice
Rudin, C. D., & Ustun, B. (2018) · INFORMS Journal on Applied Analytics · doi:10.1287/inte.2018.0957 · Cited by 114
Human Evaluation of Models Built for Interpretability
Lage, I., Chen, E., He, J., Narayanan, M., Kim, B., Gershman, S. J., & Doshi‐Velez, F. (2019) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v7i1.5280 · Cited by 114
Human Decision Making with Machine Assistance: An Experiment on Bailing and Jailing
Grgić-Hlača, N., Engel, C., & Gummadi, K. P. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359280 · Cited by 113
Simple Rules for Complex DecisionsPreprint
Jung, J., Concannon, C., Shroff, R., Goel, S., & Goldstein, D. G. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2919024 · Cited by 110
Self-citationKim, Y., Reinecke, K., & Hullman, J. R. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025592 · Cited by 105
Norms of Distributive Justice in Interest Arbitration
Bazerman, M. H. (1985) · Industrial and Labor Relations Review · doi:10.1177/001979398503800403 · Cited by 101
A Semantic Approach for Estimating Consumer Content Preferences from Online Search Queries
Liu, J., & Toubia, O. (2018) · Marketing Science · doi:10.1287/mksc.2018.1112 · Cited by 101
COGAM: Measuring and Moderating Cognitive Load in Machine Learning Model Explanations
Abdul, A., von der Weth, C., Kankanhalli, M., & Lim, B. Y. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376615 · Cited by 96
Human-in-the-Loop Interpretability Prior
Lage, I., Ross, A. S., Kim, B., Gershman, S. J., & Doshi‐Velez, F. (2018) · PubMed · doi:10.57702/j7gpfk6v · Cited by 90
Assessing long‐term fire risk at local scale by means of decision tree technique
Amatulli, G., Rodrigues, M. J., Trombetti, M., & Lovreglio, R. (2006) · Journal of Geophysical Research Atmospheres · doi:10.1029/2005jg000133 · Cited by 89
A Workflow for Visual Diagnostics of Binary Classifiers using Instance-Level Explanations
Krause, J., Dasgupta, A., Swartz, J. L., Aphinyanaphongs, Y., & Bertini, E. (2017) · IEEE Conference on Visual Analytics Science and Technology (VAST) · doi:10.1109/vast.2017.8585720 · Cited by 87
Design of an intelligible mobile context-aware application
Lim, B. Y., & Dey, A. (2011) · Mobile HCI · doi:10.1145/2037373.2037399 · Cited by 72
The Illusion of Control: Placebo Effects of Control Settings
Vaccaro, K., Huang, D., Eslami, M., Sandvig, C., Hamilton, K., & Karahalios, K. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173590 · Cited by 65
Algorithm Appreciation: People Prefer Algorithmic To Human JudgmentPreprint
Logg, J. M., Minson, J., & Moore, D. A. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2941774 · Cited by 59
Visualizable and interpretable regression models with good prediction power
Kim, H., Loh, W., Shih, Y., & Chaudhuri, P. (2007) · IIE Transactions · doi:10.1080/07408170600897502 · Cited by 52
Effects of Sensemaking Translucence on Distributed Collaborative Analysis
Goyal, N., & Fussell, S. R. (2016) · ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW) · doi:10.1145/2818048.2820071 · Cited by 52
A Mathematical Model for Interpretable Clinical Decision Support with Applications in Gynecology
Belle, V. V., CALSTER, B. V., Timmerman, D., Bourne, T., Bottomley, C., Valentin, L., Neven, P., Huffel, S. V., Suykens, J. A. K., & Boyd, S. (2012) · PLoS ONE · doi:10.1371/journal.pone.0034312 · Cited by 42
Kleinberg, J. M., & Mullainathan, S. (2019) · ACM Conference on Economics and Computation (EC) · doi:10.1145/3328526.3329621 · Cited by 37
Effects of Sensemaking Translucence on Distributed Collaborative AnalysisPreprint
Goyal, N., & Fussell, S. R. (2016) · Cited by 35
The Earth is spherical (p < 0.05): alternative methods of statistical inference
Vicente, K. J., & Torenvliet, G. (2000) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220110037065 · Cited by 28
Interpretable & Explorable Approximations of Black Box Models
Lakkaraju, H., Kamar, E., Caruana, R., & Leskovec, J. (2017) · arXiv · doi:10.48550/arxiv.1707.01154 · Cited by 23
Using Human and Machine Processing in Recommendation Systems
Colson, E. (2013) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v1i1.13100 · Cited by 10
Research works printed but not linked: 10
- Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine bias: There’s software used across the country to predict future criminals. And it’s biased against blacks. ProPublica (2016).
- 10.2139/ssrn.3465622 · DOI · Paper page
- Philip Johnson-Laird. 1983. Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness. Cambridge University Press.
- Jongbin Jung, Connor Concannon, Ravi Shroff, Sharad Goel, and Daniel G Goldstein. 2020. Simple rules to guide expert classifications. Journal of the Royal Statistical Society: Series A (Statistics in Society) (2020).
- 10.5555/3305381.3305576 · DOI
- Zachary C Lipton. 2016. The mythos of model interpretability. arXiv preprint arXiv:1606.03490(2016). · arXiv
- Jennifer M. Logg. 2017. Theory of Machine: When Do People Rely on Algorithms? (2017). Harvard Business School NOM Unit Working Paper No. 17-086.
- 10.18574/nyu/9781479833641.001.0001 · DOI
- Don A. Norman. 1987. Some Observations on Mental Models. In Human-Computer Interaction: A Multidisciplinary Approach, R. M. Baecker and W. A. S. Buxton (Eds.). Morgan Kaufmann Publishers Inc., 241–244.
- Christian Rudder. 2014. Dataclysm: Love, Sex, Race, and Identity–What Our Online Lives Tell Us about Our Offline Selves. Crown.
Cited by
Sorted by citations
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
The false hope of current approaches to explainable artificial intelligence in health care
Ghassemi, M., Oakden-Rayner, L., & Beam, A. L. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00208-9 · Cited by 1,724
Brynjolfsson, E., Li, D., & Raymond, L. (2025) · The Quarterly Journal of Economics · doi:10.1093/qje/qjae044 · Cited by 1,083
A Survey on the Explainability of Supervised Machine Learning
Burkart, N., & Huber, M. F. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12228 · Cited by 1,008
Interpretable machine learning: Fundamental principles and 10 grand challenges
Rudin, C., Chen, C., Chen, Z., Huang, H., Semenova, L., & Zhong, C. (2022) · Statistics Surveys · doi:10.1214/21-ss133 · Cited by 958
Metadata
- Authors
- Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H.
- Affiliations
- Microsoft (United States) (US)
- Microsoft Research, United States
- Venue
- Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2021-05-06
- Pages
- 1-52
- OpenAlex
- W2788362053
- Semantic Scholar
- 2f4e42ab1368b83e38e3011fcec4edb57267b5d2
- Cited by
- 728
- References
- 116
- Citations by year
- 2026: 94 · 2025: 97 · 2024: 118 · 2023: 106 · 2022: 111 · 2021: 90
- FWCI
- 48.67
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- model interpretability, explainable AI, black-box models, model transparency, feature count, error detection, information overload, trust in machine learning, pre-registered experiment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
