CoChat
Sign Up
Cited by 103Explainable Artificial Intelligence (XAI)Ethics and Social Impacts of AIE-Government and Public Services

It’s Just Not That Simple: An Empirical Study of the Accuracy-Explainability Trade-off in Machine Learning for Public Policy

Open access
Bell, Andrew, Solano-Kamaiko, Ian, Nov, Oded, Stoyanovich, Julia·2022·2022 ACM Conference on Fairness Accountability and Transparency
DOI 10.1145/3531146.3533090
Open this paper in CoChat — free accountOpen PDF

Abstract

To achieve high accuracy in machine learning (ML) systems, practitioners often use complex “black-box” models that are not easily understood by humans. The opacity of such models has resulted in public concerns about their use in high-stakes contexts and given rise to two conflicting arguments about the nature — and even the existence — of the accuracy-explainability trade-off. One side postulates that model accuracy and explainability are inversely related, leading practitioners to use black-box models when high accuracy is important. The other side of this argument holds that the accuracy-explainability trade-off is rarely observed in practice and consequently, that simpler interpretable models should always be preferred. Both sides of the argument operate under the assumption that some types of models, such as low-depth decision trees and linear regression are more explainable, while others such as neural networks and random forests, are inherently opaque.

AI snapshot

Written from the paper’s full text.

TL;DR

In two public-policy user studies, the paper finds no statistically significant difference in explainability between black-box and interpretable models for human-in-the-loop users, and black-box models were sometimes both more accurate and more explainable. The accuracy-explainability trade-off is metric- and context-dependent rather than a simple inverse relationship.

Key findings

  • No statistically significant difference in explainability between black-box and interpretable models across the four explainability metrics; pairwise model-type differences existed. For example, ASO task p=0.296 in both domains, IMIF task p=0.087 (education) and p=0.108 (housing). Interpretable systems were perceived as more confusing than black-box systems (System Confusing p=0.004 education, p<0.000 housing); 33.4% overall reported confusion (35.0% education, 31.9% housing) at score ≥4.0.
  • Accuracy: in both use-cases, the difference between the most accurate black-box and most accurate interpretable model was less than 0.01 on accuracy score. In housing, the best black-box model outperformed the top interpretable model by 7% on precision@25% (0.85 vs. 0.78). Education best black-box XGBoost: accuracy 0.89, precision@25% 0.77; best interpretable logistic regression: accuracy 0.88, precision@25% 0.77.
  • Two drivers: additional information may confuse users, and intrinsic explainability mechanisms have weaknesses. Decision trees had the lowest IMIF performance; participants often defaulted to the root or a terminal node. Linear regression’s linear formula performed poorly on the local ASO task due to a scope mismatch.
  • In some cases there was no trade-off: with precision@25% vs. ASO, black-box models were both more accurate and more explainable; with accuracy score vs. IMIF, black-box and interpretable models performed similarly.

Methods

  • Two public-policy use-cases: Education (Portuguese student performance; 1,044 records, 33 features; predict failure grade ≤10) and Housing (King County, WA; 21,613 records, 20 features; predict sale price ≥ $645,000). Per domain, trained 8 classifiers: 3 black-box (XGBoost, extra trees, random forests), 4 interpretable (decision tree, Ridge, Lasso, logistic regression), and 1 dummy baseline; tuned via hyperparameter sweep, validated with stratified k-fold cross-validation, and evaluated on 7 metrics including accuracy and precision@25%. Selected random forest, linear regression (Ridge for Education, Lasso for Housing), and decision tree for the survey.
  • IRB-approved user study via Prolific: 168 participants per domain, pre-screened as US residents, over 18, English-speaking; education-sector participants for Education and real-estate participants for Housing. Each participant viewed one black-box and one interpretable model in random order, completed the ASO and IMIF tasks, and answered an 8-item Likert questionnaire (5-item Understood System, 3-item System Confusing). Analysis used McNemar’s test and random effects models.

Limitations

  • Stated by the authors: proxy tasks may not accurately measure explainability in real-world decision-making; the lab environment has no real stakes; only two use-cases are included; qualitative research is absent, leaving open how explainability interacts with users’ mental models.
  • Evident from the text: the Education dataset had data leakage because absences and failures were aggregated across all three years, including the prediction year, and could not be disentangled; this improves accuracy metrics but applies to both model types. Housing dropped 10 records and omitted zip codes. The IMIF task design may not be robust because some users confused local and global feature importance, likely making IMIF results overly pessimistic.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3531146.3533090

Cites

55 of 55 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 38 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Model accuracy and explainability are not necessarily inversely related.

    “Finally, we find that model accuracy and explainability are not necessarily inversely related.”
    Quote in the paperSubstantial claim
  • Findingnone given

    Local explanations like SHAP are most useful to users when they are significantly different from global explanations about a model.

    “Local explanations, like those from SHAP, are most useful to users in cases where these explanations are significantly different from global explanations about a model (e.g., feature importance).”
    Quote in the paperNot a substantial claim
  • Findingown data

    There were significant pairwise differences in ASO performance across specific model types (decision tree, linear regression, black-box no SHAP, black-box SHAP).

    “However, there were significant pairwise differences when comparing ASO performance for specific model types (decision tree, linear regression, black-box (no SHAP), black-box (SHAP); see Figures 2, 3).”
    Quote in the paperSubstantial claim
  • Findingown data

    Participants were most successful at completing the ASO task when presented with the black-box model.

    “We found that participants were most successful at completing the ASO task when presented with the black-box model, as seen in Figure 2 (a).”
    Quote in the paperSubstantial claim
  • Findingown data

    Providing more information to a HITL was only useful if it did not confuse the user.

    “Providing more information to a HITL (e.g., SHAP explanations for black-box models or “intrinsic” explainability mechanisms such as a tree diagram for decision trees) was only useful if it did not confuse the user.”
    Quote in the paperSubstantial claim
  • May combine two claims. 17
  • Classified as probably not a substantial claim. 22

  • May combine two claims.

    Findingown data

    Black-box models, both with and without SHAP explanations, lead to the best performance by users on both objectively-quantifiable tasks in both policy domains.

    “In fact, black-box models, both with and without SHAP explanations, lead to the best performance by users on both objectively-quantifiable tasks in both policy domains.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingreasoning

    In our user study, the inherent opacity of black-box models meant that less information about the model was displayed.

    “In our user study, the inherent opacity of black-box models meant that less information about the model was displayed.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The distribution of participants' ratings for the System Confusing construct in the education domain when presented with the black-box model with SHAP is bimodal, suggesting two separate subgroups: those who reported being confused and those who did not.

    “The distribution is bimodal, suggesting that there are two separate subgroups: those who reported being confused by the system and those who did not.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Using the measures in the study, black-box models are both the most explainable and the most accurate.

    “Using these measures, black-box models are both the most explainable and the most accurate.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    For both education and housing, model accuracy and ability of HITL users to identify the top feature was the same, regardless of whether linear regression or a black-box with SHAP was used.

    “For both education and housing, model accuracy and ability of HITL users to identify the top feature was the same, regardless of whether linear regression or a black-box with SHAP was used.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Prior research found that the understanding of a model is negatively correlated with its complexity and that decision trees are among the model types best understood by users.

    “Prior research found that the understanding of a model is negatively correlated with its complexity and that decision trees are among the model types best understood by users [4].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [4]

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    The data leakage in the Education dataset poses no harm to the overall study objective because the accuracy improvement applies to both interpretable and black-box models.

    “However, since this accuracy improvement applies to both interpretable and black-box models, we believe it poses no harm to our overall study objective.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Blindly deploying post-hoc explainability techniques can hurt users' performance on certain tasks related to their understanding of ML models.

    “researchers have found that blindly deploying post-hoc explainability techniques can actually hurt users' performance on certain tasks related to their understanding of ML models [39].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [39] source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    Classifiers were tuned using a parameter sweep on a large hyperparameter grid, and performance was validated using stratified k-fold cross-validation.

    “The classifiers were tuned using a parameter sweep on a large hyperparameter grid and the performance of each model was validated using stratified k-fold cross-validation.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The distinction between interpretable and black-box models lacks attention to model complexity.

    “Guidotti et al. [31] point out that the distinction between interpretable and black-box models lacks an attention to model complexity.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [31] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The seven evaluation metrics reflect the performance measures typically used in ML for public policy.

    “These metrics were chosen because they reflect the performance measures typically used in ML for public policy [9,12,20,56].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [9]

      Not checked in the free run.

    • cites [12] source

      Not checked in the free run.

    • cites [20]

      Not checked in the free run.

    • cites [56]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Other model types, like random forests and neural networks, are considered blackboxes.

    “Other model types, like random forests and neural networks, are considered blackboxes.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [31] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    ML practitioners generally accept rules-based models, decision trees, and linear models as interpretable.

    “ML practitioners generally accept three types of models as interpretable: rules-based models, decision trees, and linear models [31].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [31] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The human subjects research study was IRB-approved.

    “we conducted an IRB-approved human subjects research study.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Using zip codes, which are strongly correlated with race, to predict housing prices for tax purposes raises concerns about redlining, which the Fair Housing Act of 1968 outlawed.

    “Furthermore, using zip codes -which are strongly correlated with race [21,41] -to predict housing prices for tax purposes raise concerns about redlining, which the Fair Housing Act of 1968 explicitly outlawed as a practice in the United States [14].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [21] source

      Not checked in the free run.

    • cites [41]

      Not checked in the free run.

    • cites [14]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Providing more information in ML model explanations can decrease users' perceived understandings due to information overload.

    “research shows that providing more information in ML model explanations can actually decrease users' perceived understandings due to information overload [42,49].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [42] source

      Not checked in the free run.

    • cites [49]

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    While explainability for the overarching category of black-box or interpretable did not differ, individual model types did matter.

    “Therefore, while the explainability for the overarching category of black-box or interpretable did not differ, the individual model types did matter.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    To ensure robustness, models were validated over a range of random seeds and different validation methods (classic k-fold and train-test split).

    “To ensure the robustness of our results, the models were also validated over a range of different random seeds and different validation methods such as classic k-fold and train-testsplit.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Each participant interacts with one black-box and one interpretable model, in a random order.

    “To summarize our design: Each participant interacts with one black-box and one interpretable model, in a random order.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The participants were randomly presented one of the four ML systems being studied (black-box no SHAP, black-box with SHAP, linear regression, or decision tree).

    “The participants were randomly presented one of the four ML systems being studied (black-box no SHAP, black-box with SHAP, linear regression, or decision tree)”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    The IMIF task measured the global explainability of the model since the model's feature importance did not vary with respect to each data point.

    “In contrast to the first task, this task measured the global explainability of the model since the model's feature importance did not vary with respect to each data point.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    A strength of SHAP is to assist in the local explainability of a model.

    “This makes intuitive sense, since a strength of SHAP is to assist in the local explainability of a model.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The questions in the system understanding section were adapted from work by Lim.

    “The questions in this section were adapted from work by Lim [43].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [43] source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductown data

    The methods developed in this paper have only been tested in a lab environment where there are no real stakes for users, which may impact their robustness or validity.

    “The methods developed in this paper have only been tested in a lab environment where there are no real stakes for users, which may impact their robustness or validity (e.g., the teachers from our survey did not have a connection with a real student for which the AI system was making a prediction).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The accuracy-explainability trade-off is more nuanced than initially anticipated, and in some cases there may be no trade-off at all.

    “Instead, we observed that the accuracy-explainability trade-off is more nuanced than initially anticipated-and in some cases, there may be no trade-off at all.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    The survey emulates how HITL users interact with ML systems in real-world policy settings.

    “We designed the survey to emulate how HITL users interact with ML systems in real-world policy settings [70].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [70] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Each study had 168 participants to ensure our survey would have statistical power.

    “Each study had 168 participants to ensure our survey would have statistical power.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Similar findings from XAI research have found that more explanations directly hurt participants' task performance.

    “This finding is in concert with similar findings from XAI research [42,49], some of which have found that more explanations directly hurt participants' task performance [39].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [42] source

      Not checked in the free run.

    • cites [49]

      Not checked in the free run.

    • cites [39] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    In addition to the black-box being more explainable, they were also more accurate.

    “In addition to the black-box being more explainable, they were also more accurate.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Study design and conductnone given

    By randomizing the order in which participants see each model, we eliminate any learning effect that may occur from interacting with the first model.

    “Note that by randomizing the order in which participants see each model, we eliminate any”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Participants were highly successful at the IMIF task when presented with the black-box with SHAP because local explainability aligned with the models' global explainability.

    “Additionally, participants were highly successful at the IMIF task when presented with the black-box with SHAP because local explainability aligned with the models' global explainability.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    This work only includes two use-cases - housing and education.

    “Second, this work only includes two use-cases -housing and education.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The extent to which a trade-off exists (when it exists) is dependent on the chosen accuracy metric, on how explainability is measured, and on the context of use.

    “Rather, practitioners should acknowledge that the extent to which a trade-off exists (when it exists) is dependent on the chosen accuracy metric, on how explainability is measured, and on the context of use.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Black-box models can actually be more explainable and less confusing to users, perhaps by avoiding overloading users with information.

    “We observed that black-box models can actually be more explainable and less confusing to users, perhaps by avoiding overloading users with information.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Black-box models may be both the most accurate and the most explainable.

    “As seen in Figures 4 (a) and (b), black-box models may be both the most accurate and the most explainable.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Explainability is the extent to which someone can accurately understand and predict the output of an ML model.

    “This task was adapted from Miller's work [47] where explainability is the extent to which someone can accurately understand and predict the output of an ML model.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [47]

      Not checked in the free run.

36 of 36

  • The paper relies on this without support.

    ML systems in public policy settings are almost always implemented alongside a human user who must understand the system sufficiently to take responsibility for the decisions.

    “Three such considerations are: (1) ML systems in public policy settings are almost always implemented alongside a human user who must understand the system sufficiently to take responsibility for the decisions”

  • The paper relies on this without support.

    The transparency of ML systems in public policy is often (and with increasing frequency) legally mandated due to their public and high-stakes nature.

    “(2) the transparency of these systems is often (and with increasing frequency) legally mandated due to their public and high-stakes nature”

  • The paper relies on this without support.

    Accuracy in public policy ML is typically measured relative to a resource constraint.

    “(3) accuracy is typically measured relative to a resource constraint (see section 4.2).”

  • The authors state this as an assumption.

    The dropped housing records were likely input errors and their removal would not have any significant effect on the overall model training and prediction outcomes.

    “We found it reasonable to assume that these were likely input errors and that their removal would not have any significant effect on the overall model training and prediction outcomes.”

  • The authors state this as an assumption.

    In both policy contexts, intervention resources were limited to 25% of the population.

    “In both policy contexts, we assumed that intervention resources were limited to 25% of the population (see Table 1).”

  • The paper relies on this without support.

    The model training process followed ML for public policy industry standards and best practices.

    “Our model training process followed ML for public policy industry standards and best practices.”

  • The paper relies on this without support.

    Precision@25% was the most policy-relevant metric.

    “since this was the most policy-relevant metric.”

  • The paper relies on this without support.

    An accuracy-explainability trade-off exists in ML public policy contexts.

    “The initial goal of this research was to quantify the accuracyexplainability trade-off in ML public policy contexts.”

  • The paper relies on this without support.

    Participants recruited from Prolific in the specified industries are representative of HITL users in public policy settings.

    “We recruited participants through Prolific, an online research recruitment platform.”
    “For the education domain, participants were pre-screened as being in the Education & Training employment-sector and the Primary/Secondary K-12 Education industry.For the housing study, participants were from the Real Estate Rental & Leasing industry.”
9 of 9

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Tasks that involve inherent uncertainty are often those where humans tend to rely on AI systems for advice, such as loan approval, recidivism prediction, house price estimation, and student admission.

    “Tasks that involve inherent uncertainty are often those where humans tend to rely on AI systems for advice, such as loan approval [27, 34, 124], recidivism prediction [31, 47, 86], house price estimation [2, 13, 25], and student admission [13, 24].”

    From Dealing with Uncertainty: Understanding the Impact of Prognostic Versus Diagnostic Tasks on Trust and Reliance in Human-AI Decision Making (2024) · cites this paper as [13]

    The Introduction describes humans using ML predictions to inform decisions, and §3 identifies house-price estimation as one such use case. But the paper’s education use case predicts students’ risk of failure to allocate tutoring, not student admission; it also does not characterize these tasks as ones involving inherent uncertainty. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • The claimed absence of explainability differences contradicts the subjective results

    Major

    The paper says there was no statistical difference across all four explainability metrics. Table 3 instead reports significant black-box-versus-interpretable differences for System Confusing in both domains and Understood System in housing.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 19

  1. D Aha and Dennis Kibler. 1988. Instance-based prediction of heart-disease presence with the Cleveland database. University of California 3, 1 (1988), 3–2.
  2. Kasun Amarasinghe Kit Rodolfa Hemank Lamba and Rayid Ghani. 2020. Explainable machine learning for public policy: Use cases gaps and research directions. arXiv preprint arXiv:2010.14374(2020). · arXiv
  3. Alexandra Chouldechova. 2016. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. arxiv:1610.07524 [stat.AP] · arXiv
  4. Ian Covert Scott M. Lundberg and Su-In Lee. 2020. Understanding Global Feature Contributions Through Additive Importance Measures. CoRR abs/2004.00668(2020). arXiv:2004.00668https://arxiv.org/abs/2004.00668 · arXiv
  5. Inigo Martinez de Troya, Ruqian Chen, Laura O Moraes, Pranjal Bajaj, Jordan Kupersmith, Rayid Ghani, Nuno B Brás, and Leid Zejnilovic. 2018. Predicting, explaining, and understanding risk of long-term unemployment. In NeurIPS Workshop on AI for Social Good.
  6. 10.1093/jamia · DOI
  7. Graham Dove, Martina Balestra, Devin Mann, and Oded Nov. 2020. Good for the Many or Best for the Few? A Dilemma in the Design of Algorithmic Advice. Proceedings of the ACM on Human-Computer Interaction 4, CSCW2(2020), 1–22.
  8. Andreas Fuster, Paul Goldsmith-Pinkham, Tarun Ramadorai, and Ansgar Walther. 2020. Predictably unequal? the effects of machine learning on credit markets. The Effects of Machine Learning on Credit Markets (October 1, 2020) (2020).
  9. Philip Gillingham. 2019. Can predictive algorithms assist decision-making in social work with children and families?Child abuse review 28, 2 (2019), 114–126.
  10. Michael Gleicher. 2016. A framework for considering comprehensibility in modeling. Big data 4, 2 (2016), 75–88.
  11. David Gunning, Mark Stefik, Jaesik Choi, Timothy Miller, Simone Stumpf, and Guang-Zhong Yang. 2019. XAI—Explainable artificial intelligence. Science Robotics 4, 37 (2019).
  12. Nicol Turner Lee. 2018. Detecting racial bias in algorithms and machine learning. Journal of Information, Communication and Ethics in Society (2018).
  13. Scott M. Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 4765–4774. https://proceedings.neurips.cc/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html
  14. Tim Miller. 2019. Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence 267 (2019), 1–38.
  15. Robert Ross. 2017. The impact of property tax appeals on vertical equity in Cook County, IL. Univerity of Chicago, Harris School of Public Policy Working Paper (2017).
  16. 10.1145/3375627.3375830 · DOI · Paper page
  17. Yiwei Yang, Eser Kandogan, Yunyao Li, Prithviraj Sen, and Walter S Lasecki. 2019. A study on interaction in human-in-the-loop machine learning for text analytics. In IUI Workshops.
  18. Leid Zejnilovic, Susana Lavado, Carlos Soares, Íñigo Martínez De Rituerto De Troya, Andrew Bell, and Rayid Ghani. 2021. Machine Learning Informed Decision-Making with Interpreted Model’s Outputs: A Field Intervention. In Academy of Management Proceedings, Vol. 2021. Academy of Management Briarcliff Manor, NY 10510, 15424.
  19. Yujia Zhang, Kuangyan Song, Yiming Sun, Sarah Tan, and Madeleine Udell. 2019. ” Why Should You Trust My Explanation?” Understanding Uncertainty in LIME Explanations. arXiv preprint arXiv:1904.12991(2019). · arXiv

Cited by

Sorted by citations

Related

Authors
Bell, A., Solano-Kamaiko, I., Nov, O., & Stoyanovich, J.
Affiliations
New York University (US)
ORCID
Venue
2022 ACM Conference on Fairness Accountability and Transparency
Publisher
ACM
Published
2022-06-20
Pages
248-266
DOI
10.1145/3531146.3533090
OpenAlex
W4283157638
Cited by
103
References
55
Citations by year
2026: 21 · 2025: 30 · 2024: 29 · 2023: 20 · 2022: 3
FWCI
11.31
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
Funders
National Science Foundation (1916505, 1934464, 1928614, 2129076)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Political Science and International Relations › E-Government and Public Services
Keywords
explainable machine learning, black-box models, interpretable models, model opacity, public policy, algorithmic transparency, high-stakes decision making, model complexity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review