Explainable artificial intelligence (XAI) post-hoc explainability methods: Risks and limitations in non-discrimination law
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Local methods validate the model's actions for a specific circumstance or cluster of occurrences.
“Local methods make a concerted effort to validate the model's actions for a specific circumstance or cluster of occurrences [54], [59].”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [54]
There are a myriad ways to investigate global interpretability.
“There are a myriad ways to investigate global interpretability [1], [54].”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [54]
Global interpretability illustrates a model's overarching rationale and the justification for all probable consequences.
“Global interpretability illustrates a model's overarching rationale and the justification for all probable consequences [54].”
From A Review of Trustworthy and Explainable Artificial Intelligence (XAI) (2023) · cites this paper as [54]
Post-hoc models need to be explained by resorting to diverse means to enhance their interpretability, such as text explanations, visual explanations, local explanations, explanations by example, explanations by simplification and feature relevance explanations techniques.
“The post-hoc models need to explained by resorting to diverse means to enhance their interpretability, such as text explanations, visual explanations, local explanations, explanations by example, explanations by simplification and feature relevance explanations techniques.”
From Explainable Artificial Intelligence (XAI): Concepts and Challenges in Healthcare (2023) · cites this paper as [22]
Arrieta et al. classified studies on XAI into two approaches: transparent models and post-hoc models.
“Arrieta et al. [22] classified studies on XAI into two approaches: some works focus on creating transparent models, while most works wrap black-box models with a layer of explainability, the so-called post-hoc models.”
From Explainable Artificial Intelligence (XAI): Concepts and Challenges in Healthcare (2023) · cites this paper as [22]
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · see citing works
From local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551 · see citing works
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · see citing works
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · see citing works
Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)
Breiman, L. (2001) · Statistical Science · doi:10.1214/ss/1009213726 · Cited by 4,340 · see citing works
The Black Box Society: The Secret Algorithms That Control Money and Information
Pasquale, F. (2015) · doi:10.4159/harvard.9780674736061 · Cited by 3,796 · see citing works
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · see citing works
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 1× · see citing works
“…can be intentionally obstructed to protect (trade) secrets, to protect against gaming the sys-tem, and maintain a competitive advantage [11], black-box models’ opacity can arise from the distinct difficulty of interpreting their classification results, which leverage large data-sets…”
BackgroundGPT-3: Its Nature, Scope, Limits, and Consequences
Floridi, L., & Chiriatti, M. (2020) · Minds and Machines · doi:10.1007/s11023-020-09548-1 · Cited by 2,519 · see citing works
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507 · see citing works
Definitions, methods, and applications in interpretable machine learning
Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R., & Yu, B. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1900654116 · Cited by 2,167 · cited 1× · see citing works
“Post-hoc explainability takes a trained model as input and extracts the underlying relationships that the model had learned by querying the model [47] and constructing a white-box surrogate model [10].”
MethodMachine Learning Interpretability: A Survey on Methods and Metrics
Carvalho, D. V., Pereira, E. M., & Cardoso, J. S. (2019) · Electronics · doi:10.3390/electronics8080832 · Cited by 1,842 · see citing works
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827 · see citing works
Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3060483 · Cited by 1,477 · cited 12× · see citing works
“The instability of some post-hoc explainability methods such as LIME further calls into question their use.”
BackgroundInfluentialExplaining nonlinear classification decisions with deep taylor decomposition
Montavon, G., Lapuschkin, S., Binder, A., Samek, W., & Müller, K.-R. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.11.008 · Cited by 1,357 · see citing works
Techniques for interpretable machine learning
Du, M., Liu, N., & Hu, X. (2019) · Communications of the ACM · doi:10.1145/3359786 · Cited by 1,268 · see citing works
Fairness definitions explained
Verma, S., & Rubin, J. (2018) · Proceedings of the International Workshop on Software Fairness · doi:10.1145/3194770.3194776 · Cited by 1,124 · see citing works
A Survey on the Explainability of Supervised Machine Learning
Burkart, N., & Huber, M. F. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12228 · Cited by 1,008 · see citing works
Classifier Technology and the Illusion of Progress
Hand, D. J. (2006) · Statistical Science · doi:10.1214/088342306000000060 · Cited by 723 · see citing works
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · see citing works
Interpretable Machine Learning in Healthcare
Ahmad, M. A., Eckert, C., & Teredesai, A. (2018) · Proceedings of the 2018 ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics · doi:10.1145/3233547.3233667 · Cited by 583 · see citing works
Why Fairness Cannot Be Automated: Bridging the Gap Between EU Non-Discrimination Law and AIPreprint
Wachter, S., Mittelstadt, B., & Russell, C. (2020) · Computer Law and Security Review · doi:10.2139/ssrn.3547922 · Cited by 448 · see citing works
O'Sullivan, S., Nevejans, N., Allen, C., Blyth, A., Leonard, S., Pagallo, U., Holzinger, K., Holzinger, A., Sajid, M. I., & Ashrafian, H. (2019) · The International Journal of Medical Robotics and Computer Assisted Surgery · doi:10.1002/rcs.1968 · Cited by 437 · see citing works
The Intuitive Appeal of Explainable Machines
Selbst, A. D., & Barocas, S. (2018) · doi:10.2139/ssrn.3126971 · Cited by 392 · see citing works
Machine learning, medical diagnosis, and biomedical engineering research - commentary
Foster, K. R., Koprowski, R., & Skufca, J. D. (2014) · BioMedical Engineering OnLine · doi:10.1186/1475-925x-13-94 · Cited by 384 · see citing works
Strengthening legal protection against discrimination by algorithms and artificial intelligence
Zuiderveen Borgesius, F. J. (2020) · The International Journal of Human Rights · doi:10.1080/13642987.2020.1743976 · Cited by 267 · see citing works
Statistical stability indices for LIME: Obtaining reliable explanations for machine learning models
Visani, G., Bagli, E., Chesani, F., Poluzzi, A., & Capuzzo, D. (2021) · Journal of the Operational Research Society · doi:10.1080/01605682.2020.1865846 · Cited by 243 · see citing works
Meaningful Explanations of Black Box AI Decision Systems
Pedreschi, D., Giannotti, F., Guidotti, R., Monreale, A., Ruggieri, S., & Turini, F. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019780 · Cited by 233 · see citing works
Disease inference from health-related questions via sparse deep learning
Nie, L., Wang, M., Zhang, L., Yan, S., Zhang, B., & Chua, T.-S. (2015) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2015.2399298 · Cited by 195 · see citing works
The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanations
Laugel, T., Lesot, M.-J., Marsala, C., Renard, X., & Detyniecki, M. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/388 · Cited by 194 · see citing works
Onishi, T., Saha, S. K., Delgado-Montero, A., Ludwig, D. R., Onishi, T., Schelbert, E. B., SCHWARTZMAN, D. S., & Gorcsan, J. (2015) · Journal of the American Society of Echocardiography · doi:10.1016/j.echo.2014.11.018 · Cited by 172 · see citing works
Perturbation-Based Explanations of Prediction Models
Robnik-Šikonja, M., & Bohanec, M. (2018) · Human–Computer Interaction Series · doi:10.1007/978-3-319-90403-0_9 · Cited by 171 · see citing works
Wachter, S., Mittelstadt, B., & Russell, C. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3792772 · Cited by 167 · see citing works
AI and Law: A fruitful synergy
Rissland, E. L., Ashley, K. D., & Loui, R. P. (2003) · Artificial Intelligence · doi:10.1016/s0004-3702(03)00122-x · Cited by 136 · see citing works
Tan, S., Caruana, R., Hooker, G., & Lou, Y. (2018) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3278721.3278725 · Cited by 127 · see citing works
The European Union and Human Rights after the Treaty of Lisbon
Douglas‐Scott, S. (2011) · Human Rights Law Review · doi:10.1093/hrlr/ngr038 · Cited by 119 · see citing works
A Relational Theory of Data Governance
Viljoen, S. (2020) · doi:10.2139/ssrn.3727562 · Cited by 109 · see citing works
VanderWeele, T. J., & Hernán, M. Á. (2012) · American Journal of Epidemiology · doi:10.1093/aje/kwr458 · Cited by 107 · see citing works
Regulating Artificial Intelligence
Wischmeyer, T., & Rademacher, T. (2019) · Springer eBooks · doi:10.1007/978-3-030-32361-5 · Cited by 100 · see citing works
Meske, C., & Bunde, E. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-50334-5_4 · Cited by 99 · see citing works
Artificial Intelligence and Transparency: Opening the Black Box
Wischmeyer, T. (2019) · Springer eBooks · doi:10.1007/978-3-030-32361-5_4 · Cited by 94 · see citing works
Machine Learning: Between Accuracy and Interpretability
Bratko, I. (1997) · Learning, Networks and Statistics · doi:10.1007/978-3-7091-2668-4_10 · Cited by 67 · see citing works
Equality in the making: implementing European non-discrimination law
Guiraudon, V. (2009) · Citizenship Studies · doi:10.1080/13621020903174696 · Cited by 56 · see citing works
Learning Tasks for Multitask Learning: Heterogenous Patient Populations in the ICU
Suresh, H., Gong, J. J., & Guttag, J. V. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219930 · Cited by 54 · see citing works
Handbook on European non-discrimination law
(2019) · Human Rights Documents online · doi:10.1163/2210-7975_hrd-9992-20180028 · Cited by 38 · see citing works
Direct Discrimination, Indirect Discrimination and Autonomy
Doyle, O. (2007) · Oxford Journal of Legal Studies · doi:10.1093/ojls/gqm008 · Cited by 34 · see citing works
Artificial Intelligence and Discrimination: Discriminating Against Discriminatory Systems
Tischbirek, A. (2019) · Springer eBooks · doi:10.1007/978-3-030-32361-5_5 · Cited by 26 · see citing works
XNLP: A Living Survey for XAI Research in Natural Language Processing
Qian, K., Danilevsky, M., Katsis, Y., Kawas, B., Oduor, E., Popa, L. C., & Li, Y. (2021) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3397482.3450728 · Cited by 21 · see citing works
Artificial Intelligence and Autonomy: Self-Determination in the Age of Automated Systems
Ernst, C. (2019) · Springer eBooks · doi:10.1007/978-3-030-32361-5_3 · Cited by 19 · see citing works
The European Union: Initiator of a New European Anti-Discrimination Regime?
Kantola, J., & Nousiainen, K. (2012) · Palgrave Macmillan UK eBooks · doi:10.1057/9781137031068_2 · Cited by 15 · see citing works
Key concepts in EU anti-discrimination law
Ellis, E., & Watson, P. (2012) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199698462.003.0004 · Cited by 4 · see citing works
Direct Discrimination: Limited by Definition?
Mair, J. (2009) · International Journal of Discrimination and the Law · doi:10.1177/135822910901000102 · Cited by 3 · see citing works
AI U.S. Policies and Regulations
Girasa, R. (2020) · Springer eBooks · doi:10.1007/978-3-030-35975-1_3 · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Review of Trustworthy and Explainable Artificial Intelligence (XAI)
Chamola, V., Hassija, V., Sulthana, A. R., Ghosh, D., Dhingra, D., & Sikdar, B. (2023) · IEEE Access · doi:10.1109/access.2023.3294569 · Cited by 352 · see citing works
“Global interpretability illustrates a model’s overarching rationale and the justification for all probable consequences [54].”
BackgroundExplainable Artificial Intelligence (XAI): Concepts and Challenges in Healthcare
Hulsen, T. (2023) · AI · doi:10.3390/ai4030034 · Cited by 305 · see citing works
“[22] state that machine learning posthoc explanation methods cannot guarantee the insights they generate, which means that they cannot be relied upon as the only mechanism to guarantee the fairness of model outcomes in high-stake decision-making, such as in healthcare.”
BackgroundA literature review on applications of explainable artificial intelligence (XAI)
Kalasampath, K., Spoorthi, K. N., Sajeev, S., Kuppa, S. S., Ajay, K., & Maruthamuthu, A. (2025) · IEEE Access · doi:10.1109/access.2025.3546681 · Cited by 180 · see citing works
Mathew, D. E., Ebem, D. U., Ikegwu, A. C., Ukeoma, P. E., & Dibiaezue, N. F. (2025) · Neural Processing Letters · doi:10.1007/s11063-025-11732-2 · Cited by 168 · see citing works
Explainable artificial intelligence for drug discovery and development: a comprehensive survey
Alizadehsani, R., Oyelere, S. S., Hussain, S., Jagatheesaperumal, S. K., Calixto, R. R., Rahouti, M., Roshanzamir, M., & De Albuquerque, V. H. C. (2024) · IEEE Access · doi:10.1109/access.2024.3373195 · Cited by 124 · see citing works
Haitsma, L. M. (2025) · European Journal of Social Security · doi:10.1177/13882627251351093 · Cited by 4 · see citing works
“…for addressing discrimination in algorithmic pro fi ling are being developed, further research is still needed to examine their utility and effectiveness (Boonstra et al., 2024; Franzke et al., 2021; Meuwese et al., 2024; Tilburg University, 2021; Vale et al., 2022; Vetter et al., 2023).”
- Authors
- Vale, D., El-Sharif, A., & Ali, M.
- Affiliations
- Leiden University (NL)
- Nova Southeastern University (US)
- University College London (GB)
- Corresponding author
- Vale, Daniel
- Venue
- AI and Ethics
- Published
- 2022-03-15
- Volume
- 2
- Issue
- 4
- Pages
- 815-826
- ISSN
- 2730-5953 (ISSN-L)
- 2730-5961
- OpenAlex
- W4220875808
- Cited by
- 124
- References
- 55
- Citations by year
- 2026: 24 · 2025: 41 · 2024: 30 · 2023: 22 · 2022: 7
- FWCI
- 8.23
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- AI and Ethics (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Keywords
- explainable AI, post-hoc explanation methods, anti-discrimination law, algorithmic discrimination, algorithmic fairness, black-box machine learning, algorithmic decision-making, EU anti-discrimination law, high-stakes decision making, model opacity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
