Survey on Explainable AI: Techniques, challenges and open issues

Greedy function approximation: A gradient boosting machine

Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731 · see citing works

A Unified Approach to Interpreting Model Predictions

Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · see citing works

Induction of decision trees

Quinlan, J. R. (1986) · Machine Learning · doi:10.1007/bf00116251 · Cited by 21,439 · see citing works

"Why Should I Trust You?": Explaining the Predictions of Any Classifier

Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works

A fast learning algorithm for deep belief nets

Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336 · see citing works

Distilling the Knowledge in a Neural Network

Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works

On a Method to Measure Supervised Multiclass Model’s Interpretability: Application to Degradation Diagnosis (Short Paper)

Gauriat, C.-M., Pencolé, Y., Ribot, P., et al. (2024) · HAL · doi:10.4230/oasics.dx.2024.27 · Cited by 12,623 · see citing works

Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead

Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · see citing works

From local explanations to global understanding with explainable AI for trees

Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551 · see citing works

Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI

Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · see citing works

Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)

Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · see citing works

The mythos of model interpretability

Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · see citing works

On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation

Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524 · see citing works

Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion

(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1953039 · Cited by 4,051 · see citing works

Purposeful selection of variables in logistic regression

Bursac, Z., Gauss, C. H., Williams, D. K., & Hosmer, D. W. (2008) · Source Code for Biology and Medicine · doi:10.1186/1751-0473-3-17 · Cited by 4,035 · see citing works

Towards A Rigorous Science of Interpretable Machine Learning

Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · see citing works

Deep learning for healthcare: review, opportunities and challenges

Miotto, R., Wang, F., Wang, S., Jiang, X., & Dudley, J. T. (2018) · Briefings in Bioinformatics · doi:10.1093/bib/bbx044 · Cited by 3,076 · see citing works

Explainable AI: a review of machine learning interpretability methods

Linardatos, P., Papastefanopoulos, V., & Kotsiantis, S. (2020) · Entropy · doi:10.3390/e23010018 · Cited by 2,922 · see citing works

2011 International Conference on Computer Vision

(2011) · doi:10.1109/iccv17851.2011 · Cited by 2,895 · see citing works

An Introduction to Logistic Regression Analysis and Reporting

Peng, C.-Y. J., Lee, K. L., & Ingersoll, G. M. (2002) · The Journal of Educational Research · doi:10.1080/00220670209598786 · Cited by 2,671 · see citing works

Anchors: High-Precision Model-Agnostic Explanations

Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · see citing works

XAI—Explainable artificial intelligence

Gunning, D., Stefik, M., Choi, J., Miller, T., Stumpf, S., & Yang, G.-Z. (2019) · Science Robotics · doi:10.1126/scirobotics.aay7120 · Cited by 2,053 · see citing works

Peeking Inside the Black Box: Visualizing Statistical Learning With Plots of Individual Conditional Expectation

Goldstein, A., Kapelner, A., Bleich, J., & Pitkin, E. (2015) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2014.907095 · Cited by 1,861 · see citing works

Understanding deep image representations by inverting them

Mahendran, A., & Vedaldi, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299155 · Cited by 1,827 · see citing works

Understanding Neural Networks Through Deep VisualizationPreprint

Yosinski, J., Clune, J., Nguyen, A. M., Fuchs, T. J., & Lipson, H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.06579 · Cited by 1,687 · see citing works

Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission

Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., & Elhadad, N. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2788613 · Cited by 1,685 · see citing works

Visualizing the effects of predictor variables in black box supervised learning models

Apley, D. W., & Zhu, J. (2020) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/rssb.12377 · Cited by 1,535 · see citing works

2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition

(2010) · doi:10.1109/cvpr15008.2010 · Cited by 1,433 · see citing works

Explainable AI: Interpreting, Explaining and Visualizing Deep Learning

(2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-28954-6 · Cited by 1,377 · see citing works

Explaining nonlinear classification decisions with deep taylor decomposition

Montavon, G., Lapuschkin, S., Binder, A., Samek, W., & Müller, K.-R. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.11.008 · Cited by 1,357 · see citing works

Explainable AI: From black box to glass box

Rai, A. (2020) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-019-00710-5 · Cited by 1,317 · see citing works

All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously

Fisher, A., Rudin, C., & Dominici, F. (2018) · arXiv · doi:10.48550/arxiv.1801.01489 · Cited by 1,165 · see citing works

Explainable artificial intelligence: A survey

Dosilovic, F. K., Brcic, M., & Hlupic, N. (2018) · 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) · doi:10.23919/mipro.2018.8400040 · Cited by 1,147 · see citing works

Interpretable Explanations of Black Boxes by Meaningful Perturbation

Fong, R. C., & Vedaldi, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.371 · Cited by 1,097 · see citing works

Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models

Samek, W., Wiegand, T., & Müller, K.-R. (2017) · arXiv · doi:10.48550/arxiv.1708.08296 · Cited by 1,080 · see citing works

Constructing optimal binary decision trees is NP-complete

Hyafil, L., & Rivest, R. L. (1976) · Information Processing Letters · doi:10.1016/0020-0190(76)90095-8 · Cited by 959 · see citing works

Deep belief networks

Hinton, G. E. (2009) · Scholarpedia · doi:10.4249/scholarpedia.5947 · Cited by 907 · see citing works

Visualizing and Understanding Recurrent Networks

Karpathy, A., Johnson, J., & Fei-Fei, L. (2015) · arXiv · doi:10.48550/arxiv.1506.02078 · Cited by 845 · see citing works

Explainable artificial intelligence: an analytical review

Angelov, P. P., Soares, E. A., Jiang, R., Arnold, N. I., & Atkinson, P. M. (2021) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1424 · Cited by 843 · see citing works

Interpretable Convolutional Neural Networks

Zhang, Q., Wu, Y. N., & Zhu, S.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00920 · Cited by 833 · see citing works

Incremental Induction of Decision Trees

Utgoff, P. E. (1989) · Machine Learning · doi:10.1023/a:1022699900025 · Cited by 793 · see citing works

Explainable AI (XAI): A systematic meta-survey of current challenges and future opportunities

Saeed, W., & Omlin, C. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110273 · Cited by 783 · see citing works

Rationalizing Neural Predictions

Lei, T., Barzilay, R., & Jaakkola, T. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1011 · Cited by 777 · see citing works

Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model

Letham, B., Rudin, C., McCormick, T. H., & Madigan, D. (2015) · The Annals of Applied Statistics · doi:10.1214/15-aoas848 · Cited by 766 · see citing works

Extracting Refined Rules from Knowledge-Based Neural Networks

Towell, G. G., & SHAVLIK, J. W. (1993) · Machine Learning · doi:10.1023/a:1022683529158 · Cited by 756 · see citing works

How to Explain Individual Classification DecisionsPreprint

Baehrens, D., Schroeter, T. S., Stefan Harmeling, B., Kawanabe, M., Hansen, K., & Mueller, K.-R. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0912.1128 · Cited by 743 · see citing works

Not Just a Black Box: Learning Important Features Through Propagating Activation DifferencesPreprint

Shrikumar, A., Greenside, P., Shcherbina, A., & Kundaje, A. B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.01713 · Cited by 704 · see citing works

Explainability for Large Language Models: A Survey

Zhao, H., Chen, H., Yang, F., Liu, N., Deng, H., Cai, H., Wang, S., Yin, D., & Du, M. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3639372 · Cited by 681 · see citing works

Visualizing and Understanding Neural Models in NLP

Li, J., Chen, X., Hovy, E. H., & Jurafsky, D. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1082 · Cited by 673 · see citing works

What do we want from Explainable Artificial Intelligence (XAI)? – A stakeholder perspective on XAI and a conceptual model guiding interdisciplinary XAI research

Langer, M., Oster, D., Speith, T., Hermanns, H., Kästner, L., Schmidt, E., Sesing, A., & Baum, K. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103473 · Cited by 652 · see citing works

Inverting Visual Representations with Convolutional Networks

Dosovitskiy, A., & Brox, T. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.522 · Cited by 631 · see citing works

A Survey of Methods for Explaining Black Box Models

Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · see citing works

Interpretable Machine Learning in Healthcare

Ahmad, M. A., Eckert, C., & Teredesai, A. (2018) · Proceedings of the 2018 ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics · doi:10.1145/3233547.3233667 · Cited by 583 · see citing works

The 2007 International Joint Conference on Neural Networks

(2007) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2007.4370894 · Cited by 575 · see citing works

An Efficient Explanation of Individual Classifications using Game Theory

Štrumbelj, E., & Kononenko, I. (2010) · Repository of the University of Ljubljana (University of Ljubljana) · doi:10.5555/1756006.1756007 · Cited by 570 · see citing works

Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey

Das, A., & Rad, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11371 · Cited by 504 · see citing works

Using natural frequencies to improve diagnostic inferences

Hoffrage, U., & Gigerenzer, G. (1998) · Academic Medicine · doi:10.1097/00001888-199805000-00024 · Cited by 501 · see citing works

Adversarial Attacks and Defences: A SurveyPreprint

Chakraborty, A., Alam, M., Dey, V., Chattopadhyay, A., & Mukhopadhyay, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.00069 · Cited by 479 · see citing works

Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep LearningPreprint

Papernot, N., & McDaniel, P. D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.04765 · Cited by 460 · see citing works

Data Mining with Decision Trees: Theory and Applications

(2015) · Online Information Review · doi:10.1108/oir-04-2015-0121 · Cited by 447 · see citing works

Using sensitivity analysis and visualization techniques to open black box data mining models

Cortez, P. C., & Embrechts, M. J. (2012) · Information Sciences · doi:10.1016/j.ins.2012.10.039 · Cited by 444 · see citing works

A systematic review of explainable artificial intelligence models and applications: recent developments and future trends

A., S., & R., S. (2023) · Decision Analytics Journal · doi:10.1016/j.dajour.2023.100230 · Cited by 432 · see citing works

Evaluating XAI: A comparison of rule-based and example-based explanations

van der Waa, J., Nieuwburg, E., Cremers, A., & Neerincx, M. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2020.103404 · Cited by 393 · see citing works

Interacting with Predictions: Visual Inspection of Black-box Machine Learning Models

Krause, J., Perer, A., & Ng, K. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858529 · Cited by 390 · see citing works

A comprehensive taxonomy for explainable artificial intelligence: a systematic survey of surveys on methods and concepts

Schwalbe, G., & Finzel, B. (2023) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-022-00867-8 · Cited by 381 · see citing works

A Survey of the State of Explainable AI for Natural Language Processing

Danilevsky, M., Qian, K., Aharonov, R., Katsis, Y., Kawas, B., & Sen, P. (2020) · Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (AACL-IJCNLP) · doi:10.18653/v1/2020.aacl-main.46 · Cited by 380 · see citing works

Machine learning, social learning and the governance of self-driving cars

Stilgoe, J. (2018) · Social Studies of Science · doi:10.1177/0306312717741687 · Cited by 375 · see citing works

Rule generation from neural networks

Fu, L. (1994) · IEEE Transactions on Systems Man and Cybernetics · doi:10.1109/21.299696 · Cited by 368 · see citing works

Machine Learning Proceedings 1993

(1993) · Elsevier eBooks · doi:10.1016/c2009-0-27798-1 · Cited by 359 · see citing works

Interpreting CNNs via Decision Trees

Zhang, Q., Yang, Y., Ma, H., & Wu, Y. N. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00642 · Cited by 350 · see citing works

Shapley values for feature selection: the good, the bad, and the axioms

Fryer, D., Strumke, I., & Nguyen, H. (2021) · IEEE Access · doi:10.1109/access.2021.3119110 · Cited by 347 · see citing works

Learning how to explain neural networks: PatternNet and PatternAttributionPreprint

Kindermans, P.-J., Schütt, K. T., Alber, M., Müller, K. R., Erhan, D., Kim, B., & Dähne, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.05598 · Cited by 325 · see citing works

On the current state of combining human and artificial intelligence for strategic organizational decision making

Trunk, A., Birkel, H. S., & Hartmann, E. (2020) · BuR - Business Research · doi:10.1007/s40685-020-00133-x · Cited by 302 · see citing works

Auditing black-box models for indirect influence

Adler, P., Falk, C., Friedler, S. A., Nix, T., Rybeck, G., Scheidegger, C., Smith, B., & Venkatasubramanian, S. (2018) · Knowledge and Information Systems · doi:10.1007/s10115-017-1116-3 · Cited by 280 · see citing works

Assessment of Landslide Susceptibility by Decision Trees in the Metropolitan Area of Istanbul, Turkey

Nefeslioğlu, H. A., Sezer, E. A., Gökçeoğlu, C., Bozkır, A. S., & Duman, T. Y. (2010) · Mathematical Problems in Engineering · doi:10.1155/2010/901095 · Cited by 277 · see citing works

Interpretation of nonlinear relationships between process variables by use of random forests

Auret, L., & Aldrich, C. (2012) · Minerals Engineering · doi:10.1016/j.mineng.2012.05.008 · Cited by 270 · see citing works

Explaining Data-Driven Document Classifications1

Martens, D., & Provost, F. (2014) · MIS Quarterly · doi:10.25300/misq/2014/38.1.04 · Cited by 264 · see citing works

Explaining Classifications For Individual Instances

Robnik‐Šikonja, M., & Kononenko, I. (2008) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2007.190734 · Cited by 255 · see citing works

Interpreting tree ensembles with inTrees

Deng, H. (2018) · International Journal of Data Science and Analytics · doi:10.1007/s41060-018-0144-8 · Cited by 254 · see citing works

Untitled work

(2012) · Cited by 253 · see citing works

The Shapley Value: Essays in Honor of Lloyd S. Shapley.

Cubitt, R. P., & Roth, A. E. (1991) · The Economic Journal · doi:10.2307/2233574 · Cited by 250 · see citing works

Interpretable Deep Models for ICU Outcome Prediction.

Che, Z., Purushotham, S., Khemani, R. G., & Liu, Y. (2016) · PubMed Central · Cited by 240 · see citing works

Extracting comprehensible models from trained neural networks

Craven, M. W., & SHAVLIK, J. W. (1996) · Cited by 233 · see citing works

Decision trees for real-time transient stability prediction

Rovnyak, S. M., Kretsinger, S., Thorp, J. S., et al. (1994) · IEEE Transactions on Power Systems · doi:10.1109/59.336122 · Cited by 232 · see citing works

Feature space interpretation of SVMs with indefinite kernels

Haasdonk, B. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.78 · Cited by 232 · see citing works

A linear approximation method for the Shapley value

Fatima, S. S., Wooldridge, M., & Jennings, N. R. (2008) · Artificial Intelligence · doi:10.1016/j.artint.2008.05.003 · Cited by 224 · see citing works

The effects of example-based explanations in a machine learning interface

Cai, C. J., Jongejan, J., & Holbrook, J. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302289 · Cited by 220 · see citing works

Symbolic interpretation of artificial neural networks

Taha, I., & Ghosh, J. (1999) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/69.774103 · Cited by 217 · see citing works

Beyond Sparsity: Tree Regularization of Deep Models for Interpretability

Wu, M., Hughes, M., Parbhoo, S., Zazzi, M., Roth, V., & Doshi-Velez, F. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11501 · Cited by 213 · see citing works

Explainable and Interpretable Models in Computer Vision and Machine Learning

Escalante, H. J., Escalera, S., Guyon, I., Baró, X., Güçlütürk, Y., Güçlü, U., & van Gerven, M. (2018) · The Springer series on challenges in machine learning · doi:10.1007/978-3-319-98131-4 · Cited by 203 · see citing works

Interpretable Predictions of Tree-based Ensembles via Actionable Feature Tweaking

Tolomei, G., Silvestri, F., Haines, A., & Lalmas, M. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098039 · Cited by 192 · see citing works

Bayesian Methods for Support Vector Machines: Evidence and Predictive Class Probabilities

Sollich, P. (2002) · Machine Learning · doi:10.1023/a:1012489924661 · Cited by 189 · see citing works

Deep exploration of random forest model boosts the interpretability of machine learning studies of complicated immune responses and lung burden of nanoparticles

Yu, F., Wei, C., Deng, P., Peng, T., & Hu, X. (2021) · Science Advances · doi:10.1126/sciadv.abf4130 · Cited by 187 · see citing works

2015 IEEE Symposium on Security and Privacy

(2015) · doi:10.1109/sp33448.2015 · Cited by 183 · see citing works

Explainable Artificial Intelligence (XAI) for Internet of Things: A Survey

Kök, İ., Okay, F. Y., Muyanlı, Ö., & Özdemi̇r, S. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3287678 · Cited by 176 · see citing works

Extracting Rules from Artificial Neural Networks with Distributed Representations

Thrun, S. (1994) · Cited by 175 · see citing works

Reverse Engineering the Neural Networks for Rule Extraction in Classification Problems

Augasta, M. G., & Kathirvalavakumar, T. (2011) · Neural Processing Letters · doi:10.1007/s11063-011-9207-8 · Cited by 174 · see citing works

Visualisation and interpretation of Support Vector Regression models

Üstün, B., Melssen, W. J., & Buydens, L. M. C. (2007) · Analytica Chimica Acta · doi:10.1016/j.aca.2007.03.023 · Cited by 172 · see citing works

Explanations can be manipulated and geometry is to blame

Dombrowski, A.-K., Alber, M., Anders, C. J., Ackermann, M., Müller, K.-R., & Kessel, P. (2019) · arXiv · doi:10.48550/arxiv.1906.07983 · Cited by 171 · see citing works

ANN-DT: an algorithm for extraction of decision trees from artificial neural networks

Schmitz, G. P. J., Aldrich, C., & Gouws, F. S. (1999) · IEEE Transactions on Neural Networks · doi:10.1109/72.809084 · Cited by 170 · see citing works

Extracting symbolic rules from trained neural network ensembles

Zhou, Z., Jiang, Y. Y., & Chen, S. (2003) · doi:10.5555/859130.859132 · Cited by 169 · see citing works

A methodology to explain neural network classification

Féraud, R., & Clérot, F. (2002) · Neural Networks · doi:10.1016/s0893-6080(01)00127-7 · Cited by 168 · see citing works

Extracting rules from trained neural networks

Tsukimoto, H. (2000) · IEEE Transactions on Neural Networks · doi:10.1109/72.839008 · Cited by 166 · see citing works

Classification of Explainable Artificial Intelligence Methods through Their Output Formats

Vilone, G., & Longo, L. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3030032 · Cited by 157 · see citing works

FERNN: An Algorithm for Fast Extraction of Rules from Neural Networks

Setiono, R., & Leow, W. K. (2000) · Applied Intelligence · doi:10.1023/a:1008307919726 · Cited by 147 · see citing works

2009 IEEE Symposium on Computational Intelligence and Data Mining

(2009) · doi:10.1109/cidm14692.2009 · Cited by 142 · see citing works

Improving Interpretability of Deep Neural Networks with Semantic Information

Dong, Y., Su, H., Zhu, J., & Zhang, B. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.110 · Cited by 141 · see citing works

2008 IEEE International Conference on Data Mining Workshops

(2008) · doi:10.1109/icdmw14999.2008 · Cited by 141 · see citing works

Artificial intelligence explainability: the technical and ethical dimensions

McDermid, J. A., Jia, Y., Porter, Z., & Habli, I. (2021) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2020.0363 · Cited by 136 · see citing works

Eclectic Rule-Extraction From Support Vector Machines

Barakat, N., & Diederich, J. (2008) · doi:10.5281/zenodo.1055511 · Cited by 131 · see citing works

Local Interpretable Model-Agnostic Explanations For Music Content Analysis.

Mishra, S., Sturm, B. L. T., & Dixon, S. (2017) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1417386 · Cited by 128 · see citing works

A Machine Learning Approach to 5G Infrastructure Market Optimization

Bega, D., Gramaglia, M., Banchs, A., Sciancalepore, V., & Costa‐Pérez, X. (2019) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2019.2896950 · Cited by 128 · see citing works

Discovering additive structure in black box functions

Hooker, G. (2004) · doi:10.1145/1014052.1014122 · Cited by 127 · see citing works

Extracting decision trees from trained neural networks

Krishnan, R., Sivakumar, G., & Bhattacharya, P. (1999) · Pattern Recognition · doi:10.1016/s0031-3203(98)00181-2 · Cited by 127 · see citing works

Distill-and-Compare

Tan, S., Caruana, R., Hooker, G., & Lou, Y. (2018) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3278721.3278725 · Cited by 127 · see citing works

Fooling Neural Network Interpretations via Adversarial Model ManipulationPreprint

Heo, J., Joo, S., & Moon, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.02041 · Cited by 126 · see citing works

Black Box Attacks on Explainable Artificial Intelligence(XAI) methods in Cyber Security

Kuppa, A., & Le-Khac, N.-A. (2020) · 2020 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9206780 · Cited by 123 · see citing works

Malware classification and composition analysis: A survey of recent developments

Abusitta, A. H., Li, M. Q., & Fung, B. C. M. (2021) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2021.102828 · Cited by 116 · see citing works

A multiple kernel support vector machine scheme for feature selection and rule extraction from gene expression data of cancer tissue

Chen, Z.-Y., Li, J., & Wei, L. (2007) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2007.07.008 · Cited by 115 · see citing works

Rule Extraction from Support Vector Machines: A Sequential Covering Approach

Barakat, N., & Bradley, A. P. (2007) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2007.190610 · Cited by 114 · see citing works

Human Evaluation of Models Built for Interpretability

Lage, I., Chen, E., He, J., Narayanan, M., Kim, B., Gershman, S. J., & Doshi‐Velez, F. (2019) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v7i1.5280 · Cited by 114 · see citing works

A peek into the black box: exploring classifiers by randomization

Henelius, A., Puolamäki, K., Boström, H., Asker, L., & Papapetrou, P. (2014) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-014-0368-8 · Cited by 98 · see citing works

Explainable Artificial Intelligence Approaches: A SurveyPreprint

Islam, S. R., Eberle, W., Ghafoor, S., & Ahmed, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.09429 · Cited by 98 · see citing works

Nomograms for visualizing support vector machines

Jakulin, A., Možina, M., Demšar, J., Bratko, I., & Zupan, B. (2005) · Knowledge Discovery and Data Mining · doi:10.1145/1081870.1081886 · Cited by 94 · see citing works

Interpretability via Model ExtractionPreprint

Bastani, O., Kim, C., & Bastani, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.09773 · Cited by 93 · see citing works

PALM: Machine Learning Explanations For Iterative Debugging

Krishnan, S., & Wu, E. (2017) · Proceedings of the 2nd Workshop on Human-In-the-Loop Data Analytics · doi:10.1145/3077257.3077271 · Cited by 81 · see citing works

Interpreting support vector machine models for multivariate group wise analysis in neuroimaging

Gaonkar, B., Shinohara, R. T., & Davatzikos, C. A. (2015) · Medical Image Analysis · doi:10.1016/j.media.2015.06.008 · Cited by 75 · see citing works

On the Relation of Trust and Explainability: Why to Engineer for Trustworthiness

Kästner, L., Langer, M., Lazar, V., Schomäcker, A., Speith, T., & Sterz, S. (2021) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew53955.2021.00031 · Cited by 72 · see citing works

Extracting the knowledge embedded in support vector machines

Fu, X., Ong, C. J., Keerthi, S., Hung, G. G., & Goh, L. (2005) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2004.1379916 · Cited by 70 · see citing works

A new approximation method for the Shapley value applied to the WTC 9/11 terrorist attack

Campen, T., Hamers, H., Husslage, B. G. M., & Lindelauf, R. H. A. (2017) · Social Network Analysis and Mining · doi:10.1007/s13278-017-0480-z · Cited by 70 · see citing works

CoCoX: Generating Conceptual and Counterfactual Explanations via Fault-Lines

Akula, A., Wang, S., & Zhu, S. C. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i03.5643 · Cited by 68 · see citing works

Forest Floor Visualizations of Random ForestsPreprint

Welling, S. H., Refsgaard, H. H. F., Brockhoff, P. B., & Clemmensen, L. K. H. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.09196 · Cited by 65 · see citing works

Making Tree Ensembles InterpretablePreprint

Hara, S., & Hayashi, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.05390 · Cited by 60 · see citing works

Nothing Else Matters: Model-Agnostic Explanations By Identifying Prediction InvariancePreprint

Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.05817 · Cited by 60 · see citing works

IEEE Computer Society Annual Symposium on VLSI

(2004) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2004.823917 · Cited by 59 · see citing works

Rule-Based Learning Systems for Support Vector Machines

Núñez, H., Ángulo, C., & Catalá, A. (2006) · Neural Processing Letters · doi:10.1007/s11063-006-9007-8 · Cited by 58 · see citing works

Interpreting linear support vector machine models with heat map molecule coloring

Rosenbaum, L., Hinselmann, G., Jahn, A., & Zell, A. (2011) · Journal of Cheminformatics · doi:10.1186/1758-2946-3-11 · Cited by 57 · see citing works

XAI Beyond Classification: Interpretable Neural ClusteringPreprint

Peng, X., Li, Y., Tsang, I. W.-H., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.07292 · Cited by 54 · see citing works

Interpretable machine learning

Molnar, C. (2020) · doi:10.58248/pn633 · Cited by 51 · see citing works

Increasing the Interpretability of Recurrent Neural Networks Using Hidden Markov ModelsPreprint

Krakovna, V., & Doshi‐Velez, F. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.05320 · Cited by 50 · see citing works

Assessing the Local Interpretability of Machine Learning ModelsPreprint

Slack, D., Friedler, S. A., Scheidegger, C., & Roy, C. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.03501 · Cited by 50 · see citing works

Explainable AI in Healthcare and Medicine

H\"ugle, M., Kalweit, G., Huegle, T., et al. (2020) · Studies in computational intelligence · doi:10.1007/978-3-030-53352-6 · Cited by 50 · see citing works

Accuracy vs. comprehensibility in data mining models

Johansson, U., Niklasson, L., & König, R. (2004) · Cited by 49 · see citing works

Tree Space Prototypes

Tan, S., Soloviev, M., Hooker, G., & Wells, M. T. (2020) · ACM-IMS Foundations of Data Science Conference (FODS) · doi:10.1145/3412815.3416893 · Cited by 49 · see citing works

Probabilistic interpretations and bayesian methods for support vector machines

Sollich, P. (1999) · doi:10.1049/cp:19991090 · Cited by 46 · see citing works

TreeView: Peeking into Deep Neural Networks Via Feature-Space PartitioningPreprint

Thiagarajan, J. J., Kailkhura, B., Sattigeri, P., & Ramamurthy, K. N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.07429 · Cited by 44 · see citing works

Knowledge Discovery Via Multiple Models

Domingos, P. (1998) · Intelligent Data Analysis · doi:10.3233/ida-1998-2303 · Cited by 43 · see citing works

Recurrent Neural Networks

Martinez-Ramon, M., Ajith, M., & Kurup, A. R. (2024) · doi:10.1002/9781119861898.ch5 · Cited by 43 · see citing works

Interpreting Models via Single Tree ApproximationPreprint

Zhou, Y., & Hooker, G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.09036 · Cited by 39 · see citing works

Learning to Explain With Complemental Examples

Kanehira, A., & Harada, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00880 · Cited by 39 · see citing works

Enhancing Transparency and Control When Drawing Data-Driven Inferences About Individuals

Chen, D., Fraiberger, S. P., Moakler, R., & Provost, F. J. (2017) · Big Data · doi:10.1089/big.2017.0074 · Cited by 37 · see citing works

A Survey on Representation Learning Efforts in Cybersecurity Domain

Usman, M., Jan, M. A., He, X., & Chen, J. (2019) · ACM Computing Surveys · doi:10.1145/3331174 · Cited by 37 · see citing works

Rule extraction from trained neural networks using genetic algorithms

Arbatli, A. D., & Akın, H. L. (1997) · Nonlinear Analysis · doi:10.1016/s0362-546x(96)00267-2 · Cited by 34 · see citing works

Interpretable Recurrent Neural Networks Using Sequential Sparse RecoveryPreprint

Wisdom, S., Powers, T., Pitton, J. W., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.07252 · Cited by 34 · see citing works

Support vector machine interpretation

Navia-Vázquez, A., & Parrado-Hernández, E. (2006) · Neurocomputing · doi:10.1016/j.neucom.2005.12.118 · Cited by 33 · see citing works

Explaining Deep Neural NetworksPreprint

Camburu, O.-M. (2020) · arXiv.org · doi:10.48550/arxiv.2010.01496 · Cited by 33 · see citing works

Interpreting Classifiers through Attribute Interactions in DatasetsPreprint

Henelius, A., Puolamäki, K., & Ukkonen, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.07576 · Cited by 32 · see citing works

Good Explanation for Algorithmic TransparencyPreprint

Lu, J., Lee, D., Kim, T. W., & Danks, D. J. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3503603 · Cited by 30 · see citing works

Integration of Reusable Systems

Bouabana‐Tebibel, T., & Rubin, S. H. (2014) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-04717-1 · Cited by 27 · see citing works

Explainable AI: Using Shapley Value to Explain Complex Anomaly Detection ML-Based Systems

Zou, J., & Petrosian, O. (2020) · Frontiers in artificial intelligence and applications · doi:10.3233/faia200777 · Cited by 27 · see citing works

An interval mean–average absolute deviation model for multiperiod portfolio selection with risk control and cardinality constraints

Zhang, P. (2015) · Soft Computing · doi:10.1007/s00500-014-1583-3 · Cited by 24 · see citing works

A Survey on Machine Learning Adversarial Attacks

de Mello, F. L. (2020) · Journal of Information Security and Cryptography (Enigma) · doi:10.17648/jisc.v7i1.76 · Cited by 23 · see citing works

A model-agnostic, network theory-based framework for supporting XAI on classifiers

Bonifazi, G., Cauteruccio, F., Corradini, E., Marchetti, M., Giorgio Terracina, B., Domenico Ursino, A., & Virgili, L. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122588 · Cited by 23 · see citing works

Stacking with Auxiliary Features for Visual Question Answering

Rajani, N. F., & Mooney, R. J. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1201 · Cited by 20 · see citing works

2007 IEEE workshop on automatic speech recognition and understanding

(2007) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2007.4430069 · Cited by 20 · see citing works

Interpreting Deep Classifier by Visual Distillation of Dark KnowledgePreprint

Xu, K., Park, D. H., Yi, C., & Sutton, C. A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.04042 · Cited by 16 · see citing works

Support vector machines with symbolic interpretation

Nunez, H., Ángulo, C., & Catalá, A. (2003) · Brazilian Symposium on Neural Networks (SBRN) · doi:10.1109/sbrn.2002.1181456 · Cited by 15 · see citing works

Explainable AI Within the Digital Transformation and Cyber Physical Systems

(2021) · Springer eBooks · doi:10.1007/978-3-030-76409-8 · Cited by 14 · see citing works

Interpretable Two-level Boolean Rule Learning for ClassificationPreprint

Su, G., Wei, D., Varshney, K. R., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.07361 · Cited by 14 · see citing works

Fifth International Conference on Hybrid Intelligent Systems

(2005) · International Conference on Hybrid Intelligent Systems (HIS) · doi:10.1109/ichis.2005.45 · Cited by 12 · see citing works

On the Effectiveness of Interpretable Feedforward Neural Network

Li, M. Q., Fung, B. C. M., & Abusitta, A. H. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892343 · Cited by 9 · see citing works

VII Brazilian Symposium on Neural Networks, 2002. SBRN 2002. Proceedings.

(2002) · doi:10.1109/sbrn.2002 · Cited by 3 · see citing works

AI-driven design optimization for sustainable buildings: a systematic review

Manmatharasan, P., Bitsuamlak, G., & Grolinger, K. (2025) · Energy and Buildings · doi:10.1016/j.enbuild.2025.115440 · Cited by 63 · see citing works

Sustainable AI: Emerging Trends, Impacts, and Future Challenges

Philipo, A. G., Ning, H., Sarwatt, D. S., Mohamed, J. A., Yusufu, A. S., Shi, F., Urenje, S., & Ding, J. (2025) · IEEE Transactions on Sustainable Computing · doi:10.1109/tsusc.2025.3611272 · Cited by 9 · see citing works

Explainability in AI: Comparing Human-Like and System-Like Trust Repair Strategies

Konopka, B., & Wiesche, M. (2026) · Information Systems Frontiers · doi:10.1007/s10796-026-10751-1 · Cited by 1 · see citing works

ИСПОЛЬЗОВAНИЕ ПОТЕНЦИAЛA СОЦИAЛЬНЫХ ПAРТНЕРОВ В ПОДГОТОВКЕ БУДУЩИХ ПЕДAГОГОВ

Атабекова, Б. Б. (2024) · Habaršy. Pedagogika ġylymdary seriâsy · doi:10.51889/2959-5762.2023.80.4.019

DETERMINING QUALITY REQUIREMENTS AT THE UNIVERSITIES TO IMPROVE THE QUALITY OF EDUCATION

Mustafa, N. H., Kelesbayev, D., & Alibekova, Z. D. (2018) · The Journal of Economic Research & Business Administration · doi:10.26577/be-2018-2-2141

Susquehanna Chorale Spring Concert "Roots and Wings"

Hawkins, D. (2017)

Using DataGrid Control to Realize DataBase of Querying in VB6.0

Hu, S. (2000) · Modern Computer

ESKVS: efficient and secure approach for keyframes-based video summarization framework

Saini, P., & Berwal, K. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-18405-7 · Cited by 10 · see citing works