The DOI matches its Crossref record; 176 of the 176 references it cites were found in a registry or an index.
A Systematic Review of Explainable Artificial Intelligence in Terms of Different Application Domains and Tasks
Open accessAbstract
Artificial intelligence (AI) and machine learning (ML) have recently been radically improved and are now being employed in almost every application domain to develop automated or semi-automated systems. To facilitate greater human acceptability of these systems, explainable artificial intelligence (XAI) has experienced significant growth over the last couple of years with the development of highly accurate models but with a paucity of explainability and interpretability. The literature shows evidence from numerous studies on the philosophy and methodologies of XAI. Nonetheless, there is an evident scarcity of secondary studies in connection with the application domains and tasks, let alone review studies following prescribed guidelines, that can enable researchers’ understanding of the current trends in XAI, which could lead to future research for domain- and application-specific method development. Therefore, this paper presents a systematic literature review (SLR) on the recent developments of XAI methods and evaluation metrics concerning different application domains and tasks. This study considers 137 articles published in recent years and identified through the prominent bibliographic databases. This systematic synthesis of research articles resulted in several analytical findings: XAI methods are mostly developed for safety-critical domains worldwide, deep learning and ensemble models are being exploited more than other types of AI/ML models, visual explanations are more acceptable to end-users and robust evaluation metrics are being developed to assess the quality of explanations. Research studies have been performed on the addition of explanations to widely used AI/ML models for expert users. However, more attention is required to generate explanations for general users from sensitive domains such as finance and the judicial system.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561
From local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01228-7 · Cited by 6,909
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524
Wohlin, C. (2014) · Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2601248.2601268 · Cited by 4,234 · cited 1×
“During the process, 19 additional related articles were found from a complementary snowballing search [43], in simpler terms, a recursive reference search.”
BackgroundExplaining prediction models and individual predictions with feature contributions
Štrumbelj, E., & Kononenko, I. (2014) · Knowledge and Information Systems · doi:10.1007/s10115-013-0679-x · Cited by 2,286
Anchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162
Explainable machine-learning predictions for the prevention of hypoxaemia during surgery
Lundberg, S. M., Nair, B., Vavilala, M. S., Horibe, M., Eisses, M. J., Adams, T., Liston, D. E., Low, D. K.-W., Newman, S.-F., Kim, J., et al. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0304-0 · Cited by 1,967
Causability and explainability of artificial intelligence in medicine
Holzinger, A., Langs, G., Denk, H., Zatloukal, K., & Müller, H. (2019) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1312 · Cited by 1,885
Explaining nonlinear classification decisions with deep taylor decomposition
Montavon, G., Lapuschkin, S., Binder, A., Samek, W., & Müller, K.-R. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.11.008 · Cited by 1,357
Explainable AI: From black box to glass box
Rai, A. (2020) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-019-00710-5 · Cited by 1,317
Evaluating the Visualization of What a Deep Neural Network Has Learned
Samek, W., Binder, A., Montavon, G., Lapuschkin, S., & Muller, K.-R. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2599820 · Cited by 1,164
Explainable artificial intelligence: A survey
Dosilovic, F. K., Brcic, M., & Hlupic, N. (2018) · 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO) · doi:10.23919/mipro.2018.8400040 · Cited by 1,147
Drug discovery with explainable artificial intelligence
Jiménez-Luna, J., Grisoni, F., & Schneider, G. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00236-4 · Cited by 1,063
Counterfactual explanations without opening the black box: automated decisions and the GDPR
Wachter, S., Mittelstadt, B., & Russell, C. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3063289 · Cited by 1,061
Designing Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906
Interpretable Convolutional Neural Networks
Zhang, Q., Wu, Y. N., & Zhu, S.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00920 · Cited by 833
Mittelstadt, B., Russell, C., & Wachter, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287574 · Cited by 794
Explainable Reasoning over Knowledge Graphs for Recommendation
Wang, X., Wang, D., Xu, C., He, X., Cao, Y., & Chua, T.-S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33015329 · Cited by 785
Letham, B., Rudin, C., McCormick, T. H., & Madigan, D. (2015) · The Annals of Applied Statistics · doi:10.1214/15-aoas848 · Cited by 766
Explainable AI: A Brief Survey on History, Research Areas, Approaches and Challenges
Xu, F., Uszkoreit, H., Du, Y., Fan, W., Zhao, D., & Zhu, J. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-32236-6_51 · Cited by 758
Model-Agnostic Interpretability of Machine Learning
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · arXiv · doi:10.48550/arxiv.1606.05386 · Cited by 699
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629
Explainable Deep Learning for Pulmonary Disease and Coronavirus COVID-19 Detection from X-rays
Brunese, L., Mercaldo, F., Reginelli, A., & Santone, A. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105608 · Cited by 623
Notions of explainability and evaluation approaches for explainable artificial intelligence
Vilone, G., & Longo, L. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.009 · Cited by 598 · cited 1×
“Above all, several review studies were conducted by Vilone and Longo to gather and present the recent developments in XAI [8,9,27].”
BackgroundTowards Explainable Artificial Intelligence
Samek, W., & Müller, K.-R. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-28954-6_1 · Cited by 580
Loyola-Gonzalez, O. (2019) · IEEE Access · doi:10.1109/access.2019.2949286 · Cited by 559
DARPA's explainable artificial intelligence (XAI) program
Gunning, D. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3308446 · Cited by 546
Generating Visual Explanations
Hendricks, L. A., Akata, Z., Rohrbach, M., Donahue, J., Schiele, B., & Darrell, T. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_1 · Cited by 534
TriRank: Review-aware Explainable Recommendation by Modeling Aspects
He, X., Chen, T., Kan, M.-Y., & Chen, X. (2015) · Proceedings of the 24th ACM International on Conference on Information and Knowledge Management · doi:10.1145/2806416.2806504 · Cited by 456
Lauritsen, S. M., Kristensen, M., Olsen, M. V., Larsen, M. S., Lauritsen, K. M., Jørgensen, M. J., Lange, J., & Thiesson, B. (2020) · Nature Communications · doi:10.1038/s41467-020-17431-x · Cited by 448
Holzinger, A., Carrington, A., & Müller, H. (2020) · KI - Künstliche Intelligenz · doi:10.1007/s13218-020-00636-z · Cited by 406
Recommendation system based on deep learning methods: a systematic review and new directions
Da’u, A., & Salim, N. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-019-09744-1 · Cited by 402
Evaluating XAI: A comparison of rule-based and example-based explanations
van der Waa, J., Nieuwburg, E., Cremers, A., & Neerincx, M. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2020.103404 · Cited by 393
Albaum, G. (1997) · Market Research Society. Journal. · doi:10.1177/147078539703900202 · Cited by 377
COVIDGR Dataset and COVID-SDNet Methodology for Predicting COVID-19 Based on Chest X-Ray Images
Tabik, S., Gomez-Rios, A., Martin-Rodriguez, J. L., Sevillano-Garcia, I., Rey-Area, M., Charte, D., Guirado, E., Suarez, J. L., Luengo, J., Valero-Gonzalez, M. A., et al. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3037127 · Cited by 375
Interpretable Learning for Self-Driving Cars by Visualizing Causal Attention
Kim, J., & Canny, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.320 · Cited by 366
A systematic literature review: Opinion mining studies from mobile app store user reviews
Genc-Nayebi, N., & Abran, A. (2017) · Journal of Systems and Software · doi:10.1016/j.jss.2016.11.027 · Cited by 358
Factual and Counterfactual Explanations for Black Box Decision Making
Guidotti, R., Monreale, A., Giannotti, F., Pedreschi, D., Ruggieri, S., & Turini, F. (2019) · IEEE Intelligent Systems · doi:10.1109/mis.2019.2957223 · Cited by 349
FACE: Feasible and Actionable Counterfactual Explanations
Poyiadzi, R., Sokol, K., Santos-Rodriguez, R., De Bie, T., & Flach, P. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375850 · Cited by 317
Payrovnaziri, S. N., Chen, Z., Rengifo-Moreno, P., Miller, T., Bian, J., Chen, J. H., Liu, X., & He, Z. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa053 · Cited by 313
Kwon, B. C., Choi, M.-J., Kim, J. T., Choi, E., Kim, Y. B., Kwon, S., Sun, J., & Choo, J. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2865027 · Cited by 294
Towards explainable deep neural networks (xDNN)
Angelov, P., & Soares, E. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.07.010 · Cited by 292
Explainable artificial intelligence for breast cancer: A visual case-based reasoning approach
Lamy, J.-B., Sekar, B., Guezennec, G., Bouaud, J., & Séroussi, B. (2019) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2019.01.001 · Cited by 282
Explainability fact sheets: a framework for systematic assessment of explainable approaches
Sokol, K., & Flach, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372870 · Cited by 264
Goebel, R., Chander, A., Holzinger, K., Lecue, F., Akata, Z., Stumpf, S., Kieseberg, P., & Holzinger, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-99740-7_21 · Cited by 259
explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning
Spinner, T., Schlegel, U., Schafer, H., & El-Assady, M. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2934629 · Cited by 254
Rulematrix: Visualizing and understanding classifiers with rules
Ming, Y., Qu, H., & Bertini, E. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2864812 · Cited by 243
A review of explanation methods for Bayesian networks
LACAVE, C., & DÍEZ, F. J. (2002) · The Knowledge Engineering Review · doi:10.1017/s026988890200019x · Cited by 237
An explainable artificial intelligence system for small-unit tactical behavior
van Lent, M., Fisher, W. M., & Mancuso, M. R. (2004) · Cited by 215
Classification of Explainable Artificial Intelligence Methods through Their Output Formats
Vilone, G., & Longo, L. (2021) · Machine Learning and Knowledge Extraction · doi:10.3390/make3030032 · Cited by 213
Comparison-Based Inverse Classification for Interpretability in Machine Learning
Laugel, T., Lesot, M., Marsala, C., Renard, X., & Detyniecki, M. (2018) · Communications in computer and information science · doi:10.1007/978-3-319-91473-2_9 · Cited by 204
Chen, H.-Y., & Lee, C.-H. (2020) · IEEE Access · doi:10.1109/access.2020.3006491 · Cited by 178
Gade, K., Geyik, S. C., Kenthapadi, K., Mithal, V., & Taly, A. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3332281 · Cited by 172
A Deep Learning-Based Hybrid Framework for Object Detection and Recognition in Autonomous Driving
Li, Y., Wang, H., Dang, L. M., Nguyen, T. N., Han, D., Lee, A., Jang, I., & Moon, H. (2020) · IEEE Access · doi:10.1109/access.2020.3033289 · Cited by 169
Explainable Artificial Intelligence for Neuroscience: Behavioral Neurostimulation
Fellous, J., Sapiro, G., Rossi, A. F., Mayberg, H., & Ferrante, M. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.01346 · Cited by 166
A Review of Explainable Deep Learning Cancer Detection Models in Medical Imaging
Gulum, M. A., Trombley, C. M., & Kantardzic, M. M. (2021) · Applied Sciences · doi:10.3390/app11104573 · Cited by 165
Doctor XAI: an ontology-based approach to black-box sequential data classification explanations
Panigutti, C., Perotti, A., & Pedreschi, D. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372855 · Cited by 164
A methodology to explain neural network classification
Féraud, R., & Clérot, F. (2002) · Neural Networks · doi:10.1016/s0893-6080(01)00127-7 · Cited by 163
Enhanced Maintenance and Explanation of Expert Systems Through Explicit Models of Their Development
Neches, R., Swartout, W. R., & Moore, J. D. (1985) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1985.231882 · Cited by 163
Explainable Artificial Intelligence: Concepts, Applications, Research Challenges and Visions
Longo, L., Goebel, R., Lecue, F., Kieseberg, P., & Holzinger, A. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-57321-8_1 · Cited by 162
Mathews, S. M. (2019) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-22868-2_90 · Cited by 160
Local interpretable model-agnostic explanations for classification of lymph node metastases
Palatnik de Sousa, I., Maria Bernardes Rebuzzi Vellasco, M., & Costa da Silva, E. (2019) · Sensors · doi:10.3390/s19132969 · Cited by 155
Model Agnostic Supervised Local Explanations
Plumb, G., Molitor, D., & Talwalkar, A. (2018) · arXiv · doi:10.48550/arxiv.1807.02910 · Cited by 149
SurvLIME: A method for explaining machine learning survival models
Kovalev, M. S., Utkin, L. V., & Kasimov, E. M. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106164 · Cited by 145
Explaining Black-Box Algorithms Using Probabilistic Contrastive Counterfactuals
Galhotra, S., Pradhan, R., & Salimi, B. (2021) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3448016.3458455 · Cited by 138
Explainable AI in Deep Reinforcement Learning Models for Power System Emergency Control
Zhang, K., Zhang, J. J., Xu, P., Gao, T., & Gao, W. (2021) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2021.3096824 · Cited by 137
Post-hoc explanation of black-box classifiers using confident itemsets
Moradi, M., & Samwald, M. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113941 · Cited by 130
Guidelines for performing Systematic Research Projects Reviews.
García‐Holgado, A., Marcos-Pablos, S., & García‐Peñalvo, F. J. (2020) · International Journal of Interactive Multimedia and Artificial Intelligence · doi:10.9781/ijimai.2020.05.005 · Cited by 129
An Evaluation of the Human-Interpretability of Explanation
Lage, I., Chen, E., He, J., Narayanan, M., Kim, B., Gershman, S., & Doshi-Velez, F. (2019) · arXiv · doi:10.48550/arxiv.1902.00006 · Cited by 125
To trust or not to trust an explanation: using LEAF to evaluate local linear XAI methods
Amparore, E. G., Perotti, A., & Bajardi, P. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.479 · Cited by 120
Stakeholders in Explainable AI
Preece, A., Harborne, D., Braines, D., Tomsett, R., & Chakraborty, S. (2018) · arXiv · doi:10.48550/arxiv.1810.00184 · Cited by 117
Zheng, Q., Delingette, H., & Ayache, N. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.06.001 · Cited by 116
Ada-WHIPS: explaining AdaBoost classification with applications in the health sciences
Hatwell, J., Gaber, M. M., & Azad, R. M. A. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01201-2 · Cited by 111
Carletti, M., Masiero, C., Beghi, A., & Susto, G. A. (2019) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2019.8913901 · Cited by 110
A Bibliometric Analysis of the Explainable Artificial Intelligence Research Field
Alonso, J. M., Castiello, C., & Mencar, C. (2018) · Communications in computer and information science · doi:10.1007/978-3-319-91473-2_1 · Cited by 108
Evolving Rule-Based Explainable Artificial Intelligence for Unmanned Aerial Vehicles
Keneni, B., Kaur, D., Bataineh, A. S. A., Devabhaktuni, V. K., Javaid, A. Y., Zaientz, J. D., & Marinier, R. (2019) · IEEE Access · doi:10.1109/access.2019.2893141 · Cited by 108
Explainable Deep Neural Networks for Multivariate Time Series Predictions
Assaf, R., & Schumann, A. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/932 · Cited by 108
Enhancing Explainability of Neural Networks Through Architecture Constraints
Yang, Z., Zhang, A., & Sudjianto, A. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3007259 · Cited by 106
Interpretable confidence measures for decision support systems
van der Waa, J., Schoonderwoerd, T., van Diggelen, J., & Neerincx, M. A. (2020) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2020.102493 · Cited by 103
Hong, C. W., Lee, C., Lee, K., Ko, M.-S., Kim, D., & Hur, K. (2020) · Sensors · doi:10.3390/s20226626 · Cited by 100
(2018) · Human-computer interaction series · doi:10.1007/978-3-319-90403-0 · Cited by 98
Concept attribution: explaining cnn decisions to physicians
M., G., V., A., S., M.-M., & H., M. (2020) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.103865 · Cited by 95
An explainable multi-attribute decision model based on argumentation
Zhong, Q., Fan, X., Luo, X., & Toni, F. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.09.038 · Cited by 92
Towards Explainable Process Predictions for Industry 4.0 in the DFKI-Smart-Lego-Factory
Rehse, J.-R., Mehdiyev, N., & Fettke, P. (2019) · Künstliche Intell. · doi:10.1007/s13218-019-00586-1 · Cited by 90
Enabling Explainable Fusion in Deep Learning With Fuzzy Integral Neural Networks
Islam, M. A., Anderson, D. T., Pinar, A. J., Havens, T. C., Scott, G. J., & Keller, J. M. (2019) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2019.2917124 · Cited by 83
Self-citationDindorf, C., Teufl, W., Taetz, B., Bleser, G., & Fröhlich, M. (2020) · Sensors · doi:10.3390/s20164385 · Cited by 81
Le, T., Wang, S., & Lee, D. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403066 · Cited by 81
Human-Centric AI for Trustworthy IoT Systems With Explainable Multilayer Perceptrons
García‐Magariño, I., Rajarajan, M., & Lloret, J. (2019) · IEEE Access · doi:10.1109/access.2019.2937521 · Cited by 80
Vision-Based Fault Diagnostics Using Explainable Deep Learning With Class Activation Maps
Sun, K. H., Huh, H., Tama, B. A., Lee, S. Y., Jung, J. H., & Lee, S. (2020) · IEEE Access · doi:10.1109/access.2020.3009852 · Cited by 80
A method for explaining Bayesian networks for legal evidence with scenarios
Vlek, C. S., Prakken, H., Renooij, S., & Bart Verheij, A. (2016) · Artificial Intelligence and Law · doi:10.1007/s10506-016-9183-4 · Cited by 79
Explaining CNN and RNN Using Selective Layer-Wise Relevance Propagation
Jung, Y., Han, S. H., & Choi, H. (2021) · IEEE Access · doi:10.1109/access.2021.3051171 · Cited by 77
Interpretable Deep Convolutional Fuzzy Classifier
Yeganejou, M., Dick, S., & Miller, J. (2019) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2019.2946520 · Cited by 76
Random forest explainability using counterfactual sets
Fernández, R. R., Martín de Diego, I., Aceña, V., Fernández-Isabel, A., & Moguerza, J. M. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.07.001 · Cited by 73
Anguita‐Ruiz, A., Segura‐Delgado, A., Alcalá, R., Aguilera, C. M., & Alcalá‐Fdez, J. (2020) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1007792 · Cited by 73
Serradilla, O., Zugasti, E., Cernuda, C., Aranburu, A., de Okariz, J. R., & Zurutuza, U. (2020) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz48607.2020.9177537 · Cited by 68
Pintelas, E. G., Liaskos, M., Livieris, I. E., Kotsiantis, S. B., & Pintelas, P. (2020) · Journal of Imaging · doi:10.3390/jimaging6060037 · Cited by 67
Anysz, H., Brzozowski, Ł., Kretowicz, W., & Narloch, P. L. (2020) · Materials · doi:10.3390/ma13102317 · Cited by 64
Expert Level Evaluations for Explainable AI (XAI) Methods in the Medical Domain
Muddamsetty, S. M., Jahromi, M. N. S., & Moeslund, T. B. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68796-0_3 · Cited by 62
Generating and Understanding Personalized Explanations in Hybrid Recommender Systems
Kouki, P., Schaffer, J., Pujara, J., O’Donovan, J., & Getoor, L. C. (2020) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3365843 · Cited by 61
Accessible Cultural Heritage through Explainable Artificial Intelligence
Díaz-Rodríguez, N., & Pisoni, G. (2020) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3386392.3399276 · Cited by 58
An Explainable Artificial Intelligence Model for Clustering Numerical Databases
Loyola‐González, O., Gutierrez-Rodríguez, A. E., Medina‐Pérez, M. A., Monroy, R., Martínez-Trinidad, J. F., Carrasco-Ochoa, J. A., & García-Borroto, M. (2020) · IEEE Access · doi:10.1109/access.2020.2980581 · Cited by 55
Forestiero, A. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107241 · Cited by 54
De, T., Giri, P., Mevawala, A., Nemani, R., & Deo, A. (2020) · Procedia Computer Science · doi:10.1016/j.procs.2020.02.255 · Cited by 52
Ponn, T., Kröger, T., & Diermeyer, F. (2020) · Sensors · doi:10.3390/s20133699 · Cited by 52
Visual Explanation by Interpretation: Improving Visual Feedback Capabilities of Deep Neural NetworksPreprint
Jose Oramas, M., Wang, K., & Tuytelaars, T. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.06302 · Cited by 51
CASTLE: Cluster-aided space transformation for local explanations
Gatta, V. L., Moscato, V., Postiglione, M., & Sperlí, G. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115045 · Cited by 51
Sarp, S., Kuzlu, M., Cali, Ü., Elma, O., & Güler, Ö. (2021) · IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT) · doi:10.1109/isgt49243.2021.9372263 · Cited by 50
Kovalev, M. S., & Utkin, L. V. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.08.007 · Cited by 49
Interpretable and accurate prediction models for metagenomics data
Prifti, E., Chevaleyre, Y., Hanczar, B., Belda, E., Danchin, A., Clément, K., & Zucker, J. (2020) · GigaScience · doi:10.1093/gigascience/giaa010 · Cited by 47
A one-class classification decision tree based on kernel density estimation
Itani, S., Lecron, F., & Fortemps, P. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106250 · Cited by 45
Biswas, R., Barz, M., & Sonntag, D. (2020) · Künstliche Intell. · doi:10.1007/s13218-020-00679-2 · Cited by 45
Agents-Based Algorithm for a Distributed Information System in Internet of Things
Forestiero, A., & Papuzzo, G. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3074830 · Cited by 43
Explaining Deep Learning-Based Driver Models
Sesmero, M. P., Magán, E., Alvarez-Florez, L., Ledezma, A., Iglesias, J. A., & Sanchis, A. (2021) · Applied Sciences · doi:10.3390/app11083321 · Cited by 41
Locally and globally explainable time series tweaking
Karlsson, I., Rebane, J. T., Papapetrou, P., & Gionis, A. (2019) · Knowledge and Information Systems · doi:10.1007/s10115-019-01389-4 · Cited by 40
Mitigating belief projection in explainable artificial intelligence via Bayesian teaching
Yang, S. C., Vong, W. K., Sojitra, R. B., Folke, T., & Shafto, P. (2021) · Scientific Reports · doi:10.1038/s41598-021-89267-4 · Cited by 39
Predicting Heart Attack Through Explainable Artificial Intelligence
Aghamohammadi, M., Madan, M., Hong, J. K., & Watson, I. D. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-22741-8_45 · Cited by 38
Joint Banknote Recognition and Counterfeit Detection Using Explainable Artificial Intelligence
Han, M., & Kim, J. (2019) · Sensors · doi:10.3390/s19163607 · Cited by 37
Helbich, M., Hagenauer, J., Leitner, M., & Edwards, R. (2013) · Cartography and Geographic Information Science · doi:10.1080/15230406.2013.779780 · Cited by 36
Knowing the unknown: visualising consumption blind-spots in recommender systems
Tintarev, N., Rostami, S., & Smyth, B. (2018) · ACM Symposium on Applied Computing · doi:10.1145/3167132.3167419 · Cited by 35
Automated Failure Diagnosis in Aviation Maintenance Using Explainable Artificial Intelligence (XAI)
ten Zeldam, S., de Jong, A., Loendersloot, R., & Tinga, T. (2018) · PHM Society European Conference · doi:10.36001/phme.2018.v4i1.432 · Cited by 35
Understanding the decisions of CNNs: An in-model approach
Rio-Torto, I., Fernandes, K., & Teixeira, L. F. (2020) · Pattern Recognition Letters · doi:10.1016/j.patrec.2020.04.004 · Cited by 35
Holzinger, A., Kieseberg, P., Tjoa, A., & Weippl, E. (2018) · International Cross-Domain Conference on Machine Learning and Knowledge Extraction · doi:10.1007/978-3-319-99740-7 · Cited by 35
Chou, Y.-H., Hong, S., Zhou, Y., Shang, J., Song, M., & Li, H. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.07.045 · Cited by 32
Schorr, C., Goodarzi, P., Chen, F., & Dahmen, T. (2021) · Applied Sciences · doi:10.3390/app11052199 · Cited by 32
A Visual Analysis Approach for Understanding Durability Test Data of Automotive Products
Zhao, Y., Wang, L., Li, S., Zhou, F., Lin, X., Lü, Q., & Ren, L. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3345640 · Cited by 28
Senatore, R., Cioppa, A. D., & Marcelli, A. (2019) · Information · doi:10.3390/info10010030 · Cited by 27
Building Explanations for Fuzzy Decision Trees with the ExpliClas Software
Alonso, J. M., Ducange, P., Pecori, R., & Vilas, R. (2020) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz48607.2020.9177725 · Cited by 27
Investigating Textual Case-Based XAI
Weber, R. O., Johs, A. J., Li, J., & Huang, K. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01081-2_29 · Cited by 26
On Explainable Fuzzy Recommenders and their Performance Evaluation
Rutkowski, T., Łapa, K., & Nielek, R. (2019) · International Journal of Applied Mathematics and Computer Science · doi:10.2478/amcs-2019-0044 · Cited by 25
Lamy, J.-B., Sedki, K., & Tsopra, R. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103407 · Cited by 25
Porto, R., Molina, J. M., Berlanga, A., & Patricio, M. A. (2021) · Applied Sciences · doi:10.3390/app11031285 · Cited by 25
Learning Fuzzy Relations and Properties for Explainable Artificial Intelligence
Pierrard, R., Poli, J.-P., & Hudelot, C. (2018) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz-ieee.2018.8491538 · Cited by 23
Explaining VQA predictions using visual grounding and a knowledge base
Riquelme, F., Goyeneche, A. D., Zhang, Y., Niebles, J. C., & Soto, Á. (2020) · Image and Vision Computing · doi:10.1016/j.imavis.2020.103968 · Cited by 23
Advances and Applications in Deep Learning
Aceves-Fernández, M. A. (2020) · IntechOpen eBooks · doi:10.5772/intechopen.87786 · Cited by 23
Explanations for Temporal Recommendations
Bharadhwaj, H., & Joshi, S. (2018) · Künstliche Intell. · doi:10.1007/s13218-018-0560-x · Cited by 21
gbt-HIPS: Explaining the Classifications of Gradient Boosted Tree Ensembles
Hatwell, J., Gaber, M. M., & Azad, R. M. A. (2021) · Applied Sciences · doi:10.3390/app11062511 · Cited by 21
Loyola‐González, O. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33749-0_12 · Cited by 20
A Novel Decomposing Model With Evolutionary Algorithms for Feature Selection in Long Non-Coding RNAs
Bonidia, R. P., Machida, J. S., Negri, T. D. C., Alves, W. A. L., Kashiwabara, A. Y., Domingues, D. S., de Carvalho, A. C. P. L. F., Paschoal, A. R., & Sanches, D. S. (2020) · IEEE Access · doi:10.1109/access.2020.3028039 · Cited by 20
An Information-Theoretic Approach to Personalized Explainable Machine Learning
Jung, A., & Nardelli, P. H. J. (2020) · IEEE Signal Processing Letters · doi:10.1109/lsp.2020.2993176 · Cited by 19
A Combinatorial Approach to Explaining Image Classifiers
Chandrasekaran, J., Lei, Y., Kacker, R. N., & Kuhn, D. R. (2021) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw52544.2021.00019 · Cited by 19
Ferreyra, E., Hagras, H., Kern, M., & Owusu, G. (2019) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz-ieee.2019.8858933 · Cited by 18
Middle-Level Features for the Explanation of Classification Systems by Sparse Dictionary Methods
Apicella, A., Isgrò, F., Prevete, R., & Tamburrini, G. (2020) · International Journal of Neural Systems · doi:10.1142/s0129065720500409 · Cited by 18
Constrained Interval Type-2 Fuzzy Classification Systems for Explainable AI (XAI)
D'Alterio, P., Garibaldi, J. M., & John, R. I. (2020) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz48607.2020.9177671 · Cited by 18
Healthwalks: sensing fine-grained individual health condition via mobility data
Lin, Z., Lyu, S., Cao, H., Xu, F., Wei, Y., Samet, H., & Li, Y. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3432229 · Cited by 18
How to Improve Object Detection in a Driver Assistance System Applying Explainable Deep Learning
Nowak, T., Nowicki, M., Ćwian, K., & Skrzypczyński, P. (2019) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2019.8814134 · Cited by 17
Learning Explainable Decision Rules via Maximum Satisfiability
Cao, H. E. C., Sarlin, R., & Jung, A. (2020) · IEEE Access · doi:10.1109/access.2020.3041040 · Cited by 17
Huang, L., Yeung, W., Wang, Y., Cheng, H., Venkat, A., Li, S., Ma, P., Rasheed, K., & Kannan, N. (2020) · BMC Bioinformatics · doi:10.1186/s12859-020-03842-6 · Cited by 16
Schönhof, R., Werner, A., Elstner, J., Zopcsak, B., Awad, R., & Huber, M. F. (2021) · Procedia CIRP · doi:10.1016/j.procir.2021.05.075 · Cited by 16
Alonso, J. M., Toja-Alamancos, J., & Bugarin, A. (2020) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz48607.2020.9177770 · Cited by 15
Artificial Intelligence, Machine Learning and Reasoning in Health Informatics—Case Studies
Ahmed, M. U., Barua, S., & Begum, S. (2020) · Intelligent systems reference library · doi:10.1007/978-3-030-54932-9_12 · Cited by 15
Self-citationA Visualisation Tool to Explain Case-Base Reasoning Solutions for Tablet Formulation
Massie, S., Craw, S., & Wiratunga, N. (2007) · Springer eBooks · doi:10.1007/1-84628-103-2_16 · Cited by 14
Martínez-Cebrián, J., Fernández‐Torres, M., & Díaz-de-María, F. (2020) · IEEE Access · doi:10.1109/access.2020.3041606 · Cited by 14
PASTLE: Pivot-aided space transformation for local explanations
Gatta, V. L., Moscato, V., Postiglione, M., & Sperlí, G. (2021) · Pattern Recognition Letters · doi:10.1016/j.patrec.2021.05.018 · Cited by 13
Designing interpretable Hierarchical Fuzzy Systems
Magdalena, L. (2018) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz-ieee.2018.8491452 · Cited by 12
Exploration of Explainable AI in Context of Human-Machine Interface for the Assistive Driving System
Chaczko, Z., Kulbacki, M., Gudzbeler, G., Alsawwaf, M., Thai-Chyzhykau, I., & Wajs-Chaczko, P. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-42058-1_42 · Cited by 12
Locality Guided Neural Networks for Explainable Artificial Intelligence
Tan, R., Khan, N., & Guan, L. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9207559 · Cited by 12
Explainable Internet Traffic Classification
Callegari, C., Ducange, P., Fazzolari, M., & Vecchio, M. (2021) · Applied Sciences · doi:10.3390/app11104697 · Cited by 12
(2019) · Lecture notes in computer science · doi:10.1007/978-3-030-23541-3 · Cited by 12
Multi-operator Decision Trees for Explainable Time-Series Classification
Shalaeva, V., Alkhoury, S., Marinescu, J., Amblard, C., & Bisson, G. (2018) · Communications in computer and information science · doi:10.1007/978-3-319-91473-2_8 · Cited by 11
Eisenstadt, V., Espinoza-Stapelfeld, C., Mikyas, A., & Althoff, K. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01081-2_6 · Cited by 11
Information Fusion-2-Text: Explainable Aggregation via Linguistic Protoforms
Murray, B., Anderson, D. T., Havens, T. C., et al. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-50153-2_9 · Cited by 10
Hu, Z., & Beyeler, M. (2021) · International IEEE/EMBS Conference on Neural Engineering · doi:10.1109/ner49283.2021.9441194 · Cited by 9
Explainable Artificial Intelligence for Falls Prediction
Lindsay, L., Coleman, S., Kerr, D., Taylor, B. J., & Moorhead, A. (2020) · Communications in computer and information science · doi:10.1007/978-981-15-6634-9_8 · Cited by 8
Explainable Hopfield Neural Networks Using an Automatic Video-Generation System
Rubio-Manzano, C., Segura-Navarrete, A., Martínez-Araneda, C., & Vidal‐Castro, C. (2021) · Applied Sciences · doi:10.3390/app11135771 · Cited by 8
Salient Explanation for Fine-Grained Classification
Oh, K., Kim, S., & Oh, I.-S. (2020) · IEEE Access · doi:10.1109/access.2020.2980742 · Cited by 7
Reinterpreting Interpretability for Fuzzy Linguistic Descriptions of Data
Ramos-Soto, A., & Pereira-Fariña, M. (2018) · Communications in computer and information science · doi:10.1007/978-3-319-91473-2_4 · Cited by 6
Semantically Explainable Fuzzy Classifier
Sabol, P., Sinčák, P., Magyar, J., & Hartono, P. (2019) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001420510064 · Cited by 6
Explainable Deep Convolutional Candlestick LearnerPreprint
Chen, J., Chen, S. Y.-C., Tsai, Y., & Shur, C.-S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.02767 · Cited by 6
XAI-Based Fuzzy SWOT Maps for Analysis of Complex Systems
Meskauskas, Z., Jasinevičius, R., Kazanavičius, E., & Petrauskas, V. (2020) · IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) · doi:10.1109/fuzz48607.2020.9177792 · Cited by 5
Cited by
Explainable AI for Healthcare 5.0: Opportunities and Challenges
Saraswat, D., Bhattacharya, P., Verma, A., Prasad, V. K., Tanwar, S., Sharma, G., Bokoro, P. N., & Sharma, R. (2022) · IEEE Access · doi:10.1109/access.2022.3197671 · Cited by 523
A., S., & R., S. (2023) · Decision Analytics Journal · doi:10.1016/j.dajour.2023.100230 · Cited by 432
Explainable Artificial Intelligence in CyberSecurity: A Survey
Capuano, N., Fenza, G., Loia, V., & Stanzione, C. (2022) · IEEE Access · doi:10.1109/access.2022.3204171 · Cited by 330
“In [24] are examined and presented several XAI approaches, validation measures, and the types of explana-360 tions that can be generated to improve the acceptance of expert systems among general users.”
BackgroundWeber, P., Carl, K. V., & Hinz, O. (2024) · Management Review Quarterly · doi:10.1007/s11301-023-00320-0 · Cited by 292
“Our research offers broad guidance for scientists to understand the growing emphasis on XAI in Finance research, which we also found in other domains (Wells and Bednarz 2021; Islam et al. 2022).”
XAIR: A Systematic Metareview of Explainable AI (XAI) Aligned to the Software Development Process
Clement, T., Kemmerzell, N., Abdelaal, M., & Amberg, M. (2023) · Machine Learning and Knowledge Extraction · doi:10.3390/make5010006 · Cited by 131
Metadata
- Authors
- Islam, M. R., Ahmed, M. U., Barua, S., & Begum, S.
- Venue
- Applied Sciences
- Publisher
- MDPI AG
- Published
- 2022-01-27
- Volume
- 12
- Issue
- 3
- Pages
- 1353
- ISSN
- 2076-3417 (ISSN-L)
- OpenAlex
- W4210361514
- Cited by
- 311
- References
- 176
- FWCI
- 5.46
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.