Making deep neural networks right for the right scientific reasons by interacting with their explanations
Open accessThe claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Subjective trust in the AI system was assessed using the Trust in Automation questionnaire (TiA), a commonly employed and validated tool for measuring trust.
“The subjective trust in the AI system was assessed using the Trust in Automation questionnaire (TiA) [63], which is a commonly employed and validated tool for measuring trust [78, 116, 120].”
From Dealing with Uncertainty: Understanding the Impact of Prognostic Versus Diagnostic Tasks on Trust and Reliance in Human-AI Decision Making (2024) · cites this paper as [116]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421 · see citing works
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
(2008) · Journal of Machine Learning Research · Cited by 33,051 · see citing works
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428 · see citing works
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561 · see citing works
A tutorial on spectral clustering
von Luxburg, U. (2007) · Statistics and Computing · doi:10.1007/s11222-007-9033-z · Cited by 10,519 · see citing works
Machine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955 · see citing works
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084 · see citing works
Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
Xiao, H., Rasul, K., & Vollgraf, R. (2017) · arXiv · doi:10.48550/arxiv.1708.07747 · Cited by 6,108 · see citing works
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507 · see citing works
Buciluǎ, C., Caruana, R., & Niculescu-Mizil, A. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150464 · Cited by 2,068 · see citing works
Probabilistic machine learning and artificial intelligence
Ghahramani, Z. (2015) · Nature · doi:10.1038/nature14541 · Cited by 1,995 · see citing works
Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., & Oermann, E. K. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002683 · Cited by 1,719 · see citing works
Sanity Checks for Saliency Maps
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., & Kim, B. (2018) · arXiv · doi:10.48550/arxiv.1810.03292 · Cited by 1,294 · see citing works
Settles, B. (2012) · Cited by 1,183 · see citing works
Unmasking Clever Hans predictors and assessing what machines really learn
Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., & Müller, K.-R. (2019) · Nature Communications · doi:10.1038/s41467-019-08987-4 · Cited by 1,047 · see citing works
Deep Bayesian Active Learning with Image Data
Gal, Y., Islam, R., & Ghahramani, Z. (2017) · arXiv · doi:10.48550/arxiv.1703.02910 · Cited by 900 · see citing works
This Looks Like That: Deep Learning for Interpretable Image Recognition
Chen, C., Li, O., Tao, C., Barnett, A. J., Su, J., & Rudin, C. (2018) · arXiv · doi:10.48550/arxiv.1806.10574 · Cited by 702 · see citing works
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · see citing works
Plant Phenomics, From Sensors to Knowledge
Tardieu, F., Cabrera‐Bosquet, L., Pridmore, T., & Bennett, M. J. (2017) · Current Biology · doi:10.1016/j.cub.2017.05.055 · Cited by 628 · see citing works
Settles, B. (2012) · Synthesis Lectures on Artificial Intelligence and Machine Learning · doi:10.2200/s00429ed1v01y201207aim018 · Cited by 605 · see citing works
Unmasking Clever Hans predictors and assessing what machines really learn
Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., & Müller, K. (2019) · RePEc: Research Papers in Economics · Cited by 580 · see citing works
Principles of Explanatory Debugging to Personalize Interactive Machine Learning
Kulesza, T., Burnett, M., Wong, W.-K., & Stumpf, S. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701399 · Cited by 576 · see citing works
Psychological Foundations of Trust
Simpson, J. A. (2007) · Current Directions in Psychological Science · doi:10.1111/j.1467-8721.2007.00517.x · Cited by 541 · see citing works
Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations
Ross, A. S., Hughes, M. C., & Doshi-Velez, F. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/371 · Cited by 515 · see citing works
A Benchmark for Interpretability Methods in Deep Neural NetworksPreprint
Hooker, S., Erhan, D., Kindermans, P.-J., & Kim, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.10758 · Cited by 438 · see citing works
Multimodal Explanations: Justifying Decisions and Pointing to the Evidence
Park, D. H., Hendricks, L. A., Akata, Z., Rohrbach, A., Schiele, B., Darrell, T., & Rohrbach, M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00915 · Cited by 418 · see citing works
Deep Bayesian Active Learning with Image Data
Ghahramani, Z., Gal, Y., & Islam, R. (2017) · PMLR · doi:10.17863/cam.11070 · Cited by 392 · see citing works
Deep machine learning provides state-of-the-art performance in image-based plant phenotyping
Pound, M. P., Atkinson, J. A., Townsend, A. J., Wilson, M. H., Griffiths, M., Jackson, A. S., Bulat, A., Tzimiropoulos, G., Wells, D. M., Murchie, E. H., et al. (2017) · GigaScience · doi:10.1093/gigascience/gix083 · Cited by 390 · see citing works
Theoretical Considerations and Development of a Questionnaire to Measure Trust in Automation
Körber, M. (2019) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-319-96074-6_2 · Cited by 317 · see citing works
Deep learning predicts hip fracture using confounding patient and healthcare variables
Badgeley, M. A., Zech, J. R., Oakden-Rayner, L., Glicksberg, B. S., Liu, M., Gale, W., McConnell, M. V., Percha, B., Snyder, T. M., & Dudley, J. T. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0105-1 · Cited by 290 · see citing works
Explanatory Interactive Machine Learning
Teso, S., & Kersting, K. (2019) · AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314293 · Cited by 276 · see citing works
Self-citationTaking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded
Selvaraju, R. R., Lee, S., Shen, Y., Jin, H., Ghosh, S., Heck, L. P., Batra, D., & Parikh, D. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00268 · Cited by 253 · see citing works
Using “annotator rationales” to improve machine learning for text categorization
Zaidan, O. F., Eisner, J. M., & Piatko, C. (2007) · Cited by 252 · see citing works
Hoffman, R. R., Johnson, M., Bradshaw, J. M., & Underbrink, A. J. (2013) · IEEE Intelligent Systems · doi:10.1109/mis.2013.24 · Cited by 238 · see citing works
Mochida, K., Koda, S., Inoue, K., Hirayama, T., Tanaka, S., Nishii, R., & Melgani, F. (2018) · GigaScience · doi:10.1093/gigascience/giy153 · Cited by 210 · see citing works
Theory of Disagreement-Based Active Learning
Hanneke, S. (2014) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000037 · Cited by 180 · see citing works
Explanations can be manipulated and geometry is to blame
Dombrowski, A.-K., Alber, M., Anders, C. J., Ackermann, M., Müller, K.-R., & Kessel, P. (2019) · arXiv · doi:10.48550/arxiv.1906.07983 · Cited by 171 · see citing works
Nonmyopic active learning of Gaussian processes
Krause, A., & Guestrin, C. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273553 · Cited by 168 · see citing works
Settles, B. (2011) · Cited by 167 · see citing works
Narayanan, M., Chen, E., He, J., Kim, B., Gershman, S., & Doshi-Velez, F. (2018) · arXiv · doi:10.48550/arxiv.1802.00682 · Cited by 153 · see citing works
The true sample complexity of active learning
Balcan, M.-F., Hanneke, S., & Vaughan, J. W. (2010) · Machine Learning · doi:10.1007/s10994-010-5174-y · Cited by 137 · see citing works
Interpreting Blackbox Models via Model ExtractionPreprint
Bastani, O., Kim, C., & Bastani, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08504 · Cited by 126 · see citing works
Mahlein, A., Kuśka, M. T., Thomas, S., Wahabzada, M., Behmann, J., Rascher, U., & Kersting, K. (2019) · Current Opinion in Plant Biology · doi:10.1016/j.pbi.2019.06.007 · Cited by 109 · see citing works
Self-citationIncorporating Priors with Feature Attribution on Text Classification
Liu, F., & Avci, B. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1631 · Cited by 104 · see citing works
An unexpected unity among methods for interpreting model predictionsPreprint
Lundberg, S., & Lee, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.07478 · Cited by 101 · see citing works
Learning Explainable Models Using Attribution Priors
Erion, G., Janizek, J. D., Sturmfels, P., Lundberg, S., & Lee, S. (2019) · Cited by 53 · see citing works
Shivaswamy, P., & Joachims, T. (2015) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4539 · Cited by 52 · see citing works
Short-Term Satisfaction and Long-Term Coverage
Schnabel, T., Bennett, P. N., Dumais, S., & Joachims, T. (2018) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3159652.3159700 · Cited by 47 · see citing works
The Constrained Weight Space SVM: Learning with Ranked Features
Small, K., Wallace, B., Trikalinos, T. A., & Brodley, C. E. (2011) · Cited by 39 · see citing works
Learning to Explain With Complemental Examples
Kanehira, A., & Harada, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00880 · Cited by 39 · see citing works
Active Imitation Learning via Reduction to I.I.D. Active LearningPreprint
Judah, K., Fern, A., & Dietterich, T. G. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1210.4876 · Cited by 32 · see citing works
Confounding variables can degrade generalization performance of radiological deep learning models
Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., & Oermann, E. K. (2018) · arXiv · doi:10.48550/arxiv.1807.00431 · Cited by 26 · see citing works
How and What Can Humans Learn from Being in the Loop?
Abdel-Karim, B. M., Pfeuffer, N., Rohde, G. G. U., & Hinz, O. F. (2020) · Künstliche Intell. · doi:10.1007/s13218-020-00638-x · Cited by 20 · see citing works
Human-Guided Learning for Probabilistic Logic Models
Odom, P., & Natarajan, S. (2018) · Frontiers in Robotics and AI · doi:10.3389/frobt.2018.00056 · Cited by 15 · see citing works
A Permutation Approach to Assess Confounding in Machine Learning Applications for Digital Health
Neto, E. C., Pratap, A., Perumal, T. M., Tummalacherla, M., Bot, B. M., Mangravite, L. M., & Omberg, L. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330903 · Cited by 13 · see citing works
de Souza, N. (2009) · Nature Methods · doi:10.1038/nmeth.f.289 · Cited by 10 · see citing works
Analyzing ImageNet with Spectral Relevance Analysis: Towards ImageNet un-Hans'ed.Preprint
Anders, C. J., Marinč, T., Neumann, D., Samek, W., Müller, K., & Lapuschkin, S. (2019) · arXiv (Cornell University) · Cited by 10 · see citing works
High-throughput phenotyping of rice growth traits
Lau, E. (2014) · Nature Reviews Genetics · doi:10.1038/nrg3857 · Cited by 2 · see citing works
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244302760185243
Flagged first, then by citations
3 of the 68 references couldn’t be matched to a known paper.
Research works printed but not linked: 6
- [13][13] Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J. & Zisserman, A. The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results. http://www.pascalnetwork.org/challenges/VOC/voc2007/workshop/index.html. -
- [15][15] Herbert, F. P., Kersting, K. & J¨ akel, F. Why Should I Trust in AI? Master's thesis, Technical University Darmstadt (2019). -
- [19][19] Cakmak, M. et al. Mixed-initiative active learning. ICML 2011 Workshop on Combining Learning Strategies to Reduce Label Cost (2011). -
- [40][40] Roy, N. et al. Toward optimal active learning through monte carlo estimation of error reduction. ICML 441-448 (2001). -
- [41][41] Castro, R. M. et al. Upper and lower error bounds for active learning. In Proceedings of Conference on Communication, Control and Computing , 2.1, 1 (2006). -
- [43][43] Tong, S. & Koller, D. Support vector machine active learning with applications to text classification. Journal of machine learning research 2 , 45-66 (2001). -
Cited by
Sorted by citations
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762 · see citing works
Interpretable machine learning: Fundamental principles and 10 grand challenges
Rudin, C., Chen, C., Chen, Z., Huang, H., Semenova, L., & Zhong, C. (2022) · Statistics Surveys · doi:10.1214/21-ss133 · Cited by 958 · see citing works
Erion, G., Janizek, J. D., Sturmfels, P., Lundberg, S. M., & Lee, S.-I. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00343-w · Cited by 300 · see citing works
Large pre-trained language models contain human-like biases of what is right and wrong to do
Schramowski, P., Turan, C., Andersen, N., Rothkopf, C. A., & Kersting, K. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00458-8 · Cited by 291 · see citing works
Self-citationNazir, S., Dickson, D. M., & Akram, M. U. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.106668 · Cited by 283 · see citing works
“The focus shifted to correcting the underlying reasons for a wrong model decision which can aid in understanding the confounding factors in image classification [259].”
BackgroundInfluentialA Review on Explainability in Multimodal Deep Neural Nets
Joshi, G., Walambe, R., & Kotecha, K. (2021) · IEEE Access · doi:10.1109/access.2021.3070212 · Cited by 204 · see citing works
From attribution maps to human-understandable explanations through Concept Relevance Propagation
Achtibat, R., Dreyer, M., Eisenbraun, I., Bosse, S., Wiegand, T., Samek, W., & Lapuschkin, S. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00711-8 · Cited by 181 · see citing works
Explainable Generative AI (GenXAI): a survey, conceptualization, and research agenda
Schneider, J. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10916-x · Cited by 184 · see citing works
Barnett, A. J., Schwartz, F. R., Tao, C., Chen, C., Ren, Y., Lo, J. Y.-C., & Rudin, C. D. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00423-x · Cited by 162 · see citing works
Concepts and applications of digital twins in healthcare and medicine
Zhang, K., Zhou, H.-Y., Baptista‐Hon, D. T., Gao, Y., Liu, X., Oermann, E. K., Xu, S. H., Jin, S., Zhang, J., Sun, Z., et al. (2024) · Patterns · doi:10.1016/j.patter.2024.101028 · Cited by 170 · see citing works
Xu, R., Cao, J., Feng, G., Luo, J., Feng, Q., Ni, B., & Fang, F. (2021) · Chemical Engineering Journal · doi:10.1016/j.cej.2021.132893 · Cited by 146 · see citing works
Beyond explaining: Opportunities and challenges of XAI-based model improvement
Weber, L., Lapuschkin, S., Binder, A., & Samek, W. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.11.013 · Cited by 148 · see citing works
Can language models learn from explanations in context?
Lampinen, A., Dasgupta, I., Chan, S., Mathewson, K., Tessler, M., Creswell, A., McClelland, J., Wang, J., & Hill, F. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.38 · Cited by 142 · see citing works
Finding and removing Clever Hans: Using explanation methods to debug and improve deep models
Anders, C. J., Weber, L., Neumann, D., Samek, W., Müller, K., & Lapuschkin, S. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.07.015 · Cited by 122 · see citing works
Knowing About Knowing: An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systems
He, G., Kuiper, L., & Gadiraju, U. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581025 · Cited by 107 · see citing works
“For a deeper analysis of our results, a number of additional measures were considered based on observations from existing literature [49, 67, 76]:”
MethodExplainable deep learning in plant phenotyping
Mostafa, S., Mondal, D., Panjvani, K., Kochian, L. V., & Stavness, I. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1203546 · Cited by 95 · see citing works
EnD: Entangling and Disentangling deep representations for bias correction
Tartaglione, E., Barbano, C. A., & Grangetto, M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01330 · Cited by 85 · see citing works
Garrett, K. A., Bebber, D. P., Etherton, B. A., Gold, K. M., Sulá, A. I. P., & Selvaraj, M. G. (2022) · Annual Review of Phytopathology · doi:10.1146/annurev-phyto-021021-042636 · Cited by 77 · see citing works
Right for the Right Concept: Revising Neuro-Symbolic Concepts by Interacting with their Explanations
Stammer, W., Schramowski, P., & Kersting, K. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00362 · Cited by 73 · see citing works
Self-citationGoing Beyond XAI: A Systematic Survey for Explanation-Guided Learning
Gao, Y., Gu, S., Jiang, J., Hong, S.-S., Yu, D., & Zhao, L. (2024) · ACM Computing Surveys · doi:10.1145/3644073 · Cited by 71 · see citing works
Shen, C., Wang, R., Nawazish, H., Wang, B., Cai, K., & Xu, B. (2024) · Comprehensive Reviews in Food Science and Food Safety · doi:10.1111/1541-4337.70054 · Cited by 74 · see citing works
Data management challenges for artificial intelligence in plant and agricultural research
Williamson, H. F., Brettschneider, J. A., Cáccamo, M., Davey, R., Goble, C., Kersey, P. J., May, S. T., Morris, R. J., Ostler, R., Pridmore, T., et al. (2023) · F1000Research · doi:10.12688/f1000research.52204.2 · Cited by 52 · see citing works
Leveraging explanations in interactive machine learning: An overview
Teso, S., Alkan, Ö., Stammer, W., & Daly, E. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1066049 · Cited by 55 · see citing works
Self-citationExplanation-Based Human Debugging of NLP Models: A Survey
Lertvittayakumjorn, P., & Toni, F. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00440 · Cited by 51 · see citing works
Visualization for Recommendation Explainability: A Survey and New Perspectives
Chatti, M. A., Guesmi, M., & Muslim, A. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3672276 · Cited by 56 · see citing works
Mahlein, A., Barbedo, J. G. A., Chiang, K., Ponte, E. M. D., & Bock, C. H. (2024) · Phytopathology · doi:10.1094/phyto-01-24-0009-per · Cited by 53 · see citing works
Self-citationSalimzadeh, S., He, G., & Gadiraju, U. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641905 · Cited by 41 · see citing works
“The subjective trust in the AI system was assessed using the Trust in Automation questionnaire (TiA) [63], which is a commonly employed and validated tool for measuring trust [78, 116, 120].”
MethodPotentials and caveats of AI in hybrid imaging
Sundar, L. K. S., Muzik, O., Buvat, I., Bidaut, L. M., & Beyer, T. (2020) · Methods · doi:10.1016/j.ymeth.2020.10.004 · Cited by 40 · see citing works
Digital plant pathology: a foundation and guide to modern agriculture
Kuśka, M. T., Heim, R. H. J., Geedicke, I., Gold, K. M., Brugger, A., & Paulus, S. (2022) · Journal of Plant Diseases and Protection · doi:10.1007/s41348-022-00600-z · Cited by 38 · see citing works
Self-citationEXMOS: Explanatory Model Steering through Multifaceted Explanations and Data Configurations
Bhattacharya, A., Stumpf, S., Gosak, L., Štiglic, G., & Verbert, K. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642106 · Cited by 38 · see citing works
Explainable AI reveals Clever Hans effects in unsupervised learning models
Kauffmann, J., Dippel, J., Ruff, L., Samek, W., Müller, K., & Montavon, G. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01000-2 · Cited by 37 · see citing works
Towards Trustable Skin Cancer Diagnosis via Rewriting Model's Decision
Yan, S., Yu, Z. T., Zhang, X., Mahapatra, D., Chandra, S. S., Janda, M., Soyer, H. P., & Ge, Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01113 · Cited by 36 · see citing works
An Empirical Evaluation of the Rashomon Effect in Explainable Machine Learning
Müller, S., Toborek, V., Beckh, K., Jakobs, M., Bauckhage, C., & Welke, P. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43418-1_28 · Cited by 35 · see citing works
Adversarial Attacks in Explainable Machine Learning: A Survey of Threats Against Models and Humans
Vadillo, J., Santana, R., & Lozano, J. A. (2024) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1567 · Cited by 38 · see citing works
A graph-based interpretability method for deep neural networks
Wang, T., Zheng, X., Zhang, L., Cui, Z., & Xu, C. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126651 · Cited by 34 · see citing works
A Roadmap of Explainable Artificial Intelligence: Explain to Whom, When, What and How?
Wang, Z., Huang, C., & Yao, X. (2024) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3702004 · Cited by 32 · see citing works
Explainable Machine Learning with Prior Knowledge: An Overview
Beckh, K., Müller, S., Jakobs, M., Toborek, V., Tan, H., Fischer, R., Welke, P., Houben, S., & von Rueden, L. (2021) · Lancaster EPrints (Lancaster University) · doi:10.48550/arxiv.2105.10172 · Cited by 40 · see citing works
Happiness Prediction With Domain Knowledge Integration and Explanation Consistency
Wu, X., Li, L., Tao, X., Xing, F., & Yuan, J. (2025) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2025.3529946 · Cited by 29 · see citing works
Mumuni, F., & Mumuni, A. (2023) · Cognitive Systems Research · doi:10.1016/j.cogsys.2023.101188 · Cited by 28 · see citing works
Stadtler, S., Betancourt, C., & Roscher, R. (2022) · Machine Learning and Knowledge Extraction · doi:10.3390/make4010008 · Cited by 28 · see citing works
Deep Neural Networks Based Approach for Battery Life Prediction
Bhattacharya, S., Maddikunta, P. K. R., Iyapparaja, M., Gadekallu, T. R., Sharma, S., Alkahtani, M. S., & Abidi, M. H. (2021) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2021.016229 · Cited by 28 · see citing works
Explanatory Interactive Machine Learning
Pfeuffer, N., Baum, L., Stammer, W., Abdel-Karim, B. M., Schramowski, P., Bucher, A. M., Hügel, C., Rohde, G. G. U., Kersting, K., & Hinz, O. F. (2023) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00806-x · Cited by 30 · see citing works
Self-citationDreyer, M., Achtibat, R., Wiegand, T., Samek, W., & Lapuschkin, S. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00397 · Cited by 28 · see citing works
Climate Change Impact on Plant Pathogen Emergence: Artificial Intelligence (AI) Approach
Ali, F., Rehman, A., Hameed, A., Sarfraz, S., Rajput, N. A., & Atiq, M. (2024) · Springer eBooks · doi:10.1007/978-3-031-56011-8_9 · Cited by 26 · see citing works
Brugger, A., Schramowski, P., Paulus, S., Steiner, U., Kersting, K., & Mahlein, A. (2021) · Plant Pathology · doi:10.1111/ppa.13411 · Cited by 29 · see citing works
Self-citationRenouard, A., Maggi, A., Grunberg, M., Doubre, C., & Hibert, C. (2021) · Seismological Research Letters · doi:10.1785/0220200305 · Cited by 26 · see citing works
Luo, L., Chen, H., Xiao, Y., Zhou, Y., Wang, X., Vardhanabhuti, V., Wu, M., Han, C., Liu, Z., Fang, B., et al. (2022) · Radiology Artificial Intelligence · doi:10.1148/ryai.210299 · Cited by 29 · see citing works
Brugger, A., Yamati, F. R. I., Barreto, A., Paulus, S., Schramowsk, P., Kristian Kersting, B., Steiner, U., Neugart, S., & Mahlein, A. (2022) · Phytopathology · doi:10.1094/phyto-03-22-0086-r · Cited by 26 · see citing works
Self-citationXu, R., Cao, J.-S., Luo, J., Ni, B., Fang, F., Liu, W., & Wang, P. (2024) · npj Clean Water · doi:10.1038/s41545-024-00367-w · Cited by 25 · see citing works
Data management challenges for artificial intelligence in plant and agricultural research
Williamson, H. F., Brettschneider, J. A., Cáccamo, M., Davey, R., Goble, C., Kersey, P. J., May, S. T., Morris, R. J., Ostler, R., Pridmore, T., et al. (2021) · F1000Research · doi:10.12688/f1000research.52204.1 · Cited by 24 · see citing works
Djimeli-Tsajio, A., Noulamo, T., Jean-Pierre, L. T., Kapche, T. F., & Nagabhushan, P. N. (2022) · Journal of Plant Diseases and Protection · doi:10.1007/s41348-022-00608-5 · Cited by 23 · see citing works
ICICLE: Interpretable Class Incremental Continual Learning
Rymarczyk, D., van de Weijer, J., Zieliński, B., & Twardowski, B. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00181 · Cited by 25 · see citing works
Dhar, S., & Shamir, L. (2022) · Astronomy and Computing · doi:10.1016/j.ascom.2022.100545 · Cited by 23 · see citing works
Prompt-Driven Latent Domain Generalization for Medical Image Classification
Yan, S., Yu, Z. T., Liu, C., Ju, L., Mahapatra, D., Betz‐Stablein, B., Mar, V., Janda, M., Soyer, H. P., & Ge, Z. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3443119 · Cited by 22 · see citing works
Harnessing Prior Knowledge for Explainable Machine Learning: An Overview
Beckh, K., Müller, S., Jakobs, M., Toborek, V., Tan, H., Fischer, R., Welke, P., Houben, S., & von Rueden, L. (2023) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00038 · Cited by 22 · see citing works
Reveal to Revise: An Explainable AI Life Cycle for Iterative Bias Correction of Deep Models
Pahde, F., Dreyer, M., Samek, W., & Lapuschkin, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43895-0_56 · Cited by 21 · see citing works
Multiview Shapelet Prototypical Network for Few-Shot Fault Incremental Learning
Wan, X., Cen, L., Chen, X., Xie, Y., & Gui, W. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3413304 · Cited by 20 · see citing works
Towards Guidelines for Designing Human-in-the-Loop Machine Training Interfaces
van der Stappen, A., & Funk, M. (2021) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3397481.3450668 · Cited by 20 · see citing works
Interactive Disentanglement: Learning Concepts by Interacting with their Prototype Representations
Stammer, W., Memmel, M., Schramowski, P., & Kristian Kersting, B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01007 · Cited by 19 · see citing works
Self-citationExplainable deep learning: concepts, methods, and new developments
Samek, W. (2023) · Elsevier eBooks · doi:10.1016/b978-0-32-396098-4.00008-9 · Cited by 19 · see citing works
Pseudo-class part prototype networks for interpretable breast cancer classification
Choukali, M. A., Amirani, M. C., Valizadeh, M., Abbasi, A., & Komeili, M. (2024) · Scientific Reports · doi:10.1038/s41598-024-60743-x · Cited by 18 · see citing works
Decision Boundary Visualization for Counterfactual Reasoning
Sohns, J., Garth, C., & Leitte, H. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14650 · Cited by 17 · see citing works
A typology for exploring the mitigation of shortcut behaviour
Friedrich, F., Stammer, W., Schramowski, P., & Kersting, K. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00612-w · Cited by 18 · see citing works
Self-citationGranDIHC-BR 2025-2035 - GC5 - Human-Data Interaction Data Literacy and Usable Privacy
Coleti, T. A., Divino, S. B. S., Salgado, A. D. L., Zacarias, R. O., Saraiva, J. D. A. G., Gonçalves, D. A., Morandini, M., & dos Santos, R. P. (2024) · Brazilian Symposium on Human Factors in Computing Systems (IHC) · doi:10.1145/3702038.3702058 · Cited by 17 · see citing works
To Err Is AI! Debugging as an Intervention to Facilitate Appropriate Reliance on AI Systems
He, G., Bharos, A., & Gadiraju, U. (2024) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3648188.3675130 · Cited by 17 · see citing works
Dublino, R., & Ercolano, M. R. (2025) · The Crop Journal · doi:10.1016/j.cj.2025.05.008 · Cited by 16 · see citing works
Ayorinde, J., Citterio, F., Landrò, M., Peruzzo, E., Islam, T., Tilley, S., Taylor, G., Bardsley, V., Lió, P., Samoshkin, A., et al. (2022) · Journal of the American Society of Nephrology · doi:10.1681/asn.2022010069 · Cited by 16 · see citing works
Navigating the landscape of concept-supported XAI: Challenges, innovations, and future directions
Khoozani, Z. S., Sabri, A. Q. M., Seng, W. C., Seera, M., & Eg, K. Y. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17666-y · Cited by 18 · see citing works
Yang, J., Sun, Z., Tian, S., Jiang, H., Feng, J., Ting, K. C., Lin, T., & Ying, Y. B. (2024) · Postharvest Biology and Technology · doi:10.1016/j.postharvbio.2024.113009 · Cited by 16 · see citing works
Dreyer, M., Achtibat, R., Samek, W., & Lapuschkin, S. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00353 · Cited by 16 · see citing works
CAIPI in Practice: Towards Explainable Interactive Medical Image Classification
Slany, E., Ott, Y., Scheele, S., Paulus, J., & Schmid, U. (2022) · IFIP advances in information and communication technology · doi:10.1007/978-3-031-08341-9_31 · Cited by 16 · see citing works
Bock, C. H., Barbedo, J. G. A., Mahlein, A., & Ponte, E. M. D. (2022) · Tropical Plant Pathology · doi:10.1007/s40858-022-00498-w · Cited by 16 · see citing works
Self-citationBonse, M. J., Gebhard, T. D., Dannert, F., Absil, O., Cantalloube, F., Christiaens, V., Cugno, G., Garvin, E. O., Hayoz, J., Kasper, M. E., et al. (2025) · The Astronomical Journal · doi:10.3847/1538-3881/adab79 · Cited by 14 · see citing works
Enterprise imaging and big data: A review from a medical physics perspective
McCarthy, N., Dahlan, A., Cook, T. S., Hare, N. O., Ryan, M.-L., John, B. S., Lawlor, A., & Curran, K. M. (2021) · Physica Medica · doi:10.1016/j.ejmp.2021.04.004 · Cited by 14 · see citing works
Improve the Deep Learning Models in Forestry Based on Explanations and Expertise
Cheng, X., Doosthosseini, A., & Kunkel, J. M. (2022) · Frontiers in Plant Science · doi:10.3389/fpls.2022.902105 · Cited by 14 · see citing works
Yamati, F. R. I., Günder, M., Barreto, A., Bömer, J., Laufer, D., Bauckhage, C., & Mahlein, A. (2023) · Plant Disease · doi:10.1094/pdis-04-23-0779-re · Cited by 13 · see citing works
Self-citationStudying How to Efficiently and Effectively Guide Models with Explanations
Rao, S., Böhle, M., Parchami-Araghi, A., & Schiele, B. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00184 · Cited by 13 · see citing works
Debiased-CAM to mitigate image perturbations with faithful visual explanations of machine learning
Zhang, W., Dimiccoli, M., & Lim, B. Y. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517522 · Cited by 15 · see citing works
Interactive Label Cleaning with Example-based ExplanationsPreprint
Teso, S., Bontempelli, A., Giunchiglia, F., & Passerini, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.03922 · Cited by 13 · see citing works
Self-citationDon’t Be Fooled: The Misinformation Effect of Explanations in Human–AI Collaboration
Spitzer, P., Holstein, J., Morrison, K., Holstein, K., Satzger, G., & Kühl, N. (2026) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2025.2574511 · Cited by 12 · see citing works
Schnake, T., Jafari, F. R., Lederer, J., Xiong, P., Nakajima, S., Gugler, S., Montavon, G., & Müller, K. (2025) · Information Fusion · doi:10.1016/j.inffus.2024.102923 · Cited by 12 · see citing works
Strategies to Exploit XAI to Improve Classification Systems
Apicella, A., Lorenzo, L. D., Isgrò, F., Pollastro, A., & Prevete, R. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-44064-9_9 · Cited by 12 · see citing works
Barnett, A. J., Schwartz, F. R., Tao, C., Chen, C., Ren, Y., Lo, J. Y.-C., & Rudin, C. D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.12308 · Cited by 12 · see citing works
Kierdorf, J., & Roscher, R. (2023) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2023.3293802 · Cited by 11 · see citing works
Interpretability Analysis of Convolutional Neural Networks for Crack Detection
Wu, J., He, Y., Xu, C., Jia, X., Huang, Y., Chen, Q., Huang, C., Eslamlou, A. D., & Huang, S. (2023) · Buildings · doi:10.3390/buildings13123095 · Cited by 12 · see citing works
Towards Directive Explanations: Crafting Explainable AI Systems for Actionable Human-AI Interactions
Bhattacharya, A. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3638177 · Cited by 10 · see citing works
Position Papers for the ASCR Workshop on Basic Research Needs in Quantum Computing and Networking
Lougovski, P., Parekh, O., Broz, J., Byrd, M., Chembo, Y. K., de Jong, W. A., Figueroa, E. V., Humble, T. S., Larson, J. M., Quiroz, G., et al. (2023) · doi:10.2172/2283320 · Cited by 10 · see citing works
2025 photonics for agrifood roadmap: towards a sustainable and healthier planet
Smeesters, L., Venturini, F., Paulus, S., Mahlein, A., Perpetuini, D., Cardone, D., Merla, A., Gac, B., Perrin, S., Trégoat, D., et al. (2025) · Journal of Physics Photonics · doi:10.1088/2515-7647/adbea9 · Cited by 10 · see citing works
Self-citationPark, D., Lee, J., Park, K., & Ryu, S. (2023) · Advanced Engineering Materials · doi:10.1002/adem.202300867 · Cited by 8 · see citing works
From Hope to Safety: Unlearning Biases of Deep Models via Gradient Penalization in Latent Space
Dreyer, M., Pahde, F., Anders, C. J., Samek, W., & Lapuschkin, S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i19.30096 · Cited by 9 · see citing works
Okole, N., Yamati, F. R. I., Hossain, R., Varrelmann, M., Mahlein, A., & Heim, R. H. J. (2024) · Plant Pathology · doi:10.1111/ppa.13973 · Cited by 9 · see citing works
Self-citationA Rate-Distortion Framework for Explaining Black-Box Model Decisions
Kolek, S., Nguyễn, Đ. A., Levie, R., Bruna, J., & Kutyniok, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-04083-2_6 · Cited by 9 · see citing works
An Explanatory Model Steering System for Collaboration between Domain Experts and AI
Bhattacharya, A., Stumpf, S., & Verbert, K. (2024) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3631700.3664886 · Cited by 8 · see citing works
Debbagh, M., Sun, S., & Lefsrud, M. G. (2025) · Plant Phenomics · doi:10.1016/j.plaphe.2025.100089 · Cited by 8 · see citing works
Enhancing Descriptive Image Quality Assessment With a Large-Scale Multi-Modal Dataset
You, Z., Gu, J., Cai, X., Li, Z., Zhu, K., Dong, C., & Xue, T. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3639998 · Cited by 23 · see citing works
Impact of Feedback Type on Explanatory Interactive Learning
Hagos, M. T., Curran, K. M., & Namee, B. M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16564-1_13 · Cited by 8 · see citing works
Representation Debiasing of Generated Data Involving Domain Experts
Bhattacharya, A., Stumpf, S., & Verbert, K. (2024) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3631700.3664910 · Cited by 8 · see citing works
User Driven Model Adjustment via Boolean Rule Explanations
Daly, E., Mattetti, M., Alkan, Ö., & Nair, R. S. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i7.16737 · Cited by 8 · see citing works
Impact of High-Cadence Earth Observation in Maize Crop Phenology Classification
Nieto, L., Houborg, R., Zajdband, A. D., Jumpasut, A., Prasad, P. V. V., Olson, B. J. S. C., & Ciampitti, I. A. (2022) · Remote Sensing · doi:10.3390/rs14030469 · Cited by 8 · see citing works
A Marr's Three‐Level Analytical Framework for Neuromorphic Electronic Systems
Guo, Y., Zou, X., Hu, Y., Yang, Y., Wang, X., He, Y., Kong, R., Guo, Y., Li, G., Zhang, W., et al. (2021) · Advanced Intelligent Systems · doi:10.1002/aisy.202100054 · Cited by 10 · see citing works
- Authors
- Schramowski, P., Stammer, W., Teso, S., Brugger, A., Herbert, F., Shao, X., Luigs, H.-G., Mahlein, A.-K., & Kersting, K.
- Affiliations
- Technische Universität Darmstadt (DE)
- University of Trento (IT)
- University of Bonn (DE)
- Corresponding author
- Schramowski, Patrick
- ORCID
- Venue
- Nature Machine Intelligence
- Published
- 2020-08-12
- Volume
- 2
- Issue
- 8
- Pages
- 476-486
- ISSN
- 2522-5839 (ISSN-L)
- OpenAlex
- W3035989815
- Cited by
- 207
- References
- 68
- Citations by year
- 2026: 11 · 2025: 33 · 2024: 45 · 2023: 46 · 2022: 31 · 2021: 32 · 2020: 9
- FWCI
- 11.58
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Nature Machine Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Iris (University of Trento) (submittedVersion, other-oa)
- Lirias (KU Leuven) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Code Ocean
- Funders
- European Commission (694980)
- Deutsche Forschungsgemeinschaft (EXC-2070, 390732324)
- Bundesministerium für Ernährung und Landwirtschaft (2818204715)
- Bundesamt für Landwirtschaft
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Life Sciences › Agricultural and Biological Sciences › Plant Science › Smart Agriculture and AI
- Keywords
- human-in-the-loop learning, model explanation, Clever Hans effect, confounding factors, plant phenotyping, deep neural network
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review