The DOI matches its Crossref record; 65 of the 68 references it cites were found in a registry or an index.
Making deep neural networks right for the right scientific reasons by interacting with their explanations
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The subjective trust in the AI system was assessed using the Trust in Automation questionnaire (TiA) [63], which is a commonly employed and validated tool for measuring trust [78, 116, 120].”
From Dealing with Uncertainty: Understanding the Impact of Prognostic Versus Diagnostic Tasks on Trust and Reliance in Human-AI Decision Making (2024) · cites this paper as [116]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
(2008) · Journal of Machine Learning Research · Cited by 33,051
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428
A tutorial on spectral clustering
Ulrike Von Luxburg, A. (2007) · Statistics and Computing · doi:10.1007/s11222-007-9033-z · Cited by 10,787
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561
Machine learning: Trends, perspectives, and prospects
Jordan, M. I., & Mitchell, T. M. (2015) · Science · doi:10.1126/science.aaa8415 · Cited by 9,955
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
Xiao, H., Rasul, K., & Vollgraf, R. (2017) · arXiv · doi:10.48550/arxiv.1708.07747 · Cited by 6,108
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507
Buciluǎ, C., Caruana, R., & Niculescu-Mizil, A. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150464 · Cited by 2,068
Probabilistic machine learning and artificial intelligence
Ghahramani, Z. (2015) · Nature · doi:10.1038/nature14541 · Cited by 1,995
Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., & Oermann, E. K. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002683 · Cited by 1,719
Sanity Checks for Saliency Maps
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., & Kim, B. (2018) · arXiv · doi:10.48550/arxiv.1810.03292 · Cited by 1,294
Settles, B. (2012) · Cited by 1,183
Unmasking Clever Hans predictors and assessing what machines really learn
Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., & Müller, K.-R. (2019) · Nature Communications · doi:10.1038/s41467-019-08987-4 · Cited by 1,047
Deep Bayesian Active Learning with Image Data
Gal, Y., Islam, R., & Ghahramani, Z. (2017) · arXiv · doi:10.48550/arxiv.1703.02910 · Cited by 900
This Looks Like That: Deep Learning for Interpretable Image Recognition
Chen, C., Li, O., Tao, C., Barnett, A. J., Su, J., & Rudin, C. (2018) · arXiv · doi:10.48550/arxiv.1806.10574 · Cited by 702
Plant Phenomics, From Sensors to Knowledge
Tardieu, F., Cabrera‐Bosquet, L., Pridmore, T., & Bennett, M. J. (2017) · Current Biology · doi:10.1016/j.cub.2017.05.055 · Cited by 666
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629
Settles, B. (2012) · Synthesis Lectures on Artificial Intelligence and Machine Learning · doi:10.2200/s00429ed1v01y201207aim018 · Cited by 605
Unmasking Clever Hans predictors and assessing what machines really learn
Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., & Müller, K. (2019) · RePEc: Research Papers in Economics · Cited by 580
Principles of Explanatory Debugging to Personalize Interactive Machine Learning
Kulesza, T., Burnett, M., Wong, W.-K., & Stumpf, S. (2015) · Proceedings of the 20th International Conference on Intelligent User Interfaces · doi:10.1145/2678025.2701399 · Cited by 576
Psychological Foundations of Trust
Simpson, J. A. (2007) · Current Directions in Psychological Science · doi:10.1111/j.1467-8721.2007.00517.x · Cited by 541
A Benchmark for Interpretability Methods in Deep Neural NetworksPreprint
Hooker, S., Erhan, D., Kindermans, P.-J., & Kim, B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.10758 · Cited by 531
Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations
Ross, A. S., Hughes, M. C., & Doshi-Velez, F. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/371 · Cited by 515
Multimodal Explanations: Justifying Decisions and Pointing to the Evidence
Park, D. H., Hendricks, L. A., Akata, Z., Rohrbach, A., Schiele, B., Darrell, T., & Rohrbach, M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00915 · Cited by 418
Deep machine learning provides state-of-the-art performance in image-based plant phenotyping
Pound, M. P., Atkinson, J. A., Townsend, A. J., Wilson, M. H., Griffiths, M., Jackson, A. S., Bulat, A., Tzimiropoulos, G., Wells, D. M., Murchie, E. H., et al. (2017) · GigaScience · doi:10.1093/gigascience/gix083 · Cited by 408
Deep Bayesian Active Learning with Image Data
Ghahramani, Z., Gal, Y., & Islam, R. (2017) · PMLR · doi:10.17863/cam.11070 · Cited by 392
Theoretical Considerations and Development of a Questionnaire to Measure Trust in Automation
Körber, M. (2019) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-319-96074-6_2 · Cited by 317
Deep learning predicts hip fracture using confounding patient and healthcare variables
Badgeley, M. A., Zech, J. R., Oakden-Rayner, L., Glicksberg, B. S., Liu, M., Gale, W., McConnell, M. V., Percha, B., Snyder, T. M., & Dudley, J. T. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0105-1 · Cited by 290
Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Grounded
Selvaraju, R. R., Lee, S., Shen, Y., Jin, H., Ghosh, S., Heck, L. P., Batra, D., & Parikh, D. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00268 · Cited by 262
Using “annotator rationales” to improve machine learning for text categorization
Zaidan, O. F., Eisner, J. M., & Piatko, C. (2007) · Cited by 252
Hoffman, R. R., Johnson, M., Bradshaw, J. M., & Underbrink, A. J. (2013) · IEEE Intelligent Systems · doi:10.1109/mis.2013.24 · Cited by 238
Explanatory Interactive Machine Learning
Teso, S., & Kersting, K. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3314293 · Cited by 226
Self-citationMochida, K., Koda, S., Inoue, K., Hirayama, T., Tanaka, S., Nishii, R., & Melgani, F. (2018) · GigaScience · doi:10.1093/gigascience/giy153 · Cited by 217
Theory of Disagreement-Based Active Learning
Hanneke, S. (2014) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000037 · Cited by 194
Explanations can be manipulated and geometry is to blame
Dombrowski, A.-K., Alber, M., Anders, C. J., Ackermann, M., Müller, K.-R., & Kessel, P. (2019) · arXiv · doi:10.48550/arxiv.1906.07983 · Cited by 171
Nonmyopic active learning of Gaussian processes
Krause, A., & Guestrin, C. (2007) · International Conference on Machine Learning · doi:10.1145/1273496.1273553 · Cited by 168
Settles, B. (2011) · Cited by 167
Narayanan, M., Chen, E., He, J., Kim, B., Gershman, S., & Doshi-Velez, F. (2018) · arXiv · doi:10.48550/arxiv.1802.00682 · Cited by 153
The true sample complexity of active learning
Balcan, M.-F., Hanneke, S., & Vaughan, J. W. (2010) · Machine Learning · doi:10.1007/s10994-010-5174-y · Cited by 137
Interpreting Blackbox Models via Model ExtractionPreprint
Bastani, O., Kim, C., & Bastani, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08504 · Cited by 126
Incorporating Priors with Feature Attribution on Text Classification
Liu, F., & Avci, B. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1631 · Cited by 120
Mahlein, A., Kuśka, M. T., Thomas, S., Wahabzada, M., Behmann, J., Rascher, U., & Kersting, K. (2019) · Current Opinion in Plant Biology · doi:10.1016/j.pbi.2019.06.007 · Cited by 109
Self-citationAn unexpected unity among methods for interpreting model predictionsPreprint
Lundberg, S., & Lee, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.07478 · Cited by 101
Learning Explainable Models Using Attribution Priors
Erion, G., Janizek, J. D., Sturmfels, P., Lundberg, S., & Lee, S. (2019) · Cited by 53
Shivaswamy, P., & Joachims, T. (2015) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4539 · Cited by 52
Short-Term Satisfaction and Long-Term Coverage
Schnabel, T., Bennett, P. N., Dumais, S., & Joachims, T. (2018) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3159652.3159700 · Cited by 47
Learning to Explain With Complemental Examples
Kanehira, A., & Harada, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00880 · Cited by 39
The Constrained Weight Space SVM: Learning with Ranked Features
Small, K., Wallace, B., Trikalinos, T. A., & Brodley, C. E. (2011) · Cited by 38
Active Imitation Learning via Reduction to I.I.D. Active LearningPreprint
Judah, K., Fern, A., & Dietterich, T. G. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1210.4876 · Cited by 32
Confounding variables can degrade generalization performance of radiological deep learning models
Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., & Oermann, E. K. (2018) · arXiv · doi:10.48550/arxiv.1807.00431 · Cited by 26
How and What Can Humans Learn from Being in the Loop?
Abdel-Karim, B. M., Pfeuffer, N., Rohde, G. G. U., & Hinz, O. F. (2020) · Künstliche Intell. · doi:10.1007/s13218-020-00638-x · Cited by 21
Human-Guided Learning for Probabilistic Logic Models
Odom, P., & Natarajan, S. (2018) · Frontiers in Robotics and AI · doi:10.3389/frobt.2018.00056 · Cited by 14
A Permutation Approach to Assess Confounding in Machine Learning Applications for Digital Health
Neto, E. C., Pratap, A., Perumal, T. M., Tummalacherla, M., Bot, B. M., Mangravite, L. M., & Omberg, L. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330903 · Cited by 13
de Souza, N. (2009) · Nature Methods · doi:10.1038/nmeth.f.289 · Cited by 10
Analyzing ImageNet with Spectral Relevance Analysis: Towards ImageNet un-Hans'ed.Preprint
Anders, C. J., Marinč, T., Neumann, D., Samek, W., Müller, K., & Lapuschkin, S. (2019) · arXiv (Cornell University) · Cited by 10
High-throughput phenotyping of rice growth traits
Lau, E. (2014) · Nature Reviews Genetics · doi:10.1038/nrg3857 · Cited by 2
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244302760185243
Research works printed but not linked: 6
- Herbert, F. P., Kersting, K. & Jäkel, F. Why should I trust in AI? Master’s thesis (Technical Univ. Darmstadt, 2019).
- Cakmak, M. et al. Mixed-initiative active learning. In ICML 2011 Workshop on Combining Learning Strategies to Reduce Label Cost (ACM, 2011).
- Roy, N. et al. Toward optimal active learning through Monte Carlo estimation of error reduction. In International Conference for Machine Learning 441–448 (ICML, 2001).
- Castro, R. M. et al. Upper and lower error bounds for active learning. In Proc. Conference on Communication, Control and Computing 1 (Univ. Illinois, 2007).
- Tong, S. & Koller, D. Support vector machine active learning with applications to text classification. J. Mach. Learn. Res. 2, 45–66 (2001).
- Meier, U. et al. Phenological growth stages of sugar beet (Beta vulgaris l. ssp.) codification and description according to the general BBCH scale (with figures). Nachr. Dtsch. Pflanzenschutzd. 45, 37–41 (1993).
Cited by
Sorted by citations
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762
Interpretable machine learning: Fundamental principles and 10 grand challenges
Rudin, C., Chen, C., Chen, Z., Huang, H., Semenova, L., & Zhong, C. (2022) · Statistics Surveys · doi:10.1214/21-ss133 · Cited by 958
Erion, G., Janizek, J. D., Sturmfels, P., Lundberg, S. M., & Lee, S.-I. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00343-w · Cited by 300
Large pre-trained language models contain human-like biases of what is right and wrong to do
Schramowski, P., Turan, C., Andersen, N., Rothkopf, C. A., & Kersting, K. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00458-8 · Cited by 291
Self-citationNazir, S., Dickson, D. M., & Akram, M. U. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.106668 · Cited by 283
“The focus shifted to correcting the underlying reasons for a wrong model decision which can aid in understanding the confounding factors in image classification [259].”
BackgroundInfluential
Metadata
- Authors
- Schramowski, P., Stammer, W., Teso, S., Brugger, A., Herbert, F., Shao, X., Luigs, H.-G., Mahlein, A.-K., & Kersting, K.
- Affiliations
- Technische Universität Darmstadt (DE)
- University of Trento (IT)
- University of Bonn (DE)
- Corresponding author
- Schramowski, Patrick
- ORCID
- Venue
- Nature Machine Intelligence
- Published
- 2020-08-12
- Volume
- 2
- Issue
- 8
- Pages
- 476-486
- ISSN
- 2522-5839 (ISSN-L)
- OpenAlex
- W3035989815
- Cited by
- 207
- References
- 68
- Citations by year
- 2026: 11 · 2025: 33 · 2024: 45 · 2023: 46 · 2022: 31 · 2021: 32 · 2020: 9
- FWCI
- 11.58
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Nature Machine Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Iris (University of Trento) (submittedVersion, other-oa)
- Lirias (KU Leuven) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Code Ocean
- Funders
- European Commission (694980)
- Deutsche Forschungsgemeinschaft (EXC-2070, 390732324)
- Bundesministerium für Ernährung und Landwirtschaft (2818204715)
- Bundesamt für Landwirtschaft
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Life Sciences › Agricultural and Biological Sciences › Plant Science › Smart Agriculture and AI
- Keywords
- human-in-the-loop learning, model explanation, Clever Hans effect, confounding factors, plant phenotyping, deep neural network
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
