CoChat
Sign Up
PreprintCited by 702Explainable Artificial Intelligence (XAI)Advanced Neural Network ApplicationsDomain Adaptation and Few-Shot Learning

This Looks Like That: Deep Learning for Interpretable Image Recognition

Open access
Chen, Chaofan, Li, Oscar, Tao, Chaofan, Barnett, Alina Jade, Su, Jonathan, Rudin, Cynthia·2018·arXiv
DOI 10.48550/arxiv.1806.10574arXiv 1806.10574
Open this paper in CoChat — free accountOpen PDF

Abstract

When we are faced with challenging image classification tasks, we often explain our reasoning by dissecting the image, and pointing out prototypical aspects of one class or another. The mounting evidence for each of the classes helps us make our final decision. In this work, we introduce a deep network architecture -- prototypical part network (ProtoPNet), that reasons in a similar way: the network dissects the image by finding prototypical parts, and combines evidence from the prototypes to make a final classification. The model thus reasons in a way that is qualitatively similar to the way ornithologists, physicians, and others would explain to people on how to solve challenging image classification tasks. The network uses only image-level labels for training without any annotations for parts of images. We demonstrate our method on the CUB-200-2011 dataset and the Stanford Cars dataset. Our experiments show that ProtoPNet can achieve comparable accuracy with its analogous non-interpretable counterpart, and when several ProtoPNets are combined into a larger network, it can achieve an accuracy that is on par with some of the best-performing deep models. Moreover, ProtoPNet provides a level of interpretability that is absent in other interpretable deep models.

Integrity signals

Registry

DataCite · 10.48550/arxiv.1806.10574

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In a recent study, an interpretable neural network model for classifying objects performed at a level of accuracy on par with black box systems on benchmark computer vision datasets.

    “They found that the interpretable AI system performed at a level of accuracy on par with those black box systems on benchmark computer vision datasets.”

    From Interpretable algorithmic forensics (2023) · cites this paper as [30]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Chen, C., Li, O., Tao, C., Barnett, A. J., Su, J., & Rudin, C.
Affiliations
  • Duke University (US)
  • Massachusetts Institute of Technology (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2018
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1806.10574
arXiv
1806.10574
OpenAlex
W2811104224
Cited by
702
References
9
Citations by year
2026: 7 · 2025: 80 · 2024: 158 · 2023: 181 · 2022: 125 · 2021: 88 · 2020: 41 · 2019: 21 · 2017: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
ProtoPNet, prototype-based classification, explainable deep learning, CUB-200-2011, Stanford Cars

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review