CoChat
Sign Up
PreprintCited by 338Explainable Artificial Intelligence (XAI)Machine Learning in HealthcareDomain Adaptation and Few-Shot Learning

Concept Bottleneck Models

Open access
Koh, Pang Wei, Nguyen, Thao, Tang, Yew Siang, Mussmann, Stephen, Pierson, Emma, Kim, Been, Liang, Percy·2020·arXiv
DOI 10.48550/arxiv.2007.04612arXiv 2007.04612
Open this paper in CoChat — free accountOpen PDF

Abstract

We seek to learn models that we can interact with using high-level concepts: if the model did not think there was a bone spur in the x-ray, would it still predict severe arthritis? State-of-the-art models today do not typically support the manipulation of concepts like "the existence of bone spurs", as they are trained end-to-end to go directly from raw input (e.g., pixels) to output (e.g., arthritis severity). We revisit the classic idea of first predicting concepts that are provided at training time, and then using these concepts to predict the label. By construction, we can intervene on these concept bottleneck models by editing their predicted concept values and propagating these changes to the final prediction. On x-ray grading and bird identification, concept bottleneck models achieve competitive accuracy with standard end-to-end models, while enabling interpretation in terms of high-level clinical concepts ("bone spurs") or bird attributes ("wing color"). These models also allow for richer human-model interaction: accuracy improves significantly if we can correct model mistakes on concepts at test time.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2007.04612

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Concept-based approaches like TCAV tie internal model representations to clinically relevant events such as joint space decreasing or osteophyte growth.

    “By tying internal model representations to clinically relevant events such as joint space decreasing or osteophyte growth, concept-based approaches like TCAV go one step further [177].”

    From Biomarker-Guided Imaging and AI-Augmented Diagnosis of Degenerative Joint Disease (2025) · cites this paper as [177]

    The Introduction (pp. 1–2) describes interpreting knee-arthritis models using joint-space narrowing and bone spurs, and says prior concept-based methods examine concepts in hidden layers. The OAI appendix (p. 13) lists joint-space narrowing and osteophytes as clinical concepts. But the paper does not say that TCAV itself was applied to those clinical features; its osteoarthritis example concerns concept bottleneck models. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 38

  1. Bau, D., Zhou, B., Khosla, A., Oliva, A., and Torralba, A. Network dissection: Quantifying interpretability of deep visual representations. In Computer Vision and Pattern Recognition (CVPR) , pp. 6541-6549, 2017.
  2. Bengio, Y., Courville, A., and Vincent, P. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence , 35 (8):1798-1828, 2013.
  3. Bucher, M., Herbin, S., and Jurie, F. Semantic bottleneck for computer vision tasks. In Asian Conference on Computer Vision , pp. 695-712, 2018.
  4. Bunescu, R. C. and Mooney, R. J. A shortest path dependency kernel for relation extraction. In Empirical Methods in Natural Language Processing (EMNLP) , pp. 724-731, 2005.
  5. Chen, X., Duan, Y., Houthooft, R., Schulman, J., Sutskever, I., and Abbeel, P. InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets. In Advances in Neural Information Processing Systems (NeurIPS) , 2016.
  6. Chen, Z., Bei, Y., and Rudin, C. Concept whitening for interpretable image recognition. arXiv preprint arXiv:2002.01650 , 2020. · arXiv
  7. Cheng, J. and Bernstein, M. S. Flock: Hybrid CrowdMachine learning classifiers. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing , pp. 600-611, 2015.
  8. Clough, J. R., Oksuz, I., Puyol-Ant´ on, E., Ruijsink, B., King, A. P., and Schnabel, J. A. Global and local interpretability for cardiac MRI classification. In International Conference on Medical Image Computing and ComputerAssisted Intervention , pp. 656-664, 2019.
  9. Cui, Y., Song, Y., Sun, C., Howard, A., and Belongie, S. Large scale fine-grained categorization and domainspecific transfer learning. In Computer Vision and Pattern Recognition (CVPR) , pp. 4109-4118, 2018.
  10. Duan, K., Parikh, D., Crandall, D., and Grauman, K. Discovering localized attributes for fine-grained recognition. In Computer Vision and Pattern Recognition (CVPR) , pp. 3474-3481, 2012.
  11. Fauw, J. D., Ledsam, J. R., Romera-Paredes, B., Nikolov, S., Tomasev, N., Blackwell, S., Askham, H., Glorot, X., O'Donoghue, B., Visentin, D., et al. Clinically applicable deep learning for diagnosis and referral in retinal disease. Nature Medicine , 24(9):1342-1350, 2018.
  12. Fong, R. C. and Vedaldi, A. Interpretable explanations of black boxes by meaningful perturbation. In International Conference on Computer Vision (ICCV) , pp. 3429-3437, 2017.
  13. Ghorbani, A., Wexler, J., Zou, J. Y., and Kim, B. Towards automatic concept-based explanations. In Advances in Neural Information Processing Systems (NeurIPS) , pp. 9277-9286, 2019.
  14. Goyal, Y., Shalit, U., and Kim, B. Explaining classifiers with causal concept effect (CaCE). arXiv preprint arXiv:1907.07165 , 2019. · arXiv
  15. Graziani, M., Andrearczyk, V., and M¨ uller, H. Regression concept vectors for bidirectional explanations in histopathology. In Understanding and Interpreting Machine Learning in Medical Image Computing Applications , pp. 124-132, 2018.
  16. He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. In Computer Vision and Pattern Recognition (CVPR) , 2016.
  17. Higgins, I., Matthey, L., Pal, A., Burgess, C., Glorot, X., Botvinick, M., Mohamed, S., and Lerchner, A. betavae: Learning basic visual concepts with a constrained variational framework. In International Conference on Learning Representations (ICLR) , 2017.
  18. Huang, S., Xu, Z., Tao, D., and Zhang, Y. Part-stacked CNN for fine-grained visual categorization. In Computer Vision and Pattern Recognition (CVPR) , pp. 1173-1182, 2016.
  19. Kellgren, J. and Lawrence, J. Radiological assessment of osteo-arthrosis. Annals of the Rheumatic Diseases , 16(4), 1957.
  20. (tcav). In International Conference on Machine Learning (ICML) , pp. 2668-2677, 2018.
  21. Kumar, N., Berg, A. C., Belhumeur, P. N., and Nayar, S. K. Attribute and simile classifiers for face verification. In International Conference on Computer Vision (ICCV) , pp. 365-372, 2009.
  22. Lampert, C. H., Nickisch, H., and Harmeling, S. Learning to detect unseen object classes by between-class attribute transfer. In Computer Vision and Pattern Recognition (CVPR) , pp. 951-958, 2009.
  23. Lewis, D. D. Feature selection and feature extract ion for text categorization. In Speech and Natural Language: Proceedings of a Workshop Held at Harriman, New York, February 23-26, 1992 , 1992.
  24. Losch, M., Fritz, M., and Schiele, B. Interpretability beyond classification output: Semantic bottleneck networks. arXiv preprint arXiv:1907.10882 , 2019. · arXiv
  25. Melis, D. A. and Jaakkola, T. Towards robust interpretability with self-explaining neural networks. In Advances in Neural Information Processing Systems (NeurIPS) , pp. 7775-7784, 2018.
  26. Murty, S., Koh, P. W., and Liang, P. ExpBERT: Representation engineering with natural language explanations. In Association for Computational Linguistics (ACL) , 2020.
  27. Nixon, M. and Aguado, A. Feature extraction and image processing for computer vision . Academic press, 2019.
  28. O'Shaughnessy, M., Canal, G., Connor, M., Davenport, M., and Rozell, C. Generative causal explanations of blackbox classifiers. arXiv preprint arXiv:2006.13913 , 2020. · arXiv
  29. Pearl, J. Causality: Models, Reasoning and Inference , volume 29. Springer, 2000.
  30. Pierson, E., Cutler, D., Leskovec, J., Mullainathan, S., and Obermeyer, Z. Using machine learning to understand racial and socioeconomic differences in knee pain. NBER Machine Learning and Healthcare Conference , 2019.
  31. Sagawa, S., Koh, P. W., Hashimoto, T. B., and Liang, P. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. In International Conference on Learning Representations (ICLR) , 2020.
  32. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. Rethinking the Inception architecture for computer vision. In Computer Vision and Pattern Recognition (CVPR) , pp. 2818-2826, 2016.
  33. Tiulpin, A., Thevenot, J., Rahtu, E., Lehenkari, P., and Saarakkala, S. Automatic knee osteoarthritis diagnosis from plain radiographs: A deep learning-based approach. Scientific Reports , 8(1):1-10, 2018.
  34. Yi, K., Wu, J., Gan, C., Torralba, A., Kohli, P., and Tenenbaum, J. Neural-symbolic vqa: Disentangling reasoning from vision and language understanding. In Advances in Neural Information Processing Systems (NeurIPS) , pp. 1031-1042, 2018.
  35. Zelenko, D., Aone, C., and Richardella, A. Kernel methods for relation extraction. Journal of Machine Learning Research , 3(0):1083-1106, 2003.
  36. Zheng, A. and Casari, A. Feature engineering for machine learning: principles and techniques for data scientists . ' O'Reilly Media, Inc.', 2018.
  37. Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., and Torralba, A. Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence , 40(6):1452-1464, 2017.
  38. Zhou, B., Sun, Y., Bau, D., and Torralba, A. Interpretable basis decomposition for visual explanation. In European Conference on Computer Vision (ECCV) , pp. 119-134, 2018.

Cited by

Sorted by citations

Authors
Koh, P. W., Nguyen, T., Tang, Y. S., Mussmann, S., Pierson, E., Kim, B., & Liang, P.
Affiliations
  • Google (United States) (US)
  • Stanford University (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2020
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2007.04612
arXiv
2007.04612
OpenAlex
W3035602609
Cited by
338
References
0
Citations by year
2026: 9 · 2025: 59 · 2024: 100 · 2023: 92 · 2022: 35 · 2021: 35 · 2020: 8
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Science Foundation
  • Toyota Research Institute
  • Info-communications Media Development Authority
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
concept bottleneck models, interpretability, bird identification, clinical concepts

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review