Captum: A unified and generic model interpretability library for PyTorch
Open accessAbstract
In this paper we introduce a novel, unified, open-source model interpretability library for PyTorch [12]. The library contains generic implementations of a number of gradient and perturbation-based attribution algorithms, also known as feature, neuron and layer importance algorithms, as well as a set of evaluation metrics for these algorithms. It can be used for both classification and non-classification models including graph-structured models built on Neural Networks (NN). In this paper we give a high-level overview of supported attribution algorithms and show how to perform memory-efficient and scalable computations. We emphasize that the three main characteristics of the library are multimodality, extensibility and ease of use. Multimodality supports different modality of inputs such as image, text, audio or video. Extensibility allows adding new algorithms and features. The library is also designed for easy understanding and use. Besides, we also introduce an interactive visualization tool called Captum Insights that is built on top of Captum library and allows sample-based model debugging and visualization using feature importance metrics.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Some XAI toolkits offer XAI evaluations to facilitate quality control and comparability.
“To facilitate quality control and comparability, some of the toolkits also offer XAI evaluations37,39,42.”
From Orchestrating explainable artificial intelligence for multimodal and longitudinal data in medical imaging (2024) · cites this paper as [39]
Multiple toolkits offer readily implemented XAI methods for a wide variety of tasks applicable to medical imaging.
“There are multiple toolkits, such as AIX-36037, Alibi38, Captum39, EthicalML-XAI40, iNNvestigate41, Quantus42, among others, offering readily implemented XAI methods for a wide variety of tasks applicable to medical imaging (Table 1).”
From Orchestrating explainable artificial intelligence for multimodal and longitudinal data in medical imaging (2024) · cites this paper as [39]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473 · see citing works
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513 · see citing works
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707 · see citing works
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839 · see citing works
Gauriat, C.-M., Pencolé, Y., Ribot, P., et al. (2024) · HAL · doi:10.4230/oasics.dx.2024.27 · Cited by 12,623 · see citing works
Axiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087 · see citing works
Learning Word Vectors for Sentiment Analysis
Maas, A. L., Daly, R. E., Pham, P. T., Huang, D., Ng, A. Y., & Potts, C. (2011) · Cited by 3,710 · see citing works
Striving for Simplicity: The All Convolutional Net
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447 · see citing works
Learning Important Features Through Propagating Activation Differences
Shrikumar, A., Greenside, P., & Kundaje, A. (2017) · arXiv · doi:10.48550/arxiv.1704.02685 · Cited by 2,761 · see citing works
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507 · see citing works
Hedonic housing prices and the demand for clean air
Harrison, D., & Rubinfeld, D. L. (1978) · Journal of Environmental Economics and Management · doi:10.1016/0095-0696(78)90006-2 · Cited by 1,878 · see citing works
SmoothGrad: removing noise by adding noise
Smilkov, D., Thorat, N., Kim, B., Viégas, F., & Wattenberg, M. (2017) · arXiv · doi:10.48550/arxiv.1706.03825 · Cited by 1,467 · see citing works
Sanity Checks for Saliency Maps
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., & Kim, B. (2018) · arXiv · doi:10.48550/arxiv.1810.03292 · Cited by 1,294 · see citing works
Towards better understanding of gradient-based attribution methods for Deep Neural NetworksPreprint
Ancona, M., Ceolini, E., Öztireli, C., & Groß, M. H. (2017) · Zurich Open Repository and Archive (University of Zurich) · doi:10.3929/ethz-b-000249929 · Cited by 634 · see citing works
On the Robustness of Interpretability Methods
Alvarez-Melis, D., & Jaakkola, T. S. (2018) · arXiv · doi:10.48550/arxiv.1806.08049 · Cited by 310 · see citing works
Benchmarking Attribution Methods with Relative Feature ImportancePreprint
Yang, M., & Kim, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.09701 · Cited by 108 · see citing works
How Important Is a Neuron?Preprint
Dhamdhere, K., Sundararajan, M., & Yan, Q. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.12233 · Cited by 100 · see citing works
Learning Explainable Models Using Attribution Priors
Erion, G., Janizek, J. D., Sturmfels, P., Lundberg, S., & Lee, S. (2019) · Cited by 53 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [12][12] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas K¨ opf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, highperformance deep learning library. In NeurIPS , 2019. -
- [19][19] Chih-Kuan Yeh, Cheng-Yu Hsieh, Arun Sai Suggala, David I. Inouye, and Pradeep D. Ravikumar. On the (in)fidelity and sensitivity of explanations. In NeurIPS , 2019. -
Cited by
Sorted by citations
Pan-cancer integrative histology-genomic analysis via multimodal deep learning
Chen, R. J., Lu, M. Y., Williamson, D. F. K., Chen, T. Y., Lipkova, J., Noor, Z., Shaban, M., Shady, M., Williams, M., Joo, B., et al. (2022) · Cancer Cell · doi:10.1016/j.ccell.2022.07.004 · Cited by 660 · see citing works
Borys, K., Schmitt, Y. A., Nauta, M., Seifert, C., Krämer, N., Friedrich, C. M., & Nensa, F. (2023) · European Journal of Radiology · doi:10.1016/j.ejrad.2023.110786 · Cited by 233 · see citing works
EXplainable Artificial Intelligence (XAI)—From Theory to Methods and Applications
Ortigossa, E. S., Gonçalves, T., & Nonato, L. G. (2024) · IEEE Access · doi:10.1109/access.2024.3409843 · Cited by 124 · see citing works
Pahud de Mortanges, A., Luo, H., Shu, S. Z., Kamath, A., Suter, Y., Shelan, M., Pöllinger, A., & Reyes, M. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01190-w · Cited by 93 · see citing works
“While many of these libraries can process multiple input data types separately, only Captum explicitly offers multimodality for the joint processing of input features stemming from different data types.”
Establishing and evaluating trustworthy AI: overview and research challenges
Kowald, D., Scher, S., Pammer-Schindler, V., Müllner, P., Waxnegger, K., Demelius, L., Fessl, A., Toller, M., Mendoza Estrada, I. G., Šimić, I., et al. (2024) · Frontiers in Big Data · doi:10.3389/fdata.2024.1467222 · Cited by 59 · see citing works
Predicting the translation efficiency of messenger RNA in mammalian cells
Zheng, D., Persyn, L., Wang, J., Liu, Y., Ulloa-Montoya, F., Cenik, C., & Agarwal, V. (2026) · Nature Biotechnology · doi:10.1038/s41587-025-02712-x · Cited by 52 · see citing works
Zeng, W., Dou, Y., Pan, L., Xu, L., & Peng, S. (2024) · Nature Communications · doi:10.1038/s41467-024-52293-7 · Cited by 49 · see citing works
Chandrashekar, P. B., Alatkar, S., Wang, J., Hoffman, G. E., He, C., Jin, T., Khullar, S., Bendl, J., Fullard, J. F., Roussos, P., et al. (2023) · Genome Medicine · doi:10.1186/s13073-023-01248-6 · Cited by 37 · see citing works
Hamed, A. A., Zachara-Szymanska, M., & Wu, X. (2024) · iScience · doi:10.1016/j.isci.2024.108782 · Cited by 38 · see citing works
Hempel, P., Ribeiro, A. H., Vollmer, M., Bender, T., Dörr, M., Krefting, D., & Spicher, N. (2025) · npj Digital Medicine · doi:10.1038/s41746-024-01428-7 · Cited by 31 · see citing works
Robustness Analysis of Deep Learning-Based Lung Cancer Classification Using Explainable Methods
Malafaia, M., Silva, F., Neves, I., Pereira, T., & Oliveira, H. P. (2022) · IEEE Access · doi:10.1109/access.2022.3214824 · Cited by 25 · see citing works
A survey of post-hoc xai methods from a visualization perspective: Challenges and opportunities
Bhati, D., Amiruzzaman, M., Zhao, Y., Guercio, A., & Le, T. (2025) · IEEE Access · doi:10.1109/access.2025.3581136 · Cited by 24 · see citing works
Azher, Z. L., Suvarna, A., Chen, J.-Q., Zhang, Z., Christensen, B. C., Salas, L. A., Vaickus, L. J., & Levy, J. J. (2023) · BioData Mining · doi:10.1186/s13040-023-00338-w · Cited by 20 · see citing works
Tan, Y., Dede, M., Mohanty, V., Dou, J., Hill, H., Bernstam, E., & Chen, K. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104648 · Cited by 19 · see citing works
- Authors
- Kokhlikyan, N., Miglani, V., Martin, M., Wang, E., Alsallakh, B., Reynolds, J., Melnikov, A., Kliushkina, N., Araya, C., Yan, S., & Reblitz-Richardson, O.
- Affiliations
- Meta (United Kingdom) (GB)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2020
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2009.07896
- OpenAlex
- W3085380432
- Cited by
- 658
- References
- 18
- Citations by year
- 2026: 40 · 2025: 124 · 2024: 182 · 2023: 152 · 2022: 96 · 2021: 57 · 2020: 5 · 2019: 1 · 2013: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- model interpretability, PyTorch, gradient-based attribution, feature importance, neuron importance, layer importance, graph neural networks, multimodal models, interactive visualization, model debugging, open-source library
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
