The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Shared Interest: Measuring Human-AI Alignment to Identify Recurring Patterns in Model Behavior
Abstract
Saliency methods — techniques to identify the importance of input features on a model’s output — are a common step in understanding neural network behavior. However, interpreting saliency requires tedious manual inspection to identify and aggregate patterns in model behavior, resulting in ad hoc or cherry-picked analysis. To address these concerns, we present Shared Interest: metrics for comparing model reasoning (via saliency) to human reasoning (via ground truth annotations). By providing quantitative descriptors, Shared Interest enables ranking, sorting, and aggregating inputs, thereby facilitating large-scale systematic analysis of model behavior. We use Shared Interest to identify eight recurring patterns in model behavior, such as cases where contextual features or a subset of ground truth features are most important to the model. Working with representative real-world users, we show how Shared Interest can be used to decide if a model is trustworthy, uncover issues missed in manual analyses, and enable interactive probing.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Proper assessment and presentation of human-AI WoE differences can help effectively navigate humans' attention and efforts in the subsequent activities [14].”
From Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Making (2025) · cites this paper as [14]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Codella, N. C. F., Gutman, D., Celebi, M. E., Helba, B., Marchetti, M. A., Dusza, S. W., Kalloo, A., Liopyris, K., Mishra, N., Kittler, H., et al. (2018) · 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) · doi:10.1109/isbi.2018.8363547 · Cited by 2,620
SmoothGrad: removing noise by adding noise
Smilkov, D., Thorat, N., Kim, B., Viégas, F., & Wattenberg, M. (2017) · arXiv · doi:10.48550/arxiv.1706.03825 · Cited by 1,467 · cited 1×
“In this paper, we focus on post hoc saliency methods, also known as feature attribution methods [40], that allow us to observe model reasoning [7, 12, 26, 34, 35, 37, 38, 41].”
MethodNetwork Dissection: Quantifying Interpretability of Deep Visual Representations
Bau, D., Zhou, B., Khosla, A., Oliva, A., & Torralba, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.354 · Cited by 1,389
Sanity Checks for Saliency Maps
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., & Kim, B. (2018) · arXiv · doi:10.48550/arxiv.1810.03292 · Cited by 1,294
Rationalizing Neural Predictions
Lei, T., Barzilay, R., & Jaakkola, T. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1011 · Cited by 777 · cited 2×
“Figure 3: Using Shared Interest we identify eight recurring patterns in model behavior across two domains and saliency methods: an ImageNet [10] classifcation model with LIME [34] and a BeerAdvocate sentiment regression model [7, 25, 27] with Integrated Gradients [7, 41].”
MethodThe Building Blocks of Interpretability
Olah, C., Satyanarayan, A., Johnson, I., Carter, S., Schubert, L., Ye, K., & Mordvintsev, A. (2018) · Distill · doi:10.23915/distill.00010 · Cited by 591 · cited 1×
“limitations of instance-wise interpretation, they suggest decomposing activations through matrix factorization [30], activation atlases [8], or concept activation vectors [21].”
MethodSelf-citationVisual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers
Hohman, F., Kahng, M., Pienta, R., & Chau, D. H. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2843369 · Cited by 582
Large image datasets: A pyrrhic win for computer vision?
Birhane, A., & Prabhu, V. U. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00158 · Cited by 257
Beyer, L., Hénaff, O. J., Kolesnikov, A., Zhai, X., & Oord, A. V. D. (2020) · arXiv · doi:10.48550/arxiv.2006.07159 · Cited by 176
Noise or Signal: The Role of Image Backgrounds in Object Recognition
Xiao, K., Engstrom, L., Ilyas, A., & Madry, A. (2020) · arXiv · doi:10.48550/arxiv.2006.09994 · Cited by 161
exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformer Models
Hoover, B., Strobelt, H., & Gehrmann, S. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.18653/v1/2020.acl-demos.22 · Cited by 151
Self-citationCarter, S., Armstrong, Z., Schubert, L., Johnson, I., & Olah, C. (2019) · Distill · doi:10.23915/distill.00015 · Cited by 147
Debugging Tests for Model Explanations
Adebayo, J., Muelly, M., Liccardi, I., & Kim, B. (2020) · arXiv · doi:10.48550/arxiv.2011.05429 · Cited by 88
Research works printed but not linked: 34
- Brandon Carter , Siddhartha Jain , Jonas Mueller , and David Gifford . 2021 . Overinterpretation reveals image classification model pathologies . In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). Virtual Event. Brandon Carter, Siddhartha Jain, Jonas Mueller, and David Gifford. 2021. Overinterpretation reveals image classification model pathologies. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). Virtual Event.
- Brandon Carter , Jonas Mueller , Siddhartha Jain , and David K. Gifford . 2019 . What made you do this? Understanding black-box decisions with sufficient input subsets . In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS). PMLR , Naha, Japan, 567–576. Brandon Carter, Jonas Mueller, Siddhartha Jain, and David K. Gifford. 2019. What made you do this? Understanding black-box decisions with sufficient input subsets. In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS). PMLR, Naha, Japan, 567–576.
- Noel Codella Veronica Rotemberg Philipp Tschandl M. Emre Celebi Stephen Dusza David Gutman Brian Helba Aadi Kalloo Konstantinos Liopyris Michael Marchetti Harald Kittler and Allan Halpern. 2019. Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC). arxiv:1902.03368 [cs.CV] Noel Codella Veronica Rotemberg Philipp Tschandl M. Emre Celebi Stephen Dusza David Gutman Brian Helba Aadi Kalloo Konstantinos Liopyris Michael Marchetti Harald Kittler and Allan Halpern. 2019. Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC). arxiv:1902.03368 [cs.CV] · arXiv
- Finale Doshi-Velez and Been Kim. 2017. Towards A Rigorous Science of Interpretable Machine Learning. arxiv:1702.08608 [stat.ML] Finale Doshi-Velez and Been Kim. 2017. Towards A Rigorous Science of Interpretable Machine Learning. arxiv:1702.08608 [stat.ML] · arXiv
- Andre Esteva , Brett Kuprel , Roberto A. Novoa , Justin Ko , Susan M. Swetter , Helen M. Blau , and Sebastian Thrun . 2017. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 7639 ( 2017 ), 115–118. Andre Esteva, Brett Kuprel, Roberto A. Novoa, Justin Ko, Susan M. Swetter, Helen M. Blau, and Sebastian Thrun. 2017. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 7639 (2017), 115–118.
- 10.1109/TVCG.2019.2934659 · DOI · Paper page
- 10.1109/TVCG.2017.2744718 · DOI · Paper page
- 10.1109/TVCG.2018.2864500 · DOI · Paper page
- Been Kim , Martin Wattenberg , Justin Gilmer , Carrie Cai , James Wexler , Fernanda Viegas , and Rory Sayres . 2018 . Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV) . In Proceedings of the International Conference on Machine Learning (ICML). PMLR , Stockholm, Sweden, 2668–2677. Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory Sayres. 2018. Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV). In Proceedings of the International Conference on Machine Learning (ICML). PMLR, Stockholm, Sweden, 2668–2677.
- Pieter-Jan Kindermans , Sara Hooker , Julius Adebayo , Maximilian Alber , Kristof T. Schütt , Sven Dähne , Dumitru Erhan , and Been Kim . 2019. The (Un)reliability of Saliency Methods . In Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Vol. 11700 . Springer , 267–280. Pieter-Jan Kindermans, Sara Hooker, Julius Adebayo, Maximilian Alber, Kristof T. Schütt, Sven Dähne, Dumitru Erhan, and Been Kim. 2019. The (Un)reliability of Saliency Methods. In Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Vol. 11700. Springer, 267–280.
- Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization . In Proceedings of the International Conference on Learning Representations (ICLR) . San Diego, USA. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In Proceedings of the International Conference on Learning Representations (ICLR). San Diego, USA.
- Narine Kokhlikyan , Vivek Miglani , Miguel Martin , Edward Wang , Bilal Alsallakh , Jonathan Reynolds , Alexander Melnikov , Natalia Kliushkina , Carlos Araya , Siqi Yan , and Orion Reblitz-Richardson . 2020 . Captum: A unified and generic model interpretability library for PyTorch. arxiv:2009.07896 [cs.LG] Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan, and Orion Reblitz-Richardson. 2020. Captum: A unified and generic model interpretability library for PyTorch. arxiv:2009.07896 [cs.LG] · arXiv
- Scott M. Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions . In Proceedings of the Conference on Neural Information Processing Systems (NIPS) . Long Beach, USA, 4765–4774. Scott M. Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In Proceedings of the Conference on Neural Information Processing Systems (NIPS). Long Beach, USA, 4765–4774.
- 10.1109/ICDM.2012.110 · DOI · Paper page
- Christopher Meek. 2016. A Characterization of Prediction Errors. arxiv:1611.05955 [cs.LG] Christopher Meek. 2016. A Characterization of Prediction Errors. arxiv:1611.05955 [cs.LG] · arXiv
- Ninareh Mehrabi , Fred Morstatter , Nripsuta Saxena , Kristina Lerman , and Aram Galstyan . 2021. A Survey on Bias and Fairness in Machine Learning. Comput. Surveys 54, 6 ( 2021 ), 115:1–115:35. Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. A Survey on Bias and Fairness in Machine Learning. Comput. Surveys 54, 6 (2021), 115:1–115:35.
- Adam Paszke , Sam Gross , Francisco Massa , Adam Lerer , James Bradbury , Gregory Chanan , Trevor Killeen , Zeming Lin , Natalia Gimelshein , Luca Antiga , Alban Desmaison , Andreas Köpf , Edward Z. Yang , Zachary DeVito , Martin Raison , Alykhan Tejani , Sasank Chilamkurthy , Benoit Steiner , Lu Fang , Junjie Bai , and Soumith Chintala . 2019 . PyTorch: An Imperative Style, High-Performance Deep Learning Library . In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS) . Vancouver, Canada, 8024–8035. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). Vancouver, Canada, 8024–8035.
- 10.1007/s11747-019-00710-5 · DOI · Paper page
- 10.1109/TVCG.2016.2598828 · DOI · Paper page
- 10.1145/2939672.2939778 · DOI · Paper page
- 10.1109/ICCV.2017.74 · DOI · Paper page
- Karen Simonyan , Andrea Vedaldi , and Andrew Zisserman . 2014 . Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps . In Proceedings of the International Conference on Learning Representations (ICLR), Workshop Track . Banff, Canada. Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. In Proceedings of the International Conference on Learning Representations (ICLR), Workshop Track. Banff, Canada.
- Jost Tobias Springenberg , Alexey Dosovitskiy , Thomas Brox , and Martin A. Riedmiller . 2015 . Striving for Simplicity: The All Convolutional Net . In Proceedings of the International Conference on Learning Representations (ICLR), Workshop Track, Yoshua Bengio and Yann LeCun (Eds.) . San Diego, USA. Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin A. Riedmiller. 2015. Striving for Simplicity: The All Convolutional Net. In Proceedings of the International Conference on Learning Representations (ICLR), Workshop Track, Yoshua Bengio and Yann LeCun (Eds.). San Diego, USA.
- 10.1109/TVCG.2017.2744158 · DOI · Paper page
- Pascal Sturmfels , Scott Lundberg , and Su-In Lee . 2020. Visualizing the Impact of Feature Attribution Baselines. Distill ( 2020 ). https://doi.org/10.23915/distill.00022 https://distill.pub/2020/attribution-baselines. Pascal Sturmfels, Scott Lundberg, and Su-In Lee. 2020. Visualizing the Impact of Feature Attribution Baselines. Distill (2020). https://doi.org/10.23915/distill.00022 https://distill.pub/2020/attribution-baselines. · DOI · Paper page
- Mukund Sundararajan , Ankur Taly , and Qiqi Yan . 2017 . Axiomatic Attribution for Deep Networks . In Proceedings of the International Conference on Machine Learning (ICML). PMLR , Sydney, Australia, 3319–3328. Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic Attribution for Deep Networks. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, Sydney, Australia, 3319–3328.
- 10.1609/aaai.v34i04.6064 · DOI · Paper page
- Philipp Tschandl , Cliff Rosendahl , and Harald Kittler . 2018. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data 5, 1 ( 2018 ). Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. 2018. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data 5, 1 (2018).
- Dimitris Tsipras , Shibani Santurkar , Logan Engstrom , Andrew Ilyas , and Aleksander Madry . 2020 . From ImageNet to Image Classification: Contextualizing Progress on Benchmarks . In Proceedings of the International Conference on Machine Learning (ICML), Vol. 119 . PMLR, Virtual Event, 9625–9635. Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Andrew Ilyas, and Aleksander Madry. 2020. From ImageNet to Image Classification: Contextualizing Progress on Benchmarks. In Proceedings of the International Conference on Machine Learning (ICML), Vol. 119. PMLR, Virtual Event, 9625–9635.
- 10.1007/978-3-540-88693-8_52 · DOI · Paper page
- James Wexler , Mahima Pushkarna , Tolga Bolukbasi , Martin Wattenberg , Fernanda B. Viégas , and Jimbo Wilson . 2020 . The What-If Tool: Interactive Probing of Machine Learning Models . IEEE Transactions on Visualization and Computer Graphics 26 , 1(2020), 56 – 65 . James Wexler, Mahima Pushkarna, Tolga Bolukbasi, Martin Wattenberg, Fernanda B. Viégas, and Jimbo Wilson. 2020. The What-If Tool: Interactive Probing of Machine Learning Models. IEEE Transactions on Visualization and Computer Graphics 26, 1(2020), 56–65.
- 10.1109/TVCG.2017.2744878 · DOI · Paper page
- Mengjiao Yang and Been Kim. 2019. Benchmarking Attribution Methods with Relative Feature Importance. arxiv:1907.09701 [cs.LG] Mengjiao Yang and Been Kim. 2019. Benchmarking Attribution Methods with Relative Feature Importance. arxiv:1907.09701 [cs.LG] · arXiv
- Matthew D. Zeiler and Rob Fergus. 2014. Visualizing and Understanding Convolutional Networks . In Proceedings of the European Conference on Computer Vision (ECCV), Vol. 8689 . Springer, Zurich, Switzerland, 818–833. Matthew D. Zeiler and Rob Fergus. 2014. Visualizing and Understanding Convolutional Networks. In Proceedings of the European Conference on Computer Vision (ECCV), Vol. 8689. Springer, Zurich, Switzerland, 818–833.
Cited by
Artificial Intelligence and Biosensors in Healthcare and Its Clinical Relevance: A Review
(2023) · doi:10.1109/access.2023.3285596 · Cited by 205
Ma, S., Chen, Q., Wang, X., Zheng, C., Peng, Z., Yin, M., & Ma, X. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713423 · Cited by 48
A Missing Piece in the Puzzle: Considering the Role of Task Complexity in Human-AI Decision Making
Salimzadeh, S., He, G., & Gadiraju, U. (2023) · Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3565472.3592959 · Cited by 42
Salimzadeh, S., He, G., & Gadiraju, U. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641905 · Cited by 41
“Although a few studies have included multiple tasks with varying attributes [6, 14, 131], a systematic and empirical understanding of task features is notably absent from existing literature [68, 109].”
BackgroundTurri, V., Morrison, K., Robinson, K.-M., Abidi, C., Perer, A., Forlizzi, J., & Dzombak, R. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658985 · Cited by 10
Metadata
- Authors
- Boggust, A., Hoover, B., Satyanarayan, A., & Strobelt, H.
- Affiliations
- CSAIL, Massachusetts Institute of Technology, United States
- IBM Research, United States
- Venue
- CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2022-04-27
- Pages
- 1-17
- Cited by
- 45
- References
- 15
- Funders
- United States Air Force Research Laboratory and United States Air Force Artificial Intelligence Accelerator (FA8750-19-2-1000)
- MIT-IBM Watson AI Lab (Towards Intuitive AI)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.