Shared Interest: Measuring Human-AI Alignment to Identify Recurring Patterns in Model Behavior
The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Saliency methods — techniques to identify the importance of input features on a model’s output — are a common step in understanding neural network behavior. However, interpreting saliency requires tedious manual inspection to identify and aggregate patterns in model behavior, resulting in ad hoc or cherry-picked analysis. To address these concerns, we present Shared Interest: metrics for comparing model reasoning (via saliency) to human reasoning (via ground truth annotations). By providing quantitative descriptors, Shared Interest enables ranking, sorting, and aggregating inputs, thereby facilitating large-scale systematic analysis of model behavior. We use Shared Interest to identify eight recurring patterns in model behavior, such as cases where contextual features or a subset of ground truth features are most important to the model. Working with representative real-world users, we show how Shared Interest can be used to decide if a model is trustworthy, uncover issues missed in manual analyses, and enable interactive probing.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Proper assessment and presentation of human-AI WoE differences can help effectively navigate humans' attention and efforts in the subsequent activities [14].”
From Towards Human-AI Deliberation: Design and Evaluation of LLM-Empowered Deliberative AI for AI-Assisted Decision-Making (2025) · cites this paper as [14]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2019) · arXiv · doi:10.48550/arxiv.1908.09635 · Cited by 5,812
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812
Codella, N. C. F., Gutman, D., Celebi, M. E., Helba, B., Marchetti, M. A., Dusza, S. W., Kalloo, A., Liopyris, K., Mishra, N., Kittler, H., et al. (2018) · 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) · doi:10.1109/isbi.2018.8363547 · Cited by 2,620
SmoothGrad: removing noise by adding noise
Smilkov, D., Thorat, N., Kim, B., Viégas, F., & Wattenberg, M. (2017) · arXiv · doi:10.48550/arxiv.1706.03825 · Cited by 1,467 · cited 1×
“In this paper, we focus on post hoc saliency methods, also known as feature attribution methods [40], that allow us to observe model reasoning [7, 12, 26, 34, 35, 37, 38, 41].”
MethodNetwork Dissection: Quantifying Interpretability of Deep Visual Representations
Bau, D., Zhou, B., Khosla, A., Oliva, A., & Torralba, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.354 · Cited by 1,389
Explainable AI: From black box to glass box
Rai, A. (2020) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-019-00710-5 · Cited by 1,317
Sanity Checks for Saliency Maps
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., & Kim, B. (2018) · arXiv · doi:10.48550/arxiv.1810.03292 · Cited by 1,294 · cited 3×
“However, researchers have demonstrated cases where saliency methods do not accurately refect the model [1, 22].”
BackgroundCodella, N., Rotemberg, V., Tschandl, P., Celebi, M. E., Dusza, S., Gutman, D., Helba, B., Kalloo, A., Liopyris, K., Marchetti, M., et al. (2019) · arXiv · doi:10.48550/arxiv.1902.03368 · Cited by 984
Research works printed but not linked: 30
- Brandon Carter , Siddhartha Jain , Jonas Mueller , and David Gifford . 2021 . Overinterpretation reveals image classification model pathologies . In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). Virtual Event. Brandon Carter, Siddhartha Jain, Jonas Mueller, and David Gifford. 2021. Overinterpretation reveals image classification model pathologies. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). Virtual Event.
- Brandon Carter , Jonas Mueller , Siddhartha Jain , and David K. Gifford . 2019 . What made you do this? Understanding black-box decisions with sufficient input subsets . In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS). PMLR , Naha, Japan, 567–576. Brandon Carter, Jonas Mueller, Siddhartha Jain, and David K. Gifford. 2019. What made you do this? Understanding black-box decisions with sufficient input subsets. In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS). PMLR, Naha, Japan, 567–576.
- Finale Doshi-Velez and Been Kim. 2017. Towards A Rigorous Science of Interpretable Machine Learning. arxiv:1702.08608 [stat.ML] Finale Doshi-Velez and Been Kim. 2017. Towards A Rigorous Science of Interpretable Machine Learning. arxiv:1702.08608 [stat.ML] · arXiv
- Andre Esteva , Brett Kuprel , Roberto A. Novoa , Justin Ko , Susan M. Swetter , Helen M. Blau , and Sebastian Thrun . 2017. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 7639 ( 2017 ), 115–118. Andre Esteva, Brett Kuprel, Roberto A. Novoa, Justin Ko, Susan M. Swetter, Helen M. Blau, and Sebastian Thrun. 2017. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 7639 (2017), 115–118.
- 10.1109/TVCG.2019.2934659 · DOI · Paper page
- 10.1109/TVCG.2017.2744718 · DOI · Paper page
- 10.1109/TVCG.2018.2864500 · DOI · Paper page
- Been Kim , Martin Wattenberg , Justin Gilmer , Carrie Cai , James Wexler , Fernanda Viegas , and Rory Sayres . 2018 . Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV) . In Proceedings of the International Conference on Machine Learning (ICML). PMLR , Stockholm, Sweden, 2668–2677. Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, and Rory Sayres. 2018. Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV). In Proceedings of the International Conference on Machine Learning (ICML). PMLR, Stockholm, Sweden, 2668–2677.
- Pieter-Jan Kindermans , Sara Hooker , Julius Adebayo , Maximilian Alber , Kristof T. Schütt , Sven Dähne , Dumitru Erhan , and Been Kim . 2019. The (Un)reliability of Saliency Methods . In Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Vol. 11700 . Springer , 267–280. Pieter-Jan Kindermans, Sara Hooker, Julius Adebayo, Maximilian Alber, Kristof T. Schütt, Sven Dähne, Dumitru Erhan, and Been Kim. 2019. The (Un)reliability of Saliency Methods. In Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Vol. 11700. Springer, 267–280.
- Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization . In Proceedings of the International Conference on Learning Representations (ICLR) . San Diego, USA. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In Proceedings of the International Conference on Learning Representations (ICLR). San Diego, USA.
- Scott M. Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions . In Proceedings of the Conference on Neural Information Processing Systems (NIPS) . Long Beach, USA, 4765–4774. Scott M. Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In Proceedings of the Conference on Neural Information Processing Systems (NIPS). Long Beach, USA, 4765–4774.
- 10.1109/ICDM.2012.110 · DOI · Paper page
- Christopher Meek. 2016. A Characterization of Prediction Errors. arxiv:1611.05955 [cs.LG] Christopher Meek. 2016. A Characterization of Prediction Errors. arxiv:1611.05955 [cs.LG] · arXiv
- Adam Paszke , Sam Gross , Francisco Massa , Adam Lerer , James Bradbury , Gregory Chanan , Trevor Killeen , Zeming Lin , Natalia Gimelshein , Luca Antiga , Alban Desmaison , Andreas Köpf , Edward Z. Yang , Zachary DeVito , Martin Raison , Alykhan Tejani , Sasank Chilamkurthy , Benoit Steiner , Lu Fang , Junjie Bai , and Soumith Chintala . 2019 . PyTorch: An Imperative Style, High-Performance Deep Learning Library . In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS) . Vancouver, Canada, 8024–8035. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS). Vancouver, Canada, 8024–8035.
- 10.1109/TVCG.2016.2598828 · DOI · Paper page
- 10.1145/2939672.2939778 · DOI · Paper page
- 10.1109/ICCV.2017.74 · DOI · Paper page
- Karen Simonyan , Andrea Vedaldi , and Andrew Zisserman . 2014 . Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps . In Proceedings of the International Conference on Learning Representations (ICLR), Workshop Track . Banff, Canada. Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014. Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. In Proceedings of the International Conference on Learning Representations (ICLR), Workshop Track. Banff, Canada.
- Jost Tobias Springenberg , Alexey Dosovitskiy , Thomas Brox , and Martin A. Riedmiller . 2015 . Striving for Simplicity: The All Convolutional Net . In Proceedings of the International Conference on Learning Representations (ICLR), Workshop Track, Yoshua Bengio and Yann LeCun (Eds.) . San Diego, USA. Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin A. Riedmiller. 2015. Striving for Simplicity: The All Convolutional Net. In Proceedings of the International Conference on Learning Representations (ICLR), Workshop Track, Yoshua Bengio and Yann LeCun (Eds.). San Diego, USA.
- 10.1109/TVCG.2017.2744158 · DOI · Paper page
- Pascal Sturmfels , Scott Lundberg , and Su-In Lee . 2020. Visualizing the Impact of Feature Attribution Baselines. Distill ( 2020 ). https://doi.org/10.23915/distill.00022 https://distill.pub/2020/attribution-baselines. Pascal Sturmfels, Scott Lundberg, and Su-In Lee. 2020. Visualizing the Impact of Feature Attribution Baselines. Distill (2020). https://doi.org/10.23915/distill.00022 https://distill.pub/2020/attribution-baselines. · DOI · Paper page
- Mukund Sundararajan , Ankur Taly , and Qiqi Yan . 2017 . Axiomatic Attribution for Deep Networks . In Proceedings of the International Conference on Machine Learning (ICML). PMLR , Sydney, Australia, 3319–3328. Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic Attribution for Deep Networks. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, Sydney, Australia, 3319–3328.
- 10.1609/aaai.v34i04.6064 · DOI · Paper page
- Philipp Tschandl , Cliff Rosendahl , and Harald Kittler . 2018. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data 5, 1 ( 2018 ). Philipp Tschandl, Cliff Rosendahl, and Harald Kittler. 2018. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data 5, 1 (2018).
- Dimitris Tsipras , Shibani Santurkar , Logan Engstrom , Andrew Ilyas , and Aleksander Madry . 2020 . From ImageNet to Image Classification: Contextualizing Progress on Benchmarks . In Proceedings of the International Conference on Machine Learning (ICML), Vol. 119 . PMLR, Virtual Event, 9625–9635. Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Andrew Ilyas, and Aleksander Madry. 2020. From ImageNet to Image Classification: Contextualizing Progress on Benchmarks. In Proceedings of the International Conference on Machine Learning (ICML), Vol. 119. PMLR, Virtual Event, 9625–9635.
- 10.1007/978-3-540-88693-8_52 · DOI · Paper page
- James Wexler , Mahima Pushkarna , Tolga Bolukbasi , Martin Wattenberg , Fernanda B. Viégas , and Jimbo Wilson . 2020 . The What-If Tool: Interactive Probing of Machine Learning Models . IEEE Transactions on Visualization and Computer Graphics 26 , 1(2020), 56 – 65 . James Wexler, Mahima Pushkarna, Tolga Bolukbasi, Martin Wattenberg, Fernanda B. Viégas, and Jimbo Wilson. 2020. The What-If Tool: Interactive Probing of Machine Learning Models. IEEE Transactions on Visualization and Computer Graphics 26, 1(2020), 56–65.
- 10.1109/TVCG.2017.2744878 · DOI · Paper page
- Mengjiao Yang and Been Kim. 2019. Benchmarking Attribution Methods with Relative Feature Importance. arxiv:1907.09701 [cs.LG] Mengjiao Yang and Been Kim. 2019. Benchmarking Attribution Methods with Relative Feature Importance. arxiv:1907.09701 [cs.LG] · arXiv
- Matthew D. Zeiler and Rob Fergus. 2014. Visualizing and Understanding Convolutional Networks . In Proceedings of the European Conference on Computer Vision (ECCV), Vol. 8689 . Springer, Zurich, Switzerland, 818–833. Matthew D. Zeiler and Rob Fergus. 2014. Visualizing and Understanding Convolutional Networks. In Proceedings of the European Conference on Computer Vision (ECCV), Vol. 8689. Springer, Zurich, Switzerland, 818–833.
Cited by
Artificial Intelligence and Biosensors in Healthcare and Its Clinical Relevance: A Review
(2023) · doi:10.1109/access.2023.3285596 · Cited by 205
Ma, S., Chen, Q., Wang, X., Zheng, C., Peng, Z., Yin, M., & Ma, X. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713423 · Cited by 48
A Missing Piece in the Puzzle: Considering the Role of Task Complexity in Human-AI Decision Making
Salimzadeh, S., He, G., & Gadiraju, U. (2023) · Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3565472.3592959 · Cited by 42
“Lastly, we can see distinct types of images in image classification tasks [3, 13], from clinical, nutrition-related, animal, and animal images.”
BackgroundSalimzadeh, S., He, G., & Gadiraju, U. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641905 · Cited by 41
“Although a few studies have included multiple tasks with varying attributes [6, 14, 131], a systematic and empirical understanding of task features is notably absent from existing literature [68, 109].”
BackgroundTurri, V., Morrison, K., Robinson, K.-M., Abidi, C., Perer, A., Forlizzi, J., & Dzombak, R. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658985 · Cited by 10
On the Need to Rethink Trust in AI Assistants for Software Development: A Critical Review
Baltes, S., Speith, T., Chiteri, B., Mohsenimofidi, S., Chakraborty, S., & Buschek, D. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3659804 · Cited by 5
“…[23], [29], [55], [63], [66], [72], [87], [156], [162] [26], [59] None None SE-AI [3], [66] [71], [83], [88], [90], [175] [91] None SE-CR [4], [22], [78], [111], [165] None None None PH+PS [30], [38], [56], [62], [86], [96], [110], [117], [118], [124], [133], [161], [164], [169] None None…”
A Case Report On The "A.I. Locked-In Problem": social concerns with modern NLP
Walter, Y. (2022) · arXiv · doi:10.48550/arxiv.2209.12687 · Cited by 1
Metadata
- Authors
- Boggust, A., Hoover, B., Satyanarayan, A., & Strobelt, H.
- Affiliations
- CSAIL, Massachusetts Institute of Technology, United States
- IBM Research, United States
- Venue
- CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2022-04-27
- Pages
- 1-17
- Cited by
- 45
- References
- 21
- Funders
- United States Air Force Research Laboratory and United States Air Force Artificial Intelligence Accelerator (FA8750-19-2-1000)
- MIT-IBM Watson AI Lab (Towards Intuitive AI)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.