Towards relatable explainable AI with the perceptual process
Open accessAbstract
Machine learning models need to provide contrastive explanations, since people often seek to understand why a puzzling prediction occurred instead of some expected outcome. Current contrastive explanations are rudimentary comparisons between examples or raw features, which remain difficult to interpret, since they lack semantic meaning. We argue that explanations must be more relatable to other concepts, hypotheticals, and associations. Inspired by the perceptual process from cognitive psychology, we propose the XAI Perceptual Processing Framework and RexNet model for relatable explainable AI with Contrastive Saliency, Counterfactual Synthetic, and Contrastive Cues explanations. We investigated the application of vocal emotion recognition, and implemented a modular multi-task deep neural network to predict and explain emotions from speech. From think-aloud and controlled studies, we found that counterfactual explanations were useful and further enhanced with semantic cues, but not saliency explanations. This work provides insights into providing and evaluating relatable contrastive explainable AI for perception applications.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Example-based and counterexample-based explainability methods are widely used in audio-based emotion recognition to generate sonified explanations.
“example-based and counterexample-based explainability methods are widely used in audio-based emotion recognition [73] to generate sonified explanations.”
From Explainable and Interpretable AI for Voice and Speech Analysis in Clinical Care: Systematic Review (2026) · cites this paper as [73]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513 · see citing works
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714 · see citing works
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428 · see citing works
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · see citing works
Picard, R. W. (1997) · MIT Press · doi:10.7551/mitpress/1140.001.0001 · Cited by 7,382 · see citing works
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682 · see citing works
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · see citing works
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · see citing works
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302 · see citing works
On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., & Samek, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0130140 · Cited by 4,524 · see citing works
StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation
Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., & Choo, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00916 · Cited by 4,163 · see citing works
Axiomatic attribution for deep networks
Sundararajan, M., Taly, A., & Yan, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.01365 · Cited by 4,087 · see citing works
CNN architectures for large-scale audio classification
Hershey, S., Chaudhuri, S., Ellis, D. P. W., Gemmeke, J. F., Jansen, A., Moore, R. C., Plakal, M., Platt, D., Saurous, R. A., Seybold, B., et al. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7952132 · Cited by 2,604 · see citing works
Principles of Gestalt Psychology
Pratt, C. C., & Koffka, K. (1936) · The American Journal of Psychology · doi:10.2307/1415906 · Cited by 2,215 · see citing works
Deep Speech 2: End-to-End Speech Recognition in English and MandarinPreprint
Amodei, D., Anubhai, R., Battenberg, E., Case, C. T., Casper, J., Catanzaro, B., Chen, J., Chrzanowski, M., Coates, A., Diamos, G., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.02595 · Cited by 2,191 · see citing works
Anchors: High-Precision Model-Agnostic Explanations
Ribeiro, M. T., Singh, S., & Guestrin, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11491 · Cited by 2,162 · cited 2× · see citing works
“Current counterfactual methods focus on structured (tabular) data by minimizing changes to the target instance [76, 103], or identifying anchor rules [91], but this is not possible for unstructured data (e.”
BackgroundMethodLivingstone, S. R., & Russo, F. (2018) · PLoS ONE · doi:10.1371/journal.pone.0196391 · Cited by 1,955 · see citing works
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514 · see citing works
DARPA's Explainable Artificial Intelligence Program
Gunning, D., & Aha, D. W. (2019) · AI Magazine · doi:10.1609/aimag.v40i2.2850 · Cited by 1,360 · see citing works
Explaining nonlinear classification decisions with deep taylor decomposition
Montavon, G., Lapuschkin, S., Binder, A., Samek, W., & Müller, K.-R. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.11.008 · Cited by 1,357 · see citing works
Narrative Visualization: Telling Stories with Data
Segel, E., & Heer, J. (2010) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2010.179 · Cited by 1,334 · see citing works
Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., & Sayres, R. (2017) · arXiv · doi:10.48550/arxiv.1711.11279 · Cited by 1,240 · see citing works
Explaining machine learning classifiers through diverse counterfactual explanations
Mothilal, R. K., Sharma, A., & Tan, C. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372850 · Cited by 1,233 · see citing works
Wang, R., Chen, F., Chen, Z., Li, T., Harari, G., Tignor, S., Zhou, X., Ben-Zeev, D., & Campbell, A. T. (2014) · Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2632048.2632054 · Cited by 1,116 · see citing works
Counterfactual explanations without opening the black box: automated decisions and the GDPR
Wachter, S., Mittelstadt, B., & Russell, C. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3063289 · Cited by 1,061 · see citing works
Speech emotion recognition using deep 1D & 2D CNN LSTM networks
Zhao, J., Mao, X., & Chen, L. (2019) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2018.08.035 · Cited by 1,044 · see citing works
Visual interpretability for deep learning: a survey
Zhang, Q., & Zhu, S. C. (2018) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.1700808 · Cited by 975 · see citing works
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925 · see citing works
Designing Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906 · see citing works
Self-citationSpeaker Recognition from Raw Waveform with SincNet
Ravanelli, M., & Bengio, Y. (2018) · 2018 IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639585 · Cited by 896 · see citing works
Olah, C., Mordvintsev, A., & Schubert, L. (2017) · Distill · doi:10.23915/distill.00007 · Cited by 880 · see citing works
AttGAN: Facial attribute editing by only changing what you want
He, Z., Zuo, W., Kan, M., Shan, S., & Chen, X. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2916751 · Cited by 833 · see citing works
Automatic speech emotion recognition using recurrent neural networks with local attention
Mirsamadi, S., Barsoum, E., & Zhang, C. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952552 · Cited by 770 · see citing works
Letham, B., Rudin, C., McCormick, T. H., & Madigan, D. (2015) · The Annals of Applied Statistics · doi:10.1214/15-aoas848 · Cited by 766 · see citing works
Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743 · see citing works
Self-citationManipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · see citing works
Beyond the right hemisphere: brain mechanisms mediating vocal emotional processing
Schirmer, A., & Kotz, S. A. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.11.009 · Cited by 694 · see citing works
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · see citing works
The rational imagination: how people create alternatives to reality
(2006) · Choice Reviews Online · doi:10.5860/choice.43-2737 · Cited by 603 · see citing works
Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers
Hohman, F., Kahng, M., Pienta, R., & Chau, D. H. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2843369 · Cited by 582 · see citing works
(2016) · neural information processing systems · Cited by 556 · see citing works
<i>Why and why not</i> explanations improve the intelligibility of context-aware intelligent systems
Lim, B. Y., Dey, A. K., & Avrahami, D. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1519023 · Cited by 552 · see citing works
Self-citationDARPA's explainable artificial intelligence (XAI) program
Gunning, D. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3308446 · Cited by 546 · see citing works
Tell Me Where to Look: Guided Attention Inference Network
Li, K., Wu, Z., Peng, K., Ernst, J., & Fu, Y. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00960 · Cited by 546 · see citing works
Expanding Explainability: Towards Social Transparency in AI systems
Ehsan, U., Liao, Q. V., Muller, M., Riedl, M. O., & Weisz, J. D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445188 · Cited by 538 · cited 2× · see citing works
“3501826 understand user needs [22, 58, 59, 72, 104] and evaluate their helpfulness [2, 11, 12, 18, 21, 64, 83, 105].”
BackgroundMethodUnderstanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527 · cited 2× · see citing works
“However, in a pilot study evaluating with this task, we found that participants may copy the prediction rather than study the explanation, thus leading to over-trusting [110] and diminishing the usefulness of explanations to improve decision quality.”
BackgroundRight for the Right Reasons: Training Differentiable Models by Constraining their Explanations
Ross, A. S., Hughes, M. C., & Doshi-Velez, F. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/371 · Cited by 515 · see citing works
Cognitive Psychology: Connecting Mind, Research and Everyday Experience
Goldstein, E. B. (2004) · Cited by 486 · see citing works
Cialdini, R. B., Wosinska, W., Barrett, D. W., Butner, J., & Gornik-Durose, M. (1999) · Personality and Social Psychology Bulletin · doi:10.1177/0146167299258006 · Cited by 458 · see citing works
Conversational processes and causal explanation.
Hilton, D. J. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.107.1.65 · Cited by 456 · see citing works
Ben-Zeev, D., Scherer, E. A., Wang, R., Xie, H., & Campbell, A. T. (2015) · Psychiatric Rehabilitation Journal · doi:10.1037/prj0000130 · Cited by 429 · see citing works
Speech Emotion Recognition Using CNN
Huang, Z.-W., Dong, M., Mao, Q., & Zhan, Y. (2014) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2647868.2654984 · Cited by 416 · see citing works
Human-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making
Cai, C. J., Reif, E., Hegde, N., Hipp, J., Kim, B., Smilkov, D., Wattenberg, M., Viegas, F., Corrado, G. S., Stumpe, M. C., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300234 · Cited by 405 · see citing works
Multimodal Speech Emotion Recognition Using Audio and Text
Yoon, S., Byun, S., & Jung, K. (2018) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639583 · Cited by 379 · see citing works
Towards Automatic Concept-based Explanations
Ghorbani, A., Wexler, J., Zou, J., & Kim, B. (2019) · arXiv · doi:10.48550/arxiv.1902.03129 · Cited by 378 · see citing works
“StarGAN-VC: non-parallel many-to-many voice conversion using star generative adversarial networks,”
Kameoka, H., Kaneko, T., Tanaka, K., & Hojo, N. (2018) · 2018 IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt.2018.8639535 · Cited by 371 · see citing works
Perceptual Cues in Nonverbal Vocal Expressions of Emotion
Sauter, D. A., Eisner, F., Calder, A. J., & Scott, S. K. (2010) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470211003721642 · Cited by 358 · see citing works
Artificial Intelligence Coaches for Sales Agents: Caveats and Solutions
Luo, X., Qin, M. S., Fang, Z., & Qu, Z. (2020) · Journal of Marketing · doi:10.1177/0022242920956676 · Cited by 338 · see citing works
Wang, X., & Yin, M. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450650 · Cited by 320 · see citing works
Deep Convolutional Neural Networks for Breast Cancer Histology Image Analysis
Rakhlin, A., Shvets, A. A., Iglovikov, V., & Kalinin, A. A. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93000-8_83 · Cited by 314 · see citing works
Juslin, P. N., & Laukka, P. (2001) · Emotion · doi:10.1037/1528-3542.1.4.381 · Cited by 300 · see citing works
Hybrid deep neural networks for face emotion recognition
Jain, N., Kumar, S., Kumar, A., Shamsolmoali, P., & Zareapoor, M. (2018) · Pattern Recognition Letters · doi:10.1016/j.patrec.2018.04.010 · Cited by 299 · see citing works
Explaining Decision-Making Algorithms through UI: Strategies to Help Non-Expert Stakeholders
Cheng, H.-F., Wang, R., Zhang, Z., O'Connell, F., Gray, T., Harper, F. M., & Zhu, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300789 · Cited by 292 · see citing works
End-to-End Speech Emotion Recognition Using Deep Neural Networks
Tzirakis, P., Zhang, J., & Schuller, B. W. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462677 · Cited by 289 · see citing works
EMOTION IN SPEECH: RECOGNITION AND APPLICATION TO CALL CENTERS
Petrushin, V. A. (1999) · Cited by 277 · see citing works
Interpretable Basis Decomposition for Visual Explanation
Zhou, B., Sun, Y., Bau, D., & Torralba, A. B. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_8 · Cited by 270 · see citing works
Principles of Gestalt Psychology.
Sandiford, P. (1936) · Journal of Educational Psychology · doi:10.1037/h0052629 · Cited by 267 · see citing works
Mental comparison and the symbolic distance effect
Moyer, R. S., & Bayer, R. H. (1976) · Cognitive Psychology · doi:10.1016/0010-0285(76)90025-6 · Cited by 260 · see citing works
Bringing Transparency Design into Practice
Eiband, M., Schneider, H., Bilandzic, M., Fazekas-Con, J., Haug, M., & Hussmann, H. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172961 · Cited by 250 · see citing works
Trust-inspiring explanation interfaces for recommender systems
Pu, P., & Chen, L. (2007) · Knowledge-Based Systems · doi:10.1016/j.knosys.2007.04.004 · Cited by 249 · see citing works
Assessing demand for intelligibility in context-aware applications
Lim, B. Y., & Dey, A. K. (2009) · Proceedings of the 11th international conference on Ubiquitous computing · doi:10.1145/1620545.1620576 · Cited by 246 · see citing works
Self-citationRationalization: A Neural Machine Translation Approach to Generating Natural Language Explanations
Ehsan, U., Harrison, B., Chan, L., & Riedl, M. O. (2017) · AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278736 · Cited by 225 · see citing works
The effects of example-based explanations in a machine learning interface
Cai, C. J., Jongejan, J., & Holbrook, J. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302289 · Cited by 220 · see citing works
ELEGANT: Exchanging Latent Encodings with GAN for Transferring Multiple Face Attributes
Xiao, T., Hong, J., & Ma, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01249-6_11 · Cited by 204 · see citing works
A neural network approach to ordinal regression
Cheng, J. J., Wang, Z., & Pollastri, G. (2008) · IEEE International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2008.4633963 · Cited by 194 · see citing works
Heart sound analysis for symptom detection and computer-aided diagnosis
Reed, T. R., Reed, N. E., & Fritzson, P. (2004) · Simulation Modelling Practice and Theory · doi:10.1016/j.simpat.2003.11.005 · Cited by 192 · see citing works
Contrastive explanation: a structural-model approach
Miller, T. (2021) · The Knowledge Engineering Review · doi:10.1017/s0269888921000102 · Cited by 186 · see citing works
On the Time Course of Vocal Emotion Recognition
Pell, M. D., & Kotz, S. A. (2011) · PLoS ONE · doi:10.1371/journal.pone.0027256 · Cited by 185 · see citing works
Richardson, M., Prakash, A., & Brill, E. (2006) · International World Wide Web Conference (WWW) · doi:10.1145/1135777.1135881 · Cited by 168 · see citing works
StarGAN-VC2: Rethinking Conditional Methods for StarGAN-Based Voice Conversion
Kaneko, T., Kameoka, H., Tanaka, K., & Hojo, N. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2236 · Cited by 154 · see citing works
Toolkit to support intelligibility in context-aware applications
Lim, B. Y., & Dey, A. K. (2010) · Proceedings of the 12th ACM international conference on Ubiquitous computing · doi:10.1145/1864349.1864353 · Cited by 143 · see citing works
Self-citationLay understanding of probability distributions
Goldstein, D. G., & Rothschild, D. M. (2014) · Judgment and Decision Making · doi:10.1017/s1930297500004940 · Cited by 116 · see citing works
Deep Convolutional Neural Networks for Breast Cancer Histology Image AnalysisPreprint
Rakhlin, A., Shvets, A. A., Iglovikov, V., & Kalinin, A. A. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/259911 · Cited by 113 · see citing works
Investigating intelligibility for uncertain context-aware applications
Lim, B. Y., & Dey, A. K. (2011) · Proceedings of the 13th international conference on Ubiquitous computing · doi:10.1145/2030112.2030168 · Cited by 101 · see citing works
Self-citationCOGAM: Measuring and Moderating Cognitive Load in Machine Learning Model Explanations
Abdul, A., von der Weth, C., Kankanhalli, M., & Lim, B. Y. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376615 · Cited by 96 · see citing works
Self-citationDamer, T. E. (1987) · Wadsworth Pub. Co. eBooks · Cited by 87 · see citing works
Exploring and Promoting Diagnostic Transparency and Explainability in Online Symptom Checkers
Tsai, C.-H., You, Y., Gui, X., Kou, Y., & Carroll, J. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445101 · Cited by 86 · see citing works
ConvS2S-VC: Fully Convolutional Sequence-to-Sequence Voice Conversion
Kameoka, H., Tanaka, K., Kwaśny, D., Kaneko, T., & Hojo, N. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.3001456 · Cited by 75 · see citing works
Visual dominance: An information-processing account of its origins and significance.
Posner, M. I., Nissen, M. J., & Klein, R. M. (1976) · Psychological Review · doi:10.1037//0033-295x.83.2.157 · Cited by 74 · see citing works
Design of an intelligible mobile context-aware application
Lim, B. Y., & Dey, A. (2011) · Mobile HCI · doi:10.1145/2037373.2037399 · Cited by 72 · see citing works
Self-citationMental Models of Mere Mortals with Explanations of Reinforcement Learning
Anderson, A. W., Dodge, J. E., Sadarangani, A., Juozapaitis, Z., Newman, E., Irvine, J., Chattopadhyay, S., Olson, M., Fern, A., & Burnett, M. (2020) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3366485 · Cited by 63 · see citing works
Emotion recognition and confidence ratings predicted by vocal stimulus type and prosodic parameters
Lausen, A., & Hammerschmidt, K. (2020) · Humanities and Social Sciences Communications · doi:10.1057/s41599-020-0499-z · Cited by 63 · see citing works
Le, T., Wang, S., & Lee, D. (2020) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3394486.3403066 · Cited by 57 · see citing works
Compensating for speaker or lexical variabilities in speech for emotion recognition
Mariooryad, S., & Busso, C. (2013) · Speech Communication · doi:10.1016/j.specom.2013.07.011 · Cited by 52 · see citing works
Coppers, S., den Bergh, J. V., Luyten, K., Coninx, K., van der Lek-Ciudin, I., Vanallemeersch, T., & Vandeghinste, V. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174098 · Cited by 46 · see citing works
Contrastive Explanations In Neural Networks
Prabhushankar, M., Kwon, G., Temel, D., & AlRegib, G. (2020) · Proceedings - International Conference on Image Processing · doi:10.1109/icip40778.2020.9190927 · Cited by 45 · see citing works
Ma‐Kellams, C., & Lerner, J. S. (2016) · Journal of Personality and Social Psychology · doi:10.1037/pspi0000063 · Cited by 41 · see citing works
On Learning Interpretable CNNs with Parametric Modulated Kernel-Based Filters
Loweimi, E., Bell, P., & Renals, S. J. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-1257 · Cited by 40 · see citing works
Nouparametric modeling of vibration signal features for equipment health monitoring
Wegerich, S., Wilks, A. D., & Pipke, R. M. (2004) · Proceedings - IEEE Aerospace Conference · doi:10.1109/aero.2003.1234154 · Cited by 32 · see citing works
Evaluating Intelligibility Usage and Usefulness in a Context-Aware Application
Lim, B. Y., & Dey, A. K. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39342-6_11 · Cited by 29 · see citing works
Self-citationLi, C.-Y., Yuan, P.-C., & Lee, H.-Y. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054675 · Cited by 27 · see citing works
Interpretable Representation Learning for Speech and Audio Signals Based on Relevance Weighting
Agrawal, P., & Ganapathy, S. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.3030489 · Cited by 23 · see citing works
Show or suppress? Managing input uncertainty in machine learning model explanations
Wang, D., Zhang, W., & Lim, B. Y. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103456 · Cited by 22 · see citing works
Self-citationIntrospection for convolutional automatic speech recognition
Krug, A., & Stober, S. (2018) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/w18-5421 · Cited by 16 · see citing works
Proceedings of the 2018 ICML Workshop on Human Interpretability in Machine Learning (WHI 2018)Preprint
Kim, B., Varshney, K. R., & Weller, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.01308 · Cited by 14 · see citing works
Attention-based Sequence Classification for Affect Detection
Gorrostieta, C., Brutti, R., Taylor, K. M., et al. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-1610 · Cited by 12 · see citing works
(1978) · Elsevier eBooks · doi:10.1016/c2013-0-04945-6 · Cited by 5 · see citing works
Ma‐Kellams, C., & Lerner, J. S. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2782596 · Cited by 4 · see citing works
Debiased-CAM for bias-agnostic faithful visual explanations of deep convolutional networks.Preprint
Zhang, W., Dimiccoli, M., & Lim, B. Y. (2020) · arXiv (Cornell University) · Cited by 1 · see citing works
Self-citationBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
Research works printed but not linked: 8
- Sven Coppers, Jan Van den Bergh, Kris Luyten, Karin Coninx, Iulianna Van der Lek-Ciudin, Tom Vanallemeersch, and Vincent Vandeghinste. 2018. Intellingo: an intelligible translation environment. In Proceedings of the 2018 CHI Conference CHI '22, April 29-May 5, 2022, New Orleans, LA, USA on Human Factors in Computing Systems. 1-13.
- Dedre Gentner and Linsey Smith. 2012. Analogical reasoning. Encyclopedia of human behavior 2 (2012), 130-136.
- Yash Goyal, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019. Counterfactual visual explanations. In International Conference on Machine Learning. PMLR, 2376-2384.
- Lisa Anne Hendricks, Ronghang Hu, Trevor Darrell, and Zeynep Akata. 2018. Generating Counterfactual Explanations with Natural Language. In ICML Work- shop on Human Interpretability in Machine Learning. 95-98.
- Andreas Krug, René Knaebel, and Sebastian Stober. 2018. Neuron activation profiles for interpreting convolutional speech recognition models. In NeurIPS Workshop on Interpretability and Robustness in Audio, Speech, and Language (IRASL).
- Zachary C Lipton. 2018. The Mythos of Model Interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue 16, 3 (2018), 31-57.
- Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. 2017. Feature visual- ization. Distill 2, 11 (2017), e7.
- Alec Radford, Luke Metz, and Soumith Chintala. 2015. Unsupervised represen- tation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015). · arXiv
Cited by
Sorted by citations
Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review
Verma, S., Boonsanong, V., Hoang, M., Hines, K. E., Dickerson, J. P., & Shah, C. (2020) · arXiv · doi:10.48550/arxiv.2010.10596 · Cited by 267 · see citing works
Towards Human-Centered Explainable AI: A Survey of User Studies for Model Explanations
Rong, Y., Leemann, T., Nguyen, T.-T., Fiedler, L., Qian, P., Unhelkar, V., Seidel, T., Kasneci, G., & Kasneci, E. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3331846 · Cited by 278 · see citing works
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Chen, V., Liao, Q. V., Wortman Vaughan, J., & Bansal, G. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610219 · Cited by 145 · see citing works
Explainable and Interpretable AI for Voice and Speech Analysis in Clinical Care: Systematic Review
Ebraheem, M., Toghranegar, J., Bridge2AI-Voice Consortium, Bensoussan, Y., & Templeton, J. M. (2026) · Journal of Medical Internet Research · doi:10.2196/83790
- Authors
- Zhang, W., & Lim, B. Y.
- Affiliations
- National University of Singapore (SG)
- Venue
- CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2022-04-29
- Pages
- 1-24
- OpenAlex
- W4225014345
- Cited by
- 67
- References
- 114
- Citations by year
- 2026: 12 · 2025: 17 · 2024: 14 · 2023: 19 · 2022: 5
- FWCI
- 7.60
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Ministry of Education, Singapore (T2EP20121-0040)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- explainable AI, contrastive explanations, perceptual processing, cognitive psychology, counterfactual explanations, semantic cues, vocal emotion recognition, think-aloud study, multi-task deep neural network
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
