A Deep Multi-Modal Explanation Model for Zero-Shot Learning
Open accessAbstract
Zero-shot learning (ZSL) has attracted significant attention due to its capabilities of classifying new images from unseen classes. To perform the classification task for ZSL, learning visual and semantic embeddings has been the main research approach in existing literature. At the same time, generating complementary explanations to justify the classification decision has remained largely unexplored. In this paper, we propose to address a new and challenging task, namely explainable zero-shot learning (XZSL), which aims to generate visual and textual explanations to support the classification decision. To accomplish this task, we build a novel Deep Multi-modal Explanation (DME) model that incorporates a joint visual-attribute embedding module and a multi-channel explanation module in an end-to-end fashion. In contrast to existing ZSL approaches, our visual-attribute embedding is associated not only with the decision, but also with new visual and textual explanations. For visual explanations, we first capture several attribute activation maps (AAM) and then merge them into a class activation map (CAM) that visually infers which region of an image is relevant to the class. Textual explanations are generated from the multi-channel explanation module, jointly integrating three long short-term memory models (LSTMs) each of which is conditioned on a different feature representation. Additionally, we suggest that the DME model can retain explanatory consistency for similar instances and explanatory diversity for diverse instances. We conduct qualitative and quantitative experiments to assess the model for ZSL classification and explanation. Specifically, the ablation studies verify the effectiveness of the components in our model. Our results on three well-known datasets are competitive with prior approaches. More importantly, the joint training of our embedding and explanation modules demonstrates mutual performance improvements between ZSL classification and explanation. We shed more light on DME to analyze and diagnose its advantages and limitations.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The objective of zero-shot learning is to project a set of synthesized features in order to make the model more generalizable toward recognition of unseen classes in test samples.
“The objective is to project a set of synthesized features in order to make the model more generalizable toward the recognition of the unseen class in test samples [135].”
From A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets (2022) · cites this paper as [135]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
(2008) · Journal of Machine Learning Research · Cited by 33,051 · see citing works
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · see citing works
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513 · see citing works
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
Labaca-Castro, R. (2023) · Springer eBooks · doi:10.1007/978-3-658-40442-0_9 · Cited by 21,100 · see citing works
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067 · see citing works
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,268 · see citing works
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428 · see citing works
(2004) · Cited by 9,718 · see citing works
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · see citing works
CIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845 · see citing works
A Discriminative Feature Learning Approach for Deep Face Recognition
Wen, Y., Zhang, K., Li, Z., & Qiao, Y. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46478-7_31 · Cited by 4,077 · see citing works
Caltech-UCSD Birds-200-2011 Dataset
Wah, C. (2024) · CaltechAUTHORS (California Institute of Technology) · doi:10.57702/i9oeewf6 · Cited by 3,875 · see citing works
Bleu: a method for automatic evaluation of machine translation
Papineni, K. (2024) · doi:10.57702/ljteqmrb · Cited by 3,682 · see citing works
DeViSE: A Deep Visual-Semantic Embedding Model
Frome, A., Corrado, G. S., Shlens, J., Samy Bengio, A., Dean, J., Ranzato, M., & Mikolov, T. (2013) · Cited by 2,309 · see citing works
Meteor Universal: Language Specific Translation Evaluation for Any Target Language
Denkowski, M., & Lavie, A. (2014) · WMT@ACL · doi:10.3115/v1/w14-3348 · Cited by 2,108 · see citing works
Zero-Shot Learning—A Comprehensive Evaluation of the Good, the Bad and the Ugly
Xian, Y., Lampert, C. H., Schiele, B., & Akata, Z. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2857768 · Cited by 1,678 · see citing works
Attribute-Based Classification for Zero-Shot Visual Object Categorization
Lampert, C. H., Nickisch, H., & Stefan Harmeling, B. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.140 · Cited by 1,639 · see citing works
Hierarchical Question-Image Co-Attention for Visual Question AnsweringPreprint
Lu, J., Yang, J., Batra, D., & Parikh, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.00061 · Cited by 1,303 · see citing works
An Embarrassingly Simple Approach to Zero-Shot Learning
Romera‐Paredes, B., & Torr, P. H. S. (2017) · Advances in computer vision and pattern recognition · doi:10.1007/978-3-319-50077-5_2 · Cited by 1,113 · see citing works
Feature Generating Networks for Zero-Shot Learning
Xian, Y., Lorenz, T., Schiele, B., & Akata, Z. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00581 · Cited by 1,080 · see citing works
Evaluation of output embeddings for fine-grained image classification
Akata, Z., Reed, S., Walter, D., Lee, H., & Schiele, B. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298911 · Cited by 1,011 · see citing works
Zero-Shot Learning Through Cross-Modal Transfer
Socher, R., Ganjoo, M., Sridhar, H., Bastani, O., Manning, C. D., & Ng, A. Y. (2013) · arXiv · doi:10.48550/arxiv.1301.3666 · Cited by 997 · see citing works
Semantic Autoencoder for Zero-Shot Learning
Kodirov, E., Xiang, T., & Gong, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.473 · Cited by 960 · see citing works
SUN attribute database: Discovering, annotating, and recognizing scene attributes
Patterson, G., & Hays, J. H. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247998 · Cited by 878 · see citing works
Learning Deep Representations of Fine-Grained Visual Descriptions
Reed, S., Akata, Z., Lee, H., & Schiele, B. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.13 · Cited by 802 · see citing works
Label-Embedding for Image Classification
Akata, Z., Perronnin, F., Harchaoui, Z., & Schmid, C. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2487986 · Cited by 785 · see citing works
Learning a Deep Embedding Model for Zero-Shot Learning
Zhang, L., Xiang, T., & Gong, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.321 · Cited by 764 · see citing works
Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoders
Schönfeld, E., Ebrahimi, S., Sinha, S., Darrell, T., & Akata, Z. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00844 · Cited by 747 · see citing works
Latent Embeddings for Zero-Shot Classification
Xian, Y., Akata, Z., Sharma, G., Nguyen, Q., Hein, M., & Schiele, B. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.15 · Cited by 723 · see citing works
Synthesized Classifiers for Zero-Shot Learning
Changpinyo, S., Chao, W., Gong, B., & Sha, F. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.575 · Cited by 720 · see citing works
Generative Moment Matching NetworksPreprint
Li, Y., Swersky, K., & Zemel, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.02761 · Cited by 644 · see citing works
F-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learning
Xian, Y., Sharma, S., Schiele, B., & Akata, Z. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01052 · Cited by 579 · see citing works
Evaluation of Output Embeddings for Fine-grained Image Classification
Akata, Z., Reed, S., Walter, D., Lee, H., & Schiele, B. (2015) · Max Planck Digital Library · Cited by 567 · see citing works
Generating Visual Explanations
Hendricks, L. A., Akata, Z., Rohrbach, M., Donahue, J., Schiele, B., & Darrell, T. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_1 · Cited by 534 · see citing works
Generalized Zero-Shot Learning via Synthesized Examples
Verma, V. K., Arora, G., Mishra, A., & Rai, P. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00450 · Cited by 497 · see citing works
Multimodal Explanations: Justifying Decisions and Pointing to the Evidence
Park, D. H., Hendricks, L. A., Akata, Z., Rohrbach, A., Schiele, B., Darrell, T., & Rohrbach, M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00915 · Cited by 418 · see citing works
A Generative Adversarial Approach for Zero-Shot Learning from Noisy Texts
Zhu, Y., Elhoseiny, M., Liu, B., Peng, X., & Elgammal, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00111 · Cited by 412 · see citing works
Predicting Deep Zero-Shot Convolutional Neural Networks Using Textual Descriptions
Ba, J. L., Swersky, K., Fidler, S., & Salakhutdinov, R. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.483 · Cited by 401 · see citing works
Multi-modal Cycle-Consistent Generalized Zero-Shot Learning
Felix, R., Kumar, B. G. V., Reid, I. D., & Carneiro, G. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01231-1_2 · Cited by 391 · see citing works
Leveraging the Invariant Side of Generative Zero-Shot Learning
Li, J., Jing, M., Lü, K., Ding, Z., Zhu, L., & Huang, Z. H. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00758 · Cited by 361 · see citing works
Hierarchical Question-Image Co-Attention for Visual Question Answering
Lu, J. (2024) · TIB Data Manager · doi:10.57702/uhdmfu1r · Cited by 357 · see citing works
Zero-Shot Visual Recognition Using Semantics-Preserving Adversarial Embedding Networks
Chen, L., Zhang, H., Xiao, J., Liu, W., & Chang, S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00115 · Cited by 320 · see citing works
Generalized Zero-Shot Learning with Deep Calibration Network
Liu, S., Long, M., Wang, J., & Jordan, M. I. (2018) · neural information processing systems · Cited by 202 · see citing works
Discriminative Learning of Latent Features for Zero-Shot Recognition
Li, Y., Zhang, J., Zhang, J., & Huang, K. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00779 · Cited by 178 · see citing works
Predicting Visual Exemplars of Unseen Classes for Zero-Shot Learning
Changpinyo, S., Chao, W., & Sha, F. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.376 · Cited by 174 · see citing works
Hendricks, L. A., Hu, R., Darrell, T., & Akata, Z. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01216-8_17 · Cited by 171 · see citing works
Out-Of-Distribution Detection for Generalized Zero-Shot Action Recognition
Mandal, D., Narayan, S., Dwivedi, S. K., Gupta, V., Ahmed, S., Khan, F. S., & Shao, L. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01022 · Cited by 146 · see citing works
Rethinking Zero-Shot Learning: A Conditional Visual Classification Perspective
Li, K., Min, M. R., & Fu, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00368 · Cited by 144 · see citing works
Building Explainable Artificial Intelligence Systems
Core, M. G., Lane, H. C., van Lent, M., Gomboc, D., Solomon, S., & Milton Rosenberg, B. (2006) · doi:10.21236/ada459166 · Cited by 134 · see citing works
Adaptive Confidence Smoothing for Generalized Zero-Shot Learning
Atzmon, Y., & Chechik, G. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01194 · Cited by 120 · see citing works
Zero-Shot Learning With Transferred Samples
Guo, Y., Ding, G., Han, J., & Gao, Y. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2696747 · Cited by 107 · see citing works
Triple Verification Network for Generalized Zero-Shot Learning
Zhang, H., Long, Y., Guan, Y., & Shao, L. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2869696 · Cited by 90 · see citing works
Ghosh, S., Burachas, G., Ray, A., & Ziskind, A. (2019) · arXiv.org · doi:10.48550/arxiv.1902.05715 · Cited by 69 · see citing works
Zero-Shot Learning via Attribute Regression and Class Prototype Rectification
Luo, C., Li, Z., Huang, K., Feng, J., & Wang, M. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2745109 · Cited by 63 · see citing works
Zero-Shot Learning via Robust Latent Representation and Manifold Regularization
Meng, M., & Yu, J. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2881926 · Cited by 49 · see citing works
Choose Your Neuron: Incorporating Domain Knowledge Through Neuron-Importance
Selvaraju, R. R., Chattopadhyay, P., Elhoseiny, M., Sharma, T., Batra, D., Parikh, D., & Lee, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01261-8_32 · Cited by 43 · see citing works
Learning to Explain With Complemental Examples
Kanehira, A., & Harada, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00880 · Cited by 39 · see citing works
Multimodal Explanations by Predicting Counterfactuality in Videos
Kanehira, A., Takemoto, K., Inayoshi, S., & Harada, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00879 · Cited by 33 · see citing works
Zhang, Y., Qian, S., Fang, Q., & Xu, C. (2019) · Proceedings of the 27th ACM International Conference on Multimedia · doi:10.1145/3343031.3351033 · Cited by 33 · see citing works
Zero-Shot Learning via Category-Specific Visual-Semantic Mapping and Label Refinement
Niu, L., Cai, J., Veeraraghavan, A., & Zhang, L. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2872916 · Cited by 31 · see citing works
Zero-shot Learning via Simultaneous Generating and LearningPreprint
Yu, H., & Lee, B.-H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.09446 · Cited by 29 · see citing works
Flagged first, then by citations
2 of the 72 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer, "Automatic differentiation in pytorch," in NIPS-Workshop, 2017.
Cited by
Sorted by citations
Bayoudh, K., Knani, R., Hamdaoui, F., & Mtibaa, A. (2022) · The Visual Computer · doi:10.1007/s00371-021-02166-7 · Cited by 456 · see citing works
A Review on Explainability in Multimodal Deep Neural Nets
Joshi, G., Walambe, R., & Kotecha, K. (2021) · IEEE Access · doi:10.1109/access.2021.3070212 · Cited by 204 · see citing works
Rodis, N., Sardianos, C., Grammatikis, P. I. R., Sarigiannidis, P. G., Varlamis, I., & Papadopoulos, G. T. (2024) · IEEE Access · doi:10.1109/access.2024.3467062 · Cited by 84 · see citing works
Towards Zero-Shot Sign Language Recognition
Bilge, Y. C., Cinbiş, R. G., & İkizler-Cinbiş, N. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3143074 · Cited by 61 · see citing works
A review on multimodal zero‐shot learning
Cao, W., Wu, Y., Sun, Y., Zhang, H., Ren, J., Gu, D., & Wang, X. (2023) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1488 · Cited by 50 · see citing works
Investigating the Bilateral Connections in Generative Zero-Shot Learning
Li, J., Jing, M., Lü, K., Zhu, L., & Shen, H. T. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3050803 · Cited by 47 · see citing works
Visual-Semantic Aligned Bidirectional Network for Zero-Shot Learning
Gao, R., Hou, X., Qin, J., Shen, Y., Long, Y., Liu, L., Zhang, Z., & Shao, L. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3145666 · Cited by 30 · see citing works
Augmented Multimodality Fusion for Generalized Zero-Shot Sketch-Based Visual Retrieval
Jing, T., Xia, H., Hamm, J., & Ding, Z. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3173815 · Cited by 29 · see citing works
Exemplar-Based, Semantic Guided Zero-Shot Visual Recognition
Zhang, C., Liang, C., & Zhao, Y. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3120319 · Cited by 20 · see citing works
Liu, Z., Yang, R., Liu, W., & Liu, X. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126877 · Cited by 15 · see citing works
Kim, J., Kim, S., Kim, S. T., & Ro, Y. M. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3130526 · Cited by 9 · see citing works
Enhanced VAEGAN: a zero-shot image classification method
Ding, B., Fan, Y., He, Y., & Zhao, J. (2022) · Applied Intelligence · doi:10.1007/s10489-022-03869-7 · Cited by 8 · see citing works
Recognizing Unseen Objects via Multimodal Intensive Knowledge Graph Propagation
Wu, L., Li, Z., Zhao, H., Wang, Z., Liu, Q., Huai, B., Yuan, J., & Chen, E. (2023) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3580305.3599486 · Cited by 7 · see citing works
A comprehensive review on zero-shot-learning techniques
Lazaros, K., Koumadorakis, D. E., Vrahatis, A. G., & Kotsiantis, S. B. (2024) · Intelligent Decision Technologies · doi:10.3233/idt-240297 · Cited by 6 · see citing works
Zero-shot learning based on the fusion of global and local representations
Wang, Q., HongJin, M., Wang, J., Chunxiao, W., & Zhou, Y. (2025) · Measurement Science and Technology · doi:10.1088/1361-6501/adb76d · Cited by 4 · see citing works
A transformer-based dual contrastive learning approach for zero-shot learning
Yu, L., Jing, R., Li, F., Gao, Q., & Deng, C. D. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129530 · Cited by 3 · see citing works
GT-free_XAI: A Ground Truth-Free XAI Framework for Decision Interpretation and Evaluation
Wu, Y., & Tian, F. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11209952 · Cited by 2 · see citing works
A survey on multimodal explainable Artificial Intelligence
Giovannoni, C., Metta, C., Monreale, A., & Rinzivillo, S. (2026) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2026.200671 · Cited by 2 · see citing works
Learning exclusive discriminative semantic information for zero-shot learning
Mi, J., Zhang, Z., Tai, D., Zhou, L., & Jia, W. (2022) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-022-01661-0 · Cited by 2 · see citing works
Zero-Shot Learning with Noisy Labels
Pan, Z., Tang, L., & Pan, Z. (2023) · Procedia Computer Science · doi:10.1016/j.procs.2023.08.049 · Cited by 1 · see citing works
Towards Understanding Human-AI Reliance Patterns Through Explanation Styles
Casolin, E. R., & Salim, F. D. (2024) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3675094.3678996 · Cited by 1 · see citing works
SKZC: self-distillation and k-nearest neighbor-based zero-shot classification
Sun, M., & Jia, H. (2024) · Journal of Engineering and Applied Science · doi:10.1186/s44147-024-00429-3 · Cited by 1 · see citing works
A comprehensive review on zero-shot-learning techniques
Lazaros, K., Koumadorakis, D. E., Vrahatis, A. G., & Kotsiantis, S. B. (2024) · Intelligent Decision Technologies · doi:10.3233/idt-24027 · Cited by 1 · see citing works
Zero-Shot Learning-Based Detection of Electric Insulators in the Wild
Azeem, I., & Zaidi, M. A. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-95470-3_16 · Cited by 1 · see citing works
2020 Index IEEE Transactions on Image Processing Vol. 29
(2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3046056
Explaining Deep Neural Networks with Example and Pixel Attribution
Dong, G., Boström, H., Bresson, R., & Alkhatib, A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05461-6_19
Can Multitask Learning Enhance Model Explainability?
Najjar, H., Alshbib, B., & Dengel, A. R. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-12840-9_38
Zero-Shot Image Classification by modeling Implicit Semantic Correlation Transferability
Zhao, L. L., Ban, W., Wenxiang, D., Fang, Z., Zhao, C., & Han, X. D. (2025) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2025.12.460
Pan, Y., Lu, L., Wang, M., Reja, V. K., Noichl, F., Borrmann, A., & Brilakis, I. (2026) · ASCE International Conference on Computing in Civil Engineering (i3CE) · doi:10.1061/9780784486443.029
Leveraging Pre-Trained Vision and Language Models for Zero-Shot Clinical Explanation Generation
Bobde, A., Ninawe, J., Tandulkar, R., Katulwar, A., & Kukade, A. (2025) · DMIHER International Conference on Artificial Intelligence in Healthcare, Education and Industry (IDICAIHEI) · doi:10.1109/idicaihei65991.2025.11378615
Semantic Embedding-Based Zero-Shot and Few-Shot Learning for Cold-Start Movie Recommendation
Bandara, B. C. Y., Yasara, N., Sutharshon, A., Jayasekara, P., & Mahanama, T. V. (2025) · International Conference on Advances in Technology and Computing (ICATC) · doi:10.1109/icatc68823.2025.11407746
- Authors
- Liu, Y., & Tuytelaars, T.
- Affiliations
- KU Leuven (BE)
- Venue
- IEEE Transactions on Image Processing
- Published
- 2020
- Volume
- 29
- Pages
- 4788-4803
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W3007443315
- Cited by
- 31
- References
- 72
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 5 · 2023: 4 · 2022: 7 · 2021: 4 · 2020: 1
- FWCI
- 1.57
- Type
- journal article
- Language
- en
- Open access
- green
- lirias.kuleuven.be
- Licence
- Locations
- IEEE Transactions on Image Processing (publishedVersion)
- PubMed (publishedVersion)
- Lirias (KU Leuven) (submittedVersion, other-oa)
- Funders
- KU Leuven (C14/18/065)
- Flemish Government through the Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaandere Programme
- Fonds Wetenschappelijk Onderzoek (G086617N)
- Nvidia
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- zero-shot learning, visual explanations, textual explanations, class activation maps, long short-term memory, end-to-end learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
