Cited by 334Multimodal Machine Learning ApplicationsAdvanced Image and Video Retrieval TechniquesImage Retrieval and Classification Techniques

The DOI matches its Crossref record; 38 of the 40 references it cites were found in a registry or an index.

Multimodal Convolutional Neural Networks for Matching Image and Sentence

Ma, Lin, Lu, Zhengdong, Shang, Lifeng, Li, Hang·2015·2015 IEEE International Conference on Computer Vision (ICCV)

DOI 10.1109/iccv.2015.301

Open this paper in CoChat — free account

Abstract

In this paper, we propose multimodal convolutional neural networks (m-CNNs) for matching image and sentence. Our m-CNN provides an end-to-end framework with convolutional architectures to exploit image representation, word composition, and the matching relations between the two modalities. More specifically, it consists of one image CNN encoding the image content and one matching CNN modeling the joint representation of image and sentence. The matching CNN composes different semantic fragments from words and learns the inter-modal relations between image and the composed fragments at different levels, thus fully exploit the matching relations between image and sentence. Experimental results demonstrate that the proposed m-CNNs can effectively capture the information necessary for image and sentence matching. More specifically, our proposed m-CNNs significantly outperform the state-of-the-art approaches for bidirectional image and sentence retrieval on the Flickr8K and Flickr30K datasets.

Integrity signals

Registry

Crossref · 10.1109/iccv.2015.301

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
Convolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
Convolutional networks for images, speech, and time series
LeCun, Y., & Bengio, Y. (1998) · HAL · doi:10.5555/303568.303704 · Cited by 4,646

Show all 40 references

Research works printed but not linked: 5

hinton, 2012, Improving neural networks by proventing co-adaptation of feature detecters, arXiv 1207 0580, 6
kiros, 2014, Multimodal neural language model, ICML, 2
sermanet, 2014, Overfeat: Intergrated recognition, localization and detection using convolutional networks, arXiv 1312 6229, 2
girshick, 2014, Rich feature hierachies for accurate object detection and semantic segmentation, CVPR, 8
ordonez, 2011, Im2txt: Describing images using 1 million captioned photogrphs, NIPS, 1

Sorted by citations

A survey on deep learning for multimodal data fusion
Gao, J., Li, P., Chen, Z., & Zhang, J. (2020) · Neural Computation · doi:10.1162/neco_a_01273 · Cited by 821
Unicoder-VL: A Universal Encoder for Vision and Language by Cross-Modal Pre-Training
Li, G., Duan, N., Fang, Y., Gong, M., & Jiang, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6795 · Cited by 795
Learning Deep Structure-Preserving Image-Text Embeddings
Wang, L., Li, Y., & Lazebnik, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.541 · Cited by 795
Adversarial Cross-Modal Retrieval
Wang, B., Yang, Y., Xu, X., Hanjalic, A., & Shen, H. T. (2017) · Proceedings of the 25th ACM international conference on Multimedia · doi:10.1145/3123266.3123326 · Cited by 779
Dual Attention Networks for Multimodal Reasoning and Matching
Nam, H., Ha, J.-W., & Kim, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.232 · Cited by 760
MUTAN: Multimodal Tucker Fusion for Visual Question Answering
Ben-younes, H., Cadène, R., Cord, M., & Thome, N. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.285 · Cited by 753
Dual-path Convolutional Image-Text Embeddings with Instance Loss
Zheng, Z., Zheng, L., Garrett, M., Yang, Y., Xu, M., & Shen, Y.-D. (2020) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3383184 · Cited by 652
Learning Two-Branch Neural Networks for Image-Text Matching Tasks
Wang, L., Li, Y., Huang, J. L., & Lazebnik, S. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2797921 · Cited by 548
Deep Cross-Modal Projection Learning for Image-Text Matching
Zhang, Y., & Lu, H. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_42 · Cited by 520
IMRAM: Iterative Matching With Recurrent Attention Memory for Cross-Modal Image-Text Retrieval
Chen, H., Ding, G., Liu, X., Lin, Z., Liu, J., & Han, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01267 · Cited by 451
A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets
Bayoudh, K., Knani, R., Hamdaoui, F., & Mtibaa, A. (2022) · The Visual Computer · doi:10.1007/s00371-021-02166-7 · Cited by 456
Look, Imagine and Match: Improving Textual-Visual Cross-Modal Retrieval with Generative Models
Gu, J., Cai, J., Joty, S., Niu, L., & Wang, G. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00750 · Cited by 413
TediGAN: Text-Guided Diverse Face Image Generation and Manipulation
Xia, W., Yang, Y., Xue, J.-H., & Wu, B. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00229 · Cited by 376
Stock price prediction using deep learning and frequency decomposition
Rezaei, H., Faaljou, H., & Mansourfar, G. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.114332 · Cited by 398
Learning Semantic Concepts and Order for Image and Sentence Matching
Huang, Y., Wu, Q., Song, C., & Wang, L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00645 · Cited by 356
Reconstruction Network for Video Captioning
Wang, B., Ma, L., Zhang, W., & Liu, W. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00795 · Cited by 353
Self-citation
Forecasting stock prices with a feature fusion LSTM-CNN model using different representations of the same data
Kim, T., & Kim, H. Y. (2019) · PLoS ONE · doi:10.1371/journal.pone.0212320 · Cited by 357
REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments
Qi, Y., Wu, Q., Anderson, P., Wang, X., Wang, W. Y., Shen, C., & van den Hengel, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01000 · Cited by 331
Multilevel Language and Vision Integration for Text-to-Clip Retrieval
Xu, H., He, K., Plummer, B. A., Sigal, L., Sclaroff, S., & Saenko, K. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019062 · Cited by 324
Context-Aware Attention Network for Image-Text Retrieval
Zhang, Q., Lei, Z., Zhang, Z., & Li, S. Z. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00359 · Cited by 308
Graph Structured Network for Image-Text Matching
Liu, C., Mao, Z., Zhang, T., Xie, H., Wang, B., & Zhang, Y. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01093 · Cited by 305
Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models
Plummer, B. A., Wang, L., Cervantes, C. M., Caicedo, J. C., Hockenmaier, J., & Lazebnik, S. (2016) · International Journal of Computer Vision · doi:10.1007/s11263-016-0965-7 · Cited by 429
Order-Embeddings of Images and Language
Vendrov, I., Kiros, R., Fidler, S., & Urtasun, R. (2015) · arXiv · doi:10.48550/arxiv.1511.06361 · Cited by 296
Multi-Modal Knowledge Graph Construction and Application: A Survey
Zhu, X., Li, Z., Wang, X., Jiang, X., Sun, P., Wang, X., Xiao, Y., & Yuan, N. J. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3224228 · Cited by 353
Polysemous Visual-Semantic Embedding for Cross-Modal Retrieval
Song, Y., & Soleymani, M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00208 · Cited by 287
Lung and Pancreatic Tumor Characterization in the Deep Learning Era: Novel Supervised and Unsupervised Learning Approaches
Hussein, S., Kandel, P., Bolan, C. W., Wallace, M. B., & Bagci, U. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2894349 · Cited by 285
Cross-Modal Attention With Semantic Consistence for Image–Text Matching
Xu, X., Wang, T., Yang, Y., Zuo, L., Shen, F., & Shen, H. T. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2967597 · Cited by 282
Adversarial Representation Learning for Text-to-Image Matching
Sarafianos, N., Xu, X., & Kakadiaris, I. A. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00591 · Cited by 261
Learning to Answer Questions from Image Using Convolutional Neural Network
Ma, L., Lu, Z., & Li, H. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10442 · Cited by 260
Self-citation
Detecting Sarcasm in Multimodal Social Platforms
Schifanella, R., de Juan, P., Tetreault, J., & Cao, L. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2964321 · Cited by 266
A survey on automatic image caption generation
Bai, S., & An, S. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.05.080 · Cited by 253
Instance-Aware Image and Sentence Matching with Selective Multimodal LSTM
Huang, Y., Wang, W., & Wang, L. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.767 · Cited by 237
An analytical study of information extraction from unstructured and multidimensional big data
Adnan, K., & Akbar, R. (2019) · Journal Of Big Data · doi:10.1186/s40537-019-0254-8 · Cited by 252
Bidirectional Attentive Fusion with Context Gating for Dense Video Captioning
Wang, J., Jiang, W., Ma, L., Liu, W., & Xu, Y. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00751 · Cited by 237
Self-citation
Negative-Aware Attention Framework for Image-Text Matching
Zhang, K., Mao, Z., Wang, Q., & Zhang, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01521 · Cited by 216
Position Focused Attention Network for Image-Text Matching
Wang, Y., Yang, H., Qian, X., Ma, L., Lu, J. J., Li, B., & Fan, X. N. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/526 · Cited by 208
Self-citation
Predicting Visual Features From Text for Image and Video Caption Retrieval
Dong, J., Li, X., & Snoek, C. G. M. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2832602 · Cited by 198
Discriminability Objective for Training Descriptive Captions
Luo, R., Shakhnarovich, G., Cohen, S. D., & Price, B. L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00728 · Cited by 194
Consensus-Aware Visual-Semantic Embedding for Image-Text Matching
Wang, H., Zhang, Y., Ji, Z., Pang, Y., & Ma, L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58586-0_2 · Cited by 195
Self-citation
Controllable Video Captioning With POS Sequence Guidance Based on Gated Fusion Network
Wang, B., Ma, L., Zhang, W., Jiang, W., Wang, J., & Liu, W. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00273 · Cited by 199
Self-citation
Focus Your Attention
Liu, C., Mao, Z., Liu, A.-A., Zhang, T., Wang, B., & Zhang, Y. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3350869 · Cited by 206
Linking Image and Text with 2-Way Nets
Eisenschtat, A., & Wolf, L. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.201 · Cited by 171
RNN Fisher Vectors for Action Recognition and Image Annotation
Lev, G., Sadeh, G., Klein, B., & Wolf, L. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46466-4_50 · Cited by 172
Multi-modal Attention Network Learning for Semantic Source Code Retrieval
Wan, Y., Shu, J., Sui, Y., Xu, G., Zhao, Z., Wu, J., & Yu, P. S. (2019) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2019.00012 · Cited by 175
Phrase Localization and Visual Relationship Detection with Comprehensive Image-Language Cues
Plummer, B. A., Mallya, A., Cervantes, C. M., Hockenmaier, J., & Lazebnik, S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.213 · Cited by 168
Engaging Image Captioning via Personality
Shuster, K., Humeau, S., Hu, H., Bordes, A., & Weston, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01280 · Cited by 160
Visual Sentiment Analysis by Attending on Local Image Regions
You, Q., Jin, H., & Luo, J. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10501 · Cited by 162
Learning a Recurrent Residual Fusion Network for Multimodal Matching
Liu, Y., Guo, Y., Bakker, E. M., & Lew, M. S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.442 · Cited by 163
Unified Visual-Semantic Embeddings: Bridging Vision and Language With Structured Meaning Representations
Wu, H., Mao, J., Zhang, Y., Jiang, Y., Li, L., Sun, W., & Ma, W. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00677 · Cited by 162
Towards Context-Aware Interaction Recognition for Visual Relationship Detection
Zhuang, B., Liu, L., Shen, C., & Reid, I. D. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.71 · Cited by 161
Learning Like a Child: Fast Novel Visual Concept Learning from Sentence Descriptions of Images
Mao, J., Xu, W., Yang, Y., Wang, J., Huang, Z., & Yuille, A. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.291 · Cited by 149
Composite Correlation Quantization for Efficient Multimodal Retrieval
Long, M., Cao, Y., Wang, J., & Yu, P. S. (2016) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2911451.2911493 · Cited by 146
Comprehension-Guided Referring Expressions
Luo, R., & Shakhnarovich, G. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.333 · Cited by 152
Cross-Media Learning for Image Sentiment Analysis in the Wild
Vadicamo, L., Carrara, F., Cimino, A., Cresci, S., Dell’Orletta⋄, F., Falchi, F., & Tesconi, M. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.45 · Cited by 148
Learning structure of stereoscopic image for no-reference quality assessment with convolutional neural network
Zhang, W., Qu, C., Ma, L., Guan, J., & Huang, R. (2016) · Pattern Recognition · doi:10.1016/j.patcog.2016.01.034 · Cited by 141
Self-citation
Neural information retrieval: at the end of the early years
Onal, K. D., Zhang, Y., Altıngövde, İ. S., Rahman, M. M., Karagöz, P., Braylan, A., Dang, B., Chang, H.-L., Kim, H., McNamara, Q., et al. (2017) · Information Retrieval · doi:10.1007/s10791-017-9321-y · Cited by 145
Referring Expression Generation and Comprehension via Attributes
Liu, J., Wang, L., & Yang, M. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.520 · Cited by 134
Video Summarization via Semantic Attended Networks
Wei, H., Ni, B., Yan, Y., Yu, H., Yang, X., & Yao, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11297 · Cited by 137
Learning Fragment Self-Attention Embeddings for Image-Text Matching
Wu, Y., Wang, S., Song, G., & Huang, Q. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3350940 · Cited by 132
Cross-modal Graph Matching Network for Image-text Retrieval
Cheng, Y., Zhu, X., Qian, J., Wen, F., & Liu, P. (2022) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3499027 · Cited by 125
Saliency-Guided Attention Network for Image-Sentence Matching
Ji, Z., Wang, H., Han, J., & Pang, Y. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00585 · Cited by 119
Dual-Level Representation Enhancement on Characteristic and Context for Image-Text Retrieval
Yang, S. S., Li, Q., Li, W., Li, X., & Liu, A.-A. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3182426 · Cited by 121
Step-Wise Hierarchical Alignment Network for Image-Text Matching
Ji, Z., Chen, K., & Wang, H. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/106 · Cited by 120
Enhancing Video Summarization via Vision-Language Embedding
Plummer, B. A., Brown, M., & Lazebnik, S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.118 · Cited by 112
Deep Adversarial Graph Attention Convolution Network for Text-Based Person Search
Liu, J., Zha, Z.-J., Hong, R., Wang, M., & Zhang, Y. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3350991 · Cited by 112
LapsCore: Language-guided Person Search via Color Reasoning
Wu, Y., Yan, Z., Han, X., Li, G., Zou, C., & Cui, S. R. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00165 · Cited by 117
Finding Beans in Burgers: Deep Semantic-Visual Embedding with Localization
Engilberge, M., Chevallier, L., Pérez, P., & Cord, M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00419 · Cited by 108
Robust Visual-Textual Sentiment Analysis
You, Q., Cao, L., Jin, H., & Luo, J. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2964288 · Cited by 109
Image Understanding using vision and reasoning through Scene Description Graph
Aditya, S., Yang, Y., Baral, C. R., Aloimonos, Y., & Fermuller, C. M. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.12.004 · Cited by 103
Bi-Directional Spatial-Semantic Attention Networks for Image-Text Matching
Huang, F., Zhang, X., Zhao, Z., & Li, Z. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2882225 · Cited by 104
Topic-Oriented Image Captioning Based on Order-Embedding
Yu, N., Hu, X., Song, B., Yang, J., & Zhang, J. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2889922 · Cited by 101
Reasoning on the Relation: Enhancing Visual Representation for Visual Question Answering and Cross-Modal Retrieval
Yu, J., Zhang, W., Lu, Y., Qin, Z., Hu, Y., Tan, J., & Wu, Q. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.2972830 · Cited by 96
Weakly-Supervised Spatio-Temporally Grounding Natural Sentence in Video
Chen, Z., Ma, L., Luo, W., & Wong, K.-Y. K. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1183 · Cited by 116
Self-citation
Localizing Natural Language in Videos
Chen, J., Ma, L., Chen, X., Jie, Z., & Luo, J. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018175 · Cited by 94
Self-citation
Align2Ground: Weakly Supervised Phrase Grounding Guided by Image-Caption Alignment
Datta, S., Sikka, K., Roy, A., Ahuja, K., Parikh, D., & Divakaran, A. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00269 · Cited by 90
Adaptive Cross-Modal Embeddings for Image-Text Alignment
Wehrmann, J., Kolling, C., & Barros, R. C. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6915 · Cited by 88
Syntax-Guided Hierarchical Attention Network for Video Captioning
Deng, J., Li, L., Zhang, B., Wang, S., Zha, Z.-J., & Huang, Q. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3063423 · Cited by 87
Unified Adaptive Relevance Distinguishable Attention Network for Image-Text Matching
Zhang, K., Mao, Z., Liu, A.-A., & Zhang, Y. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3141603 · Cited by 89
Leveraging Visual Question Answering for Image-Caption Ranking
Lin, X., & Parikh, D. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46475-6_17 · Cited by 81
Reconstruct and Represent Video Contents for Captioning via Reinforcement Learning
Zhang, W., Wang, B., Ma, L., & Liu, W. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2920899 · Cited by 85
Self-citation
Multimedia Intelligence: When Multimedia Meets Artificial Intelligence
Zhu, W., Wang, X. Y., & Gao, W. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.2969791 · Cited by 87
Retrieval Topic Recurrent Memory Network for Remote Sensing Image Captioning
Wang, B., Zheng, X., Qu, B., & Lu, X. (2020) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2019.2959208 · Cited by 84
Geometry Attention Transformer with position-aware LSTMs for image captioning
Wang, C., Shen, Y., & Ji, L. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117174 · Cited by 74
Cross-Modal Image-Text Retrieval with Semantic Consistency
Chen, H., Ding, G., Lin, Z., Zhao, S., & Han, J. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3351055 · Cited by 76
Student Class Behavior Dataset: a video dataset for recognizing, detecting, and captioning students’ behaviors in classroom scenes
Sun, B., Wu, Y., Zhao, K., He, J. Y. D., Yu, L., Yan, H., & Luo, A. (2021) · Neural Computing and Applications · doi:10.1007/s00521-020-05587-y · Cited by 77
Deep Transfer Learning for Art Classification Problems
Sabatelli, M., Kestemont, M., Daelemans, W. M. P., & Geurts, P. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11012-3_48 · Cited by 70
Twitter100k: A Real-World Dataset for Weakly Supervised Cross-Media Retrieval
Hu, Y., Zheng, L., Yang, Y., & Huang, Y. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2760101 · Cited by 71
Visually Grounded Neural Syntax Acquisition
Shi, H., Mao, J., Gimpel, K., & Livescu, K. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1180 · Cited by 71
ACMM: Aligned Cross-Modal Memory for Few-Shot Image and Sentence Matching
Huang, Y., & Wang, L. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00587 · Cited by 71
Improving Image Captioning by Leveraging Knowledge Graphs
Zhou, Y., Sun, Y., & Honavar, V. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00036 · Cited by 65
SoDeep: A Sorting Deep Net to Learn Ranking Loss Surrogates
Engilberge, M., Chevallier, L., Pérez, P., & Cord, M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01105 · Cited by 64
Joint Global and Co-Attentive Representation Learning for Image-Sentence Retrieval
Wang, S., Chen, Y., Zhuo, J., Huang, Q., & Tian, Q. C. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240535 · Cited by 59
Integration of textual cues for fine-grained image captioning using deep CNN and LSTM
Gupta, N., & Jalal, A. S. (2019) · Neural Computing and Applications · doi:10.1007/s00521-019-04515-z · Cited by 62
Classification of femur fracture in pelvic X-ray images using meta-learned deep neural network
Lee, C., Jang, J., Lee, S., Kim, Y. S., Jo, H. J., & Kim, Y.-S. (2020) · Scientific Reports · doi:10.1038/s41598-020-70660-4 · Cited by 59
Computer Vision and Deep Learning to Manage Safety in Construction: Matching Images of Unsafe Behavior and Semantic Rules
Fang, W., Love, P. E. D., Ding, L., Xu, S., Kong, T., & Li, H. (2021) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2021.3093166 · Cited by 58
Self-citation
Deep Learning for Information Retrieval
Li, H., & Lu, Z. (2016) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2911451.2914800 · Cited by 58
Self-citation
A Bimodal Deep Learning Architecture for EEG-fNIRS Decoding of Overt and Imagined Speech
Cooney, C., Folli, R., & Coyle, D. (2021) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2021.3132861 · Cited by 60
GraDual: Graph-based Dual-modal Representation for Image-Text Matching
Long, S., Han, S. C., Wan, X., & Poon, J. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00252 · Cited by 57
Dual-Path Rare Content Enhancement Network for Image and Text Matching
Wang, Y., Su, Y., Li, W., Xiao, J., Li, X., & Liu, A.-A. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3254530 · Cited by 56
Determination of an infill well placement using a data-driven multi-modal convolutional neural network
Chu, M.-G., Min, B., Kwon, S., Park, G., Kim, S., & Huy, N. X. (2019) · Journal of Petroleum Science and Engineering · doi:10.1016/j.petrol.2019.106805 · Cited by 60

Authors: Ma, L., Lu, Z., Shang, L., & Li, H.
Affiliations: Huawei Technologies (China) (CN)
ORCID: Ma, Lin
Lu, Zhengdong
Li, Hang
Venue: 2015 IEEE International Conference on Computer Vision (ICCV)
Publisher: IEEE
Published: 2015-12
Pages: 2623-2631
DOI: 10.1109/iccv.2015.301
OpenAlex: W1916445035
Cited by: 334
References: 40
Citations by year: 2026: 5 · 2025: 9 · 2024: 15 · 2023: 23 · 2022: 29 · 2021: 31 · 2020: 59 · 2019: 67 · 2018: 41 · 2017: 34 · 2016: 18 · 2015: 3
FWCI: 13.12
Type: conference paper
Language: en
Open access: closed
Locations: IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
Keywords: multimodal convolutional neural network, image retrieval, sentence retrieval, cross-modal matching, word composition, Flickr8k, Flickr30k

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Multimodal Convolutional Neural Networks for Matching Image and Sentence

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks