The DOI matches its DataCite record; 46 of the 46 references it cites were found in a registry or an index.
VL-BERT: Pre-training of Generic Visual-Linguistic Representations
Open accessAbstract
We introduce a new pre-trainable generic representation for visual-linguistic tasks, called Visual-Linguistic BERT (VL-BERT for short). VL-BERT adopts the simple yet powerful Transformer model as the backbone, and extends it to take both visual and linguistic embedded features as input. In it, each element of the input is either of a word from the input sentence, or a region-of-interest (RoI) from the input image. It is designed to fit for most of the visual-linguistic downstream tasks. To better exploit the generic representation, we pre-train VL-BERT on the massive-scale Conceptual Captions dataset, together with text-only corpus. Extensive empirical analysis demonstrates that the pre-training procedure can better align the visual-linguistic clues and benefit the downstream tasks, such as visual commonsense reasoning, visual question answering and referring expression comprehension. It is worth noting that VL-BERT achieved the first place of single model on the leaderboard of the VCR benchmark. Code is released at \url{https://github.com/jackroos/VL-BERT}.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Vision Transformers are commonly evaluated using the sliding window procedure from MMSegmentation [59] (e.g., [10, 101, 103]).”
From MultiMAE: Multi-modal Multi-task Masked Autoencoders (2022) · cites this paper as [59]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Flagged first, then by citations
Research works printed but not linked: 4
- Ross Girshick. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision , pp. 1440-1448, 2015.
- Ryan Kiros, Yukun Zhu, Ruslan R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Skip-thought vectors. In Advances in neural information processing systems , pp. 3294-3302, 2015.
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems , pp. 1097-1105, 2012.
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAI Blog , 1(8), 2019.
Cited by
Sorted by citations
Items 601–700 of 957
Decoding Language Spatial Relations to 2D Spatial Arrangements
Radevski, G., Collell, G., Moens, M., & Tuytelaars, T. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.408 · Cited by 8
Improving Pre-trained Vision-and-Language Embeddings for Phrase Grounding
Dou, Z.-Y., & Peng, N. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.513 · Cited by 8
MLIM: Vision-and-Language Model Pre-training with Masked Language and Image ModelingPreprint
Arici, T., Seyfioğlu, M. S., Neiman, T., Xu, Y., Tran, S., Chilimbi, T. M., Zeng, B., & Tutar, I. (2022) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/tqy7r · Cited by 16
Towards temporal sentence grounding in videos
Zhang, H. (2022) · doi:10.32657/10356/163788 · Cited by 8
Spatial-Temporal Correlation Learning for Traffic Demand Prediction
Wu, Y., Zhao, Y., Zhang, X., & Wang, Y. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3443341 · Cited by 8
SCARE: A Novel Framework to Enhance Chinese Harmful Memes Detection
Gu, T., Feng, M., Feng, X., & Wang, X. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3481419 · Cited by 8
Physically-guided open vocabulary segmentation with weighted patched alignment loss
Liu, W., Lou, J., Wang, X., Zhou, W., Cheng, J. T., & Yang, X. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128788 · Cited by 7
Cross-coupled prompt learning for few-shot image recognition
Zhang, F., Wei, R., Xie, Y., Wang, Y., Tan, X., Ma, L., Tang, M., & Fan, L. (2024) · Displays · doi:10.1016/j.displa.2024.102862 · Cited by 7
Cross-modal evidential fusion network for social media classification
Chen, Y., & Wang, Z. (2025) · Computer Speech & Language · doi:10.1016/j.csl.2025.101784 · Cited by 8
Generalized domain prompt learning for accessible scientific vision-language models
Cao, Q., Chen, Y., Lu, L., Sun, H., Zeng, Z., Yang, X., & Zhang, D. (2025) · Nexus · doi:10.1016/j.ynexs.2025.100069 · Cited by 7
Self-citationHGTFM: Hierarchical Gating-Driven Transformer Fusion Model for Robust Multimodal Sentiment Analysis
Yang, C., Liang, Z., Yan, D., Hu, Z., & Wu, T. (2025) · IEEE Access · doi:10.1109/access.2025.3560641 · Cited by 7
Ashfaq, U., Wang, Q., Merabet, B., & Zhang, J. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.108602 · Cited by 9
Probing Spatial Clues: Canonical Spatial Templates for Object Relationship Understanding
Collell, G., Deruyttere, T., & Moens, M. (2021) · IEEE Access · doi:10.1109/access.2021.3113781 · Cited by 7
TRIPS: Efficient Vision-and-Language Pre-training with Text-Relevant Image Patch Selection
Jiang, C., Xu, H., Li, C., Yan, M., Ye, W., Zhang, S., Bi, B., & Huang, S. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.273 · Cited by 14
ClipCrop: Conditioned Cropping Driven by Vision-Language Model
Zhong, Z., Cheng, M., Wu, Z., Yuan, Y., Zheng, Y., Li, J., Han, H., Lin, S. C.-F., Sato, Y., & Sato, I. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00037 · Cited by 10
Multimodality Self-distillation for Fast Inference of Vision and Language Pretrained Models
Kong, J., Wang, J., Yu, L.-C., & Zhang, X. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3384060 · Cited by 8
Zhu, Y., Chen, D., Jia, T., & Deng, S. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128460 · Cited by 8
Mo, M., Lu, W., Xie, Q., Xiao, Z., Lv, X., Yang, H. T., & Zhang, Y. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130059 · Cited by 7
Answer-checking in Context: A Multi-modal Fully Attention Network for Visual Question Answering
Huang, H., Han, T., Han, W., Yap, D., & Chiang, C.-M. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9413078 · Cited by 7
Entity-level Cross-modal Learning Improves Multi-modal Machine Translation
Huang, X., Zhang, J., & Zong, C. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.92 · Cited by 7
Modeling Coreference Relations in Visual Dialog
Li, M., & Moens, M. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.290 · Cited by 7
Premise-based Multimodal Reasoning: Conditional Inference on Joint Textual and Visual Clues
Dong, Q., Qin, Z., Xia, H., Feng, T., Tong, S., Meng, H., Xu, L., Wei, Z., Zhan, W., Chang, B., et al. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.66 · Cited by 7
Visualizing and Explaining Language Models
Brașoveanu, A., & Andonie, R. (2022) · Studies in computational intelligence · doi:10.1007/978-3-030-93119-3_8 · Cited by 8
Two-Stage Multimodality Fusion for High-Performance Text-Based Visual Question Answering
Li, B., Wang, J., Zhao, M., & Zhou, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26316-3_39 · Cited by 7
Self-citationDing, H., Wang, S., Xie, Z., Li, M., & Ma, L. (2023) · Computers & Graphics · doi:10.1016/j.cag.2023.07.025 · Cited by 7
Multiway-Adapter: Adapting Multimodal Large Language Models for Scalable Image-Text Retrieval
Long, Z., Killick, G., McCreadie, R., & Aragón-Camarasa, G. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446792 · Cited by 7
Commands 4 Autonomous Vehicles (C4AV) Workshop Summary
Deruyttere, T., Vandenhende, S., Grujicic, D., Liu, Y., Gool, L. V., Blaschko, M. B., Tuytelaars, T., & Moens, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-66096-3_1 · Cited by 7
Co-Attentional Transformers for Story-Based Video Understanding
Bebensee, B., & Zhang, B. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9413868 · Cited by 7
Practical Cross-modal Manifold Alignment for Robotic Grounded Language Learning
Nguyen, A. T., Richards, L. E., Kebe, G. Y., Raff, E., Darvish, K., Ferraro, F., & Matuszek, C. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00177 · Cited by 7
Attanasio, G., Nozza, D., & Bianchi, F. (2022) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2022.semeval-1.90 · Cited by 7
All-in-One Image-Grounded Conversational AgentsPreprint
Ju, D., Shuster, K., Boureau, Y.-L., & Weston, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.12394 · Cited by 10
Vision and Structured-Language Pretraining for Cross-Modal Food RetrievalPreprint
Shukor, M., Thome, N., & Cord, M. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4511116 · Cited by 7
Quality and Relevance Metrics for Selection of Multimodal Pretraining Data
Rao, R., Rao, S., Nouri, E., Dey, D., Celikyilmaz, A., & Dolan, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00486 · Cited by 6
SHE-Net: Syntax-Hierarchy-Enhanced Text-Video Retrieval
Yu, X., Jiang, C., Dong, X., Gan, T., Yang, M., & Guo, Q. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3543840 · Cited by 6
Towards unified bijective image–text generation for text-to-image person re-identification
Wang, Q., Ma, X., Jiang, X., Ji, J., & Pan, H. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114014 · Cited by 8
Recent, Rapid Advancement in Visual Question Answering: a Review
Kodali, V., & Berleant, D. (2022) · IEEE International Conference on Electro Information Technology · doi:10.1109/eit53891.2022.9813988 · Cited by 7
Deep Vision-Language Model for Efficient Multi-modal Similarity Search in Fashion Retrieval
Moro, G., & Salvatori, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-17849-8_4 · Cited by 6
GD-COMET: A Geo-Diverse Commonsense Inference Model
Bhatia, M., & Shwartz, V. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.496 · Cited by 6
Learning to Supervise Knowledge Retrieval Over a Tree Structure for Visual Question Answering
Xu, N., Lu, Z., Tian, H., Kang, R., Cao, J., Zhang, Y., & Liu, A.-A. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3355638 · Cited by 6
Self-Aligning Multi-Modal Transformer for Oropharyngeal Swab Point Localization
Liu, T., & Sun, F. (2024) · Tsinghua Science & Technology · doi:10.26599/tst.2023.9010070 · Cited by 6
Learning Feature Semantic Matching for Spatio-Temporal Video Grounding
Zhang, T., Fang, H., Zhang, H., Gao, J., Lu, X., Nie, X., & Yin, Y. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3387696 · Cited by 7
Potential and prospects of segment anything model： a survey
Wang, M., Huang, Z., He, H., Lu, H., Shan, H., & Zhang, J. (2024) · Journal of Image and Graphics · doi:10.11834/jig.230792 · Cited by 6
MMInstruct: a high-quality multi-modal instruction tuning dataset with extensive diversity
Liu, Y., Cao, Y. Q., Gao, Z., Wang, W., Chen, Z., Wang, W., Tian, H., Lu, L., Zhu, X., Lu, T., et al. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4187-3 · Cited by 34
Self-citationOn the (In)Effectiveness of Images for Text Classification
Ma, C., Shen, A., Yoshikawa, H., Iwakura, T., Beck, D., & Baldwin, T. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.4 · Cited by 7
Self-Supervised Audio-and-Text Pre-training with Extremely Low-Resource Parallel Data
Kang, Y., Liu, T., Li, H., Hao, Y., & Ding, W. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21334 · Cited by 6
Guiding Visual Question Answering with Attention Priors
Le, T. M., Le, V., Gupta, S., Venkatesh, S., & Tran, T. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00436 · Cited by 6
Bilaterally Slimmable Transformer for Elastic and Efficient Visual Question Answering
Zhou, Y., Jin, Z., Yu, J., Xu, M., Wang, H., & Jianping Fan, B. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3254205 · Cited by 6
Find Someone Who: Visual Commonsense Understanding in Human-Centric Grounding
You, H., Sun, R., Wang, Z., Chang, K.-W., & Chang, S. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.399 · Cited by 6
Mobile User Interface Element Detection Via Adaptively Prompt Tuning
Gu, Z., Xu, Z., Chen, H., Lan, J., Meng, C., & Wang, W. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01073 · Cited by 6
Masked Autoencoding Does Not Help Natural Language Supervision at Scale
Weers, F., Shankar, V., Katharopoulos, A., Yang, Y., & Gunter, T. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02244 · Cited by 8
Semi-supervised multimodal coreference resolution in image narrations
Goel, A., Fernando, B., Keller, F., & Bilen, H. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.682 · Cited by 7
SINet: Improving relational features in two-stage referring expression comprehension
Guo, W., Zhang, Y., & Yuan, X. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.123794 · Cited by 6
Chop Chop BERT: Visual Question Answering by Chopping VisualBERT’s Heads
Gao, C., Zhu, Q., Wang, P., & Wu, Q. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/92 · Cited by 7
Diversity and Consistency: Exploring Visual Question-Answer Pair Generation
Yang, S., Zhou, Q. S., Feng, D., Liu, Y., Li, C., Cao, Y., & Li, D. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.91 · Cited by 6
Self-citationRevisiting Knowledge Distillation for Image Captioning
Dong, J., Hu, Z., & Zhou, Y. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-93046-2_52 · Cited by 6
Concept formation through multimodal integration using multimodal BERT and VQ-VAE
Miyazawa, K., & Nagai, T. (2022) · Advanced Robotics · doi:10.1080/01691864.2022.2141583 · Cited by 6
Tencent-MVSE: A Large-Scale Benchmark Dataset for Multi-Modal Video Similarity Evaluation
Zeng, Z., Luo, Y., Liu, Z., Rao, F., Li, D., Guo, W., & Wen, Z. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00314 · Cited by 7
Chae, J., & Kim, J. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892787 · Cited by 7
World-to-Words: Grounded Open Vocabulary Acquisition through Fast Mapping in Vision-Language Models
Ma, Z., Pan, J., & Chai, J. Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.31 · Cited by 7
TFRGAN: Leveraging Text Information for Blind Face Restoration with Extreme Degradation
Xie, C., Ning, Q., Dong, W., & Shi, G. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00252 · Cited by 6
Generating More Pertinent Captions by Leveraging Semantics and Style on Multi-Source Datasets
Cornia, M., Baraldi, L., Fiameni, G., & Cucchiara, R. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01949-w · Cited by 23
Visual Grounding Strategies for Text-Only Natural Language ProcessingPreprint
Sileo, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.13942 · Cited by 9
Unsupervised Vision-and-Language Pre-training Without Parallel Images and CaptionsPreprint
Li, L. H., You, H., Wang, Z., Zareian, A., Chang, S., & Chang, K.-W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.12831 · Cited by 6
RK-VQA: Rational knowledge-aware fusion-in-decoder for knowledge-based visual question answering
Chen, W., Huang, X. Q., Liu, Z., Liu, J., Yin, H., & Yo, L. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102969 · Cited by 5
Cross-Lingual Adaptation for Vision-Language Model via Multimodal Semantic Distillation
Weng, Y., He, W., Dong, J., Chaomurilige, C., Liu, X., & Liu, Z. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3557678 · Cited by 5
Simultaneously Training and Compressing Vision-and-Language Pre-Training Model
Qi, Q., Zhang, A., Liao, Y., Sun, W., Wang, Y., Li, X., & Liu, S. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3233258 · Cited by 6
Lost Your Style? Navigating with Semantic-Level Approach for Text-to-Outfit Retrieval
Jang, J., Hwang, E., & Park, S.-H. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00788 · Cited by 6
Guo, C., Li, X., Ma, J., Li, Y., Liu, Y., Qi, H., Zhang, L., & Jin, Y. (2024) · IEEE Transactions on Fuzzy Systems · doi:10.1109/tfuzz.2024.3425664 · Cited by 5
LERCause: Deep learning approaches for causal sentence identification from nuclear safety reports
Kim, J., Kim, J., Lee, A., Kim, J., & Diesner, J. (2024) · PLoS ONE · doi:10.1371/journal.pone.0308155 · Cited by 8
Aesthetic multi-attributes network for image captioning
Yang, H., Li, Y., Jin, X., Zhou, X., Ping, S., & Liu, Y. (2025) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2025.110103 · Cited by 6
Open-Source Large Language Models: A Comprehensive Survey
Sahana Lokesh, R. (2025) · International Conference on Machine Learning and Autonomous Systems (ICMLAS) · doi:10.1109/icmlas64557.2025.10967790 · Cited by 6
A Bi-modal Automated Essay Scoring System for Handwritten Essays
Gao, J., Yang, Q., Zhang, Y., Zhang, L., & Wang, S. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533505 · Cited by 5
Testoni, A., Greco, C., & Bernardi, R. (2022) · Frontiers in Big Data · doi:10.3389/fdata.2021.736709 · Cited by 5
Semi-supervised Grounding Alignment for Multi-modal Feature Learning
Chou, S.-H., Fan, Z., Little, J. J., & Sigal, L. (2022) · Canadian Conference on Computer and Robot Vision · doi:10.1109/crv55824.2022.00015 · Cited by 5
Multi-scale Cell-based Layout Representation for Document Understanding
Shi, Y., Kim, M., & Chae, Y. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00366 · Cited by 5
Xiao, X., Li, B., Wu, C., Tseng, S.-Y., Bhiwandiwalla, A., Rosenman, S., Lal, V., Che, W., & Duan, N. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.811 · Cited by 5
Self-citationGHAN: Graph-Based Hierarchical Aggregation Network for Text-Video Retrieval
Yu, Y., Hu, B., & Li, Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.374 · Cited by 5
Understanding Social Media Cross-Modality Discourse in Linguistic Space
Xu, C., Tan, H., Li, J., & Li, P. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.182 · Cited by 5
AssistSR: Task-oriented Video Segment Retrieval for Personal AI Assistant
Lei, W., Gao, D., Wang, Y., Mao, D., Liang, Z., Ran, L., & Shou, M. Z. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.24 · Cited by 5
Cross-Modal Representation Learning
Yao, Y., Liu, Z., Lin, Y., & Sun, M. (2023) · Springer eBooks · doi:10.1007/978-981-99-1600-9_7 · Cited by 5
Interpretable Visual Question Answering Via Reasoning Supervision
Parelli, M., Mallis, D., Diomataris, M., & Pitsikalis, V. (2023) · Proceedings - International Conference on Image Processing · doi:10.1109/icip49359.2023.10223156 · Cited by 7
Who are you referring to? Coreference resolution in image narrations
Goel, A., Fernando, B., Keller, F., & Bilen, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01399 · Cited by 8
Latent Attention Network With Position Perception for Visual Question Answering
Zhang, J., Liu, X., & Wang, Z. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3377636 · Cited by 9
Leveraging Chat-Based Large Vision Language Models for Multimodal Out-of-Context Detection
Shalabi, F., Felouat, H., Nguyen, H. H., & Echizen, I. (2024) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-031-57916-5_8 · Cited by 5
Understanding mobile GUI: From pixel-words to screen-sentences
Fu, J., Zhang, X., Wang, Y., Zeng, W., & Zheng, N. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128200 · Cited by 13
Decoupled contrastive learning for multilingual multimodal medical pre-trained model
Li, Q. Y., Qiu, C., Liu, H., Gu, J., & Luo, D. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129809 · Cited by 5
Cross-Lingual Visual Grounding
Dong, W., Otani, M., García, N., Nakashima, Y., & Chu, C. (2020) · IEEE Access · doi:10.1109/access.2020.3046719 · Cited by 6
VL-BERT+: Detecting Protected Groups in Hateful Multimodal Memes
Aggarwal, P., Liman, M. E., Gold, D., & Zesch, T. (2021) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2021.woah-1.22 · Cited by 5
Semi-supervised Visual Feature Integration for Language Models through Sentence Visualization
Zhang, L., Chen, Q., Siebert, J. I., & Tang, B. (2021) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3462244.3479965 · Cited by 6
ViNTER: Image Narrative Generation with Emotion-Arc-Aware Transformer
Uehara, K., Mori, Y., Mukuta, Y., & Harada, T. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3487553.3524649 · Cited by 6
A Picture May Be Worth a Hundred Words for Visual Question Answering
Hirota, Y., García, N., Otani, M., Chu, C., & Nakashima, Y. (2024) · Electronics · doi:10.3390/electronics13214290 · Cited by 8
YNU-HPCC at SemEval-2022 Task 5: Multi-Modal and Multi-label Emotion Classification Based on LXMERT
Han, C., Wang, J., & Zhang, X. (2022) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2022.semeval-1.104 · Cited by 5
Visual Relationship Detection with Multimodal Fusion and Reasoning
Xiao, S., & Fu, W. (2022) · Sensors · doi:10.3390/s22207918 · Cited by 5
Ques-to-Visual Guided Visual Question Answering
Wu, X., Lu, J., Li, Z., & Xiong, F. (2022) · Proceedings - International Conference on Image Processing · doi:10.1109/icip46576.2022.9897277 · Cited by 5
InterREC: An Interpretable Method for Referring Expression Comprehension
Wang, W., Pagnucco, M., Xu, C., & Song, Y. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3251111 · Cited by 5
Towards Improvement of Grounded Cross-lingual Natural Language Inference with VisioTextual Attention
De, A., Desarkar, M. S., & Ekbal, A. (2023) · Natural Language Processing Journal · doi:10.1016/j.nlp.2023.100023 · Cited by 6
Grafting Pre-trained Models for Multimodal Headline Generation
Qiao, L., Wu, C., Liu, Y., Peng, H., Yin, D., & Ren, B. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-industry.25 · Cited by 5
Structural and Functional Decomposition for Personality Image Captioning in a Communication Game
Nguyen, M. T., Phung, D., Hoai, M., & Nguyen, T. H. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.411 · Cited by 5
Improving reasoning with contrastive visual information for visual question answering
Yu, L., Tang, P., Wang, H., & Yu, J. (2021) · Electronics Letters · doi:10.1049/ell2.12255 · Cited by 5
Case Relation Transformer: A Crossmodal Language Generation Model for Fetching Instructions
Kambara, M., & Sugiura, K. (2021) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2021.3107026 · Cited by 5
Metadata
- Authors
- Su, W., Zhu, X., Cao, Y., Li, B., Lu, L., Wei, F., & Dai, J.
- Affiliations
- University of Science and Technology of China (CN)
- Microsoft (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1908.08530
- OpenAlex
- W2969876226
- Cited by
- 950
- References
- 45
- Citations by year
- 2026: 2 · 2025: 47 · 2024: 170 · 2023: 255 · 2022: 194 · 2021: 197 · 2020: 80 · 2019: 5
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- pre-training, multimodal transformer, region of interest features, visual commonsense reasoning, visual question answering, referring expression comprehension
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
