The DOI matches its DataCite record; 46 of the 46 references it cites were found in a registry or an index.
VL-BERT: Pre-training of Generic Visual-Linguistic Representations
Open accessAbstract
We introduce a new pre-trainable generic representation for visual-linguistic tasks, called Visual-Linguistic BERT (VL-BERT for short). VL-BERT adopts the simple yet powerful Transformer model as the backbone, and extends it to take both visual and linguistic embedded features as input. In it, each element of the input is either of a word from the input sentence, or a region-of-interest (RoI) from the input image. It is designed to fit for most of the visual-linguistic downstream tasks. To better exploit the generic representation, we pre-train VL-BERT on the massive-scale Conceptual Captions dataset, together with text-only corpus. Extensive empirical analysis demonstrates that the pre-training procedure can better align the visual-linguistic clues and benefit the downstream tasks, such as visual commonsense reasoning, visual question answering and referring expression comprehension. It is worth noting that VL-BERT achieved the first place of single model on the leaderboard of the VCR benchmark. Code is released at \url{https://github.com/jackroos/VL-BERT}.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Vision Transformers are commonly evaluated using the sliding window procedure from MMSegmentation [59] (e.g., [10, 101, 103]).”
From MultiMAE: Multi-modal Multi-task Masked Autoencoders (2022) · cites this paper as [59]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Flagged first, then by citations
Research works printed but not linked: 4
- Ross Girshick. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision , pp. 1440-1448, 2015.
- Ryan Kiros, Yukun Zhu, Ruslan R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Skip-thought vectors. In Advances in neural information processing systems , pp. 3294-3302, 2015.
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems , pp. 1097-1105, 2012.
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAI Blog , 1(8), 2019.
Cited by
Sorted by citations
Items 701–800 of 957
Semantic VL-BERT: Visual Grounding via Attribute Learning
Wanigasekara, P., Qin, K., Barut, E., Yang, F., Ruan, W., & Su, C. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892420 · Cited by 5
Weakly supervised cross-domain alignment with optimal transportPreprint
Yuan, S., Bai, K., Chen, L., Zhang, Y., Tao, C., Li, C., Wang, G., Henao, R., & Carin, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.06597 · Cited by 6
Knowledge-Grounded Dialogue Generation with Pre-trained Language ModelsPreprint
Zhao, X., Wu, W., Xu, C., Tao, C., Zhao, D.-Y., & Yan, R. (2020) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2010.08824 · Cited by 215
Grid-VLP: Revisiting Grid Features for Vision-Language Pre-trainingPreprint
Yan, M., Xu, H., Li, C., Bi, B., Tian, J., Gui, M., & Wang, W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.09479 · Cited by 8
A Comprehensive Evaluation of Multimodal Large Language Models in Hydrological ApplicationsPreprint
Kadiyala, L., Mermer, O., Samuel, R. D. J., Sermet, Y., & Demir, İ. (2024) · doi:10.31223/x5tq37 · Cited by 5
Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Survey
Du, D., Gong, G., & Chu, X. (2024) · arXiv · doi:10.48550/arxiv.2405.00314 · Cited by 4
“…them as the new backbone for a diverse range of tasks, including image classification [7], object detection [8], image generation [9], autonomous driving [10], and visual question answering [11], showcasing their versatility and transformative impact in computer vision, as reviewed in [12].”
BackgroundChen, F., Tang, X., & Hao, K. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.111047 · Cited by 4
Enhancing Intermodal Interaction for Unified Vision-Language Understanding and Generation
Yang, Q., Xie, H., Li, Y., Tan, B., & Ding, S. (2025) · Data Intelligence · doi:10.3724/2096-7004.di.2025.0034 · Cited by 5
Emotion-Aware Multimodal Pre-training for Image-Grounded Emotional Response Generation
Tian, Z., Wen, Z., Wu, Z., Song, Y., Tang, J., Li, D., & Zhang, N. L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-00129-1_1 · Cited by 4
An, J., Zainon, W. M. N. W., & Ding, B. (2023) · Intelligent Automation & Soft Computing · doi:10.32604/iasc.2023.039763 · Cited by 5
An Empirical Study on the Language Modal in Visual Question Answering
Peng, D., Wei, W., Mao, X.-L., Fu, Y., & Chen, D. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/457 · Cited by 6
Inexactly Matched Referring Expression Comprehension With Rationale
Li, X., Fan, B., Zhang, R., Zhao, K., Guo, Z., Zhao, Y., & Li, R. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3318289 · Cited by 4
Hafeth, D. A., Lal, G. R., Al-Khafajiy, M., Baker, T., & Kollias, S. (2023) · International Conference on Developments in eSystems Engineering (DeSE) · doi:10.1109/dese60595.2023.10468995 · Cited by 5
No Head Left Behind – Multi-Head Alignment Distillation for Transformers
Zhao, T., Singh, K. Y., Appalaraju, S., Tang, P., Mahadevan, V., Manmatha, R., & Wu, Y. N. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i7.28583 · Cited by 6
VD-GR: Boosting Visual Dialog with Cascaded Spatial-Temporal Multi-Modal GRaphs
Abdessaied, A., Shi, L., & Bulling, A. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00570 · Cited by 4
Moharil, A., Vanschoren, J., Singh, P., & Tamburri, D. A. (2024) · Machine Learning · doi:10.1007/s10994-024-06568-1 · Cited by 6
Abnormal Activity Detection Using Video Action Recognition: A Review
Ojha, A., Liu, Y., & Hao, Y. (2024) · doi:10.1109/icnlp60986.2024.10692366 · Cited by 5
Multi-Scale Feature Attention Fusion for Image Splicing Forgery Detection
Liang, E., Zhang, K., Hua, Z., & Jia, X. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3698770 · Cited by 4
Relation Inference Enhancement Network for Visual Commonsense Reasoning
Yuan, M., Jia, G., & Bao, B.-K. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3521725 · Cited by 4
Text-based image retrieval system using semantic visual content for re-ranking
Topçu, B., Mıtıncık, A., Erdem, M. G., & Yanikoglu, B. A. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111770 · Cited by 4
Multimodal or Text? Retrieval or BERT? Benchmarking Classifiers for the Shared Task on Hateful Memes
Kougia, V., & Pavlopoulos, J. (2021) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2021.woah-1.24 · Cited by 4
Visual Emotion Representation Learning via Emotion-Aware Pre-training
Zhang, Y., Ding, W., Xu, R., & Hu, X. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/234 · Cited by 5
Transformer-based Sparse Encoder and Answer Decoder for Visual Question Answering
Peng, L., An, G., & Ruan, Q. (2022) · International Conference on Signal Processing · doi:10.1109/icsp56322.2022.9965298 · Cited by 4
Li, L., Fu, Y., Zhu, D., Li, X., Sun, Y., Ding, J., Wu, M., Cao, N., & Higgs, R. J. (2023) · Intelligent Automation & Soft Computing · doi:10.32604/iasc.2023.035257 · Cited by 4
Wang, Z., Xu, X., Li, X., Li, H., Wei, X., & Huang, D. (2023) · Applied Sciences · doi:10.3390/app132011249 · Cited by 4
Text-Conditioned Outfit Recommendation With Hybrid Attention Layer
Wang, X., & Zhong, Y. (2023) · IEEE Access · doi:10.1109/access.2023.3346933 · Cited by 4
Cross-Domain Product Representation Learning for Rich-Content E-Commerce
Bai, X., Li, Y., Cheng, Y., Wenjie, Y., Chen, Q., & Li, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00524 · Cited by 4
Task-Oriented Multi-Modal Mutual Learning for Vision-Language Models
Long, S., Zhao, Z. W., Yuan, J., Tan, Z., Liu, J., Zhou, L., Wang, S., & Wang, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02007 · Cited by 7
Transformer based Multi-modal Memory-augmented Masked Network for Air Crisis Event Detection
Yang, Y., Yishan, Z., Qian, S., Zhang, M., & Cai, K. (2023) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc57777.2023.10422016 · Cited by 4
Wu, W., Li, Z., Cai, Y., Zhou, H., & Shou, M. Z. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3454331 · Cited by 4
Injecting Descriptive Meta-Information into Pre-Trained Language Models with Hypernetworks
Duan, W., He, X., Zhou, Z., Rao, H., & Thiele, L. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-229 · Cited by 4
A Framework of Cross-Modal Learning for Solving Geometry Problems
Guo, F., Jian, P., Wang, Y., & Wang, Q. (2021) · IEEE International Conference on Teaching, Assessment, and Learning for Engineering · doi:10.1109/tale52509.2021.9678945 · Cited by 4
Zhou, Z., Zhao, H., Dong, J., Gao, J., & Liu, X. (2022) · doi:10.18653/v1/2022.constraint-1.2 · Cited by 4
TeamX@DravidianLangTech-ACL2022: A Comparative Analysis for Troll-Based Meme Classification
Nandi, R. N., Alam, F., & Nakov, P. (2022) · Workshop on Speech, Vision, and Language Technologies for Dravidian Languages (DravidianLangTech) · doi:10.18653/v1/2022.dravidianlangtech-1.13 · Cited by 4
Adaptive Transformers for Learning Multimodal Representations
Bhargava, P. (2020) · Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (ACL SRW) · doi:10.18653/v1/2020.acl-srw.1 · Cited by 4
Effective transfer learning for low-resource natural language understanding
Liu, Z. (2022) · doi:10.14711/thesis-991013106358603412 · Cited by 6
A Survey of Task-Oriented Knowledge Graph Reasoning: Status, Applications, and ProspectsPreprint
Niu, G., Li, B., & Lin, Y. (2025) · doi:10.36227/techrxiv.174961563.32605293/v1 · Cited by 8
Self-citationEnhanced Cross-Modal Transformer Model for Video Semantic Similarity Measurement
Li, D., Zhu, B., Xu, K., Yang, S., Feng, D., Liu, B., & Wang, H. (2023) · IEEE Transactions on Circuits & Systems II Express Briefs · doi:10.1109/tcsii.2023.3302801 · Cited by 3
SKELTER: unsupervised skeleton action denoising and recognition using transformers
Paoletti, G., Beyan, C., & Bue, A. D. (2023) · Frontiers in Computer Science · doi:10.3389/fcomp.2023.1203901 · Cited by 4
Movie Box Office Prediction With Self-Supervised and Visually Grounded Pretraining
Qin, C., Kim, E., & Li, B. (2023) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme55011.2023.00265 · Cited by 3
Self-citationScene Retrieval in Traffic Videos with Contrastive Multimodal Learning
Sadiq, T., & Omlin, C. W. P. (2023) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai59109.2023.00153 · Cited by 3
Evidence-Aware Multimodal Chinese Social Media Rumor Detection
Wu, K., & Cao, D. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447484 · Cited by 3
Self-Supervised Relation Alignment for Scene Graph Generation
Xu, B., Liao, R., & Sigal, L. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00137 · Cited by 3
PILL: Plug into LLM with adapter expert and attention gate
Yin, Y., Zhang, F., Wu, Z., Qiu, Q., Liang, T., & Zhang, X. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.112115 · Cited by 6
Group-Based Distinctive Image Captioning with Memory Difference Encoding and Attention
Wang, J., Xu, W., Wang, Q., & Chan, A. B. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02220-6 · Cited by 3
Qin, Y., Ding, S., & Xie, H. (2025) · IEEE Access · doi:10.1109/access.2025.3541194 · Cited by 3
CIGLI: Conditional Image Generation from Language & Image
Lu, X., Ng, L. H. X., Fernandez, J., & Zhu, H. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00349 · Cited by 4
A Survey on Visual Question Answering
Banchhor, M., & Singh, P. (2021) · IEEE Global Conference for Advancement in Technology (GCAT) · doi:10.1109/gcat52182.2021.9587797 · Cited by 4
Chan, H. P., Guo, M., & Xu, C. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981515 · Cited by 3
Action Recognition in Australian Rules Football Through Deep Learning
Luan, S. K., Yin, H., & Sinnott, R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-08757-8_47 · Cited by 4
Cirik, V., Morency, L., & Berg-Kirkpatrick, T. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.373 · Cited by 3
Kim, Y., Chu, C., & Kurohashi, S. (2022) · Annual Meeting of the Association for Computational Linguistics: Student Research Workshop (ACL SRW) · doi:10.18653/v1/2022.acl-srw.22 · Cited by 4
Poirot at SemEval-2022 Task 5: Leveraging Graph Network for Misogynistic Meme Detection
Srivastava, H. (2022) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2022.semeval-1.110 · Cited by 3
Caption-Aided Product Detection via Collaborative Pseudo-Label Harmonization
Dong, X. M., Zhang, G., Zhan, X., Ding, Y., Wei, Y., Lu, M., & Liang, X. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3222653 · Cited by 3
Learning by Hallucinating: Vision-Language Pre-training with Weak Supervision
Wang, T.-J. J., Laaksonen, J. T., Langer, T., Arponen, H., & Bishop, T. E. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00113 · Cited by 3
Multimodal Transformer for Risk Classification: Analyzing the Impact of Different Data Modalities
Holtz, N., & Gómez, J. M. (2023) · doi:10.5121/csit.2023.130803 · Cited by 3
Zhang, Y., Wang, L., Yang, R., Chen, N., Zhao, Y., & Dai, Q. (2023) · International Conference on Graphics and Image Processing (ICGIP) · doi:10.1117/12.2680206 · Cited by 3
Run Like a Girl! Sport-Related Gender Bias in Language and Vision
Harrison, S., Gualdoni, E., & Boleda, G. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.886 · Cited by 12
Lexi: Self-Supervised Learning of the UI Language
Banerjee, P., Mahajan, S., Arora, K., Baral, C. R., & Riva, O. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.519 · Cited by 6
Multi-modal hierarchical fusion network for fine-grained paper classification
Yue, T., Li, Y., Qin, J., & Hu, Z. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16626-w · Cited by 3
Knowledge-Embedded Mutual Guidance for Visual Reasoning
Zheng, W., Yan, L., Chen, L., Li, Q., & Wang, F. (2023) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2023.3310892 · Cited by 3
PAT: Parallel Attention Transformer for Visual Question Answering in Vietnamese
Nguyen, N. H., & Nguyen, K. V. (2023) · International Conference on Multimedia Analysis and Pattern Recognition (MAPR) · doi:10.1109/mapr59823.2023.10288833 · Cited by 4
Meta-learning For Vision-and-language Cross-lingual Transfer
Hu, H., & Keller, F. (2023) · Workshop on Multilingual Representation Learning (MRL) · doi:10.18653/v1/2023.mrl-1.2 · Cited by 4
MMTF: Multi-Modal Temporal Fusion for Commonsense Video Question Answering
Ahmad, M., Park, G., Park, D., & Park, S. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00502 · Cited by 3
VL-Match: Enhancing Vision-Language Pretraining with Token-Level and Instance-Level Matching
Bi, J., Cheng, D., Yao, P., Pang, B., Zhan, Y., Yang, C., Wang, Y., Sun, H., Deng, W., & Zhang, Q. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00244 · Cited by 3
p-Laplacian Adaptation for Generative Pre-trained Vision-Language Models
Wu, H., Zhang, X., Xu, P. H., Liao, P., Yao, X., & Yu, B. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i6.28415 · Cited by 4
CoSTA: End-to-End Comprehensive Space-Time Entanglement for Spatio-Temporal Video Grounding
Liang, Y., Liang, X., Tang, Y., Yang, Z., Li, Z., Wang, J., Ding, W., & Huang, S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i4.28118 · Cited by 5
Yi, J., Uzkent, B., Ignat, O., Li, Z., Garg, A., Yu, X., & Liu, L. L. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00543 · Cited by 3
Zhu, J., Wang, H., & Shi, M. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3428487 · Cited by 3
Towards Unified Interactive Visual Grounding in The Wild
Xu, J., Zhang, H., Si, Q., Li, Y., Lan, X., & Kong, T. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611354 · Cited by 4
Leveraging Generated Image Captions for Visual Commonsense Reasoning
Das, S., & Sekhar, C. C. (2024) · Proceedings - International Conference on Image Processing · doi:10.1109/icip51287.2024.10648051 · Cited by 3
Chain of Thought Prompting in Vision-Language Model for Vision Reasoning Tasks
Ou, J., Zhou, J. J., Dong, Y., & Chen, F. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0351-0_22 · Cited by 3
Le, T., Huy, N. T., & Minh, N. L. (2020) · International Conference on Knowledge and Systems Engineering/Proceedings · doi:10.1109/kse50997.2020.9287539 · Cited by 4
Grounding Dialogue History: Strengths and Weaknesses of Pre-trained Transformers
Greco, C., Testoni, A., & Bernardi, R. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-77091-4_17 · Cited by 3
OVIS: Open-Vocabulary Visual Instance Search via Visual-Semantic Aligned Representation Learning
Liu, S., Lin, K., Wang, L., Yuan, J., & Liu, Z. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i2.20070 · Cited by 3
Beyond Language: Learning Commonsense from Images for Reasoning
Cui, W., Lan, Y., Pang, L., Guo, J., & Cheng, X. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.392 · Cited by 5
Li, D., Ming, Y., & He, Y. (2022) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2022.semeval-1.97 · Cited by 3
Visually-augmented pretrained language models for NLP tasks without images
Guo, H., Zhou, K., Zhao, W. X., Zhang, Q., & Wen, J.-R. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.833 · Cited by 6
Deng, Y. (2022) · doi:10.36227/techrxiv.20343321.v1 · Cited by 3
Cross-modal Graph Contrastive Learning with Cellular ImagesPreprint
Rao, J., Li, C., Zhang, J., Yang, Y., Zhou, L., Cohen, E., Lu, W., & Zheng, S. (2023) · Research Square · doi:10.21203/rs.3.rs-2928545/v1 · Cited by 3
DeVi Deep Learning Framework for Misogyny Identification in Multimodal DataPreprint
Singh, N. K., Das, P., Manderna, A., & Chand, S. (2023) · Research Square · doi:10.21203/rs.3.rs-3373925/v1 · Cited by 3
A Critical Analysis of Document Out-of-Distribution Detection
Gu, J., Ming, Y., Zhou, Y., Kuen, J., Morariu, V. I., Zhao, H., Zhang, R., Barmpalios, N., Liu, A., Li, Y., et al. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.332 · Cited by 3
EDUVQA – Visual Question Answering: An Educational Perspective
Koshti, D., Gupta, A., Kalla, M., Kanjilal, P., Shanbhag, S., & Karkera, N. (2024) · Journal of Advanced Research in Applied Sciences and Engineering Technology · doi:10.37934/araset.42.1.144157 · Cited by 2
Analyzing the Robustness of Vision & Language Models
Shirnin, A., Andreev, N., Potapova, S., & Artemova, E. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3399061 · Cited by 5
Ali, T., & Roy, P. P. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10651151 · Cited by 2
Learning Multi-Scale Knowledge-Guided Features for Text-Guided Face Recognition
Hasan, M. M., Sami, S. M., Nasrabadi, N. M., & Dawson, J. M. (2024) · IEEE Transactions on Biometrics Behavior and Identity Science · doi:10.1109/tbiom.2024.3466216 · Cited by 2
FashionFAE: Fine-grained Attributes Enhanced Fashion Vision-Language Pre-training
Huang, J., Gao, D., Zhang, J., Zhan, Z., Hu, Y., & Wang, X. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889957 · Cited by 2
Augmenting Vision Language Pretraining by Learning Codebook with Visual Semantics
Guo, X., Duan, J., Kuo, C. ‐. J., Gichoya, J. W., & Banerjee, I. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956616 · Cited by 2
A multi-strategy particle swarm optimization framework based on deep reinforcement learning
Hou, L., Fan, D., Cheng, J., Wu, H., Peng, H., & Deng, C. (2023) · International Conference on Advanced Computational Intelligence (ICACI) · doi:10.1109/icaci58115.2023.10146133 · Cited by 2
A survey of medical image captioning technique： encoding， decoding and latest advance
Zhu, Y., & Li, X. (2023) · Journal of Image and Graphics · doi:10.11834/jig.211021 · Cited by 2
Fusion or defusion? Flexible vision-and-language pre-training
Sun, R., Li, Z., Ding, Y., Wang, Q., Wang, J., Zheng, H., Wu, W., & Xian, Y. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.316 · Cited by 2
Choosing What to Mask: More Informed Masking for Multimodal Machine Translation
Sato, J., Caseli, H. D. M., & Specia, L. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-srw.35 · Cited by 2
Text-to-Chinese-painting Method Based on Multi-domain VQGAN
Sun, Z., Yang, G., Wen, J., Fei, N., Lu, Z., & Wen, J.-R. (2023) · International Journal of Software and Informatics · doi:10.21655/ijsi.1673-7288.00314 · Cited by 2
Chen, T., Liu, S., Chen, Z., Hu, W., Chen, D., Wang, Y., Lyu, Q., Le, C. X., & Wang, W. (2023) · Advances in Artificial Intelligence and Machine Learning · doi:10.54364/aaiml.2023.1181 · Cited by 2
Wang, J., Ji, Y., Zhang, Y., Zhu, Y., & Sakai, T. (2023) · IEEE Access · doi:10.1109/access.2023.3347192 · Cited by 2
In-Style: Bridging Text and Uncurated Videos with Style Transfer for Text-Video Retrieval
Shvetsova, N., Kukleva, A., Schiele, B., & Kuehne, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02009 · Cited by 2
Toward Multi-Granularity Decision-Making: Explicit Visual Reasoning with Hierarchical Knowledge
Zhang, Y., Chen, S., & Zhao, Q. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00243 · Cited by 2
Device-Unimodal Cloud-Multimodal Collaboration for Livestreaming Content Understanding
Zhu, Y., Niu, C., Yan, Y., Cao, Z., Jiang, H., Lyu, C., Tang, S., & Wu, F. (2023) · Proceedings · doi:10.1109/icdm58522.2023.00210 · Cited by 2
RCAT: Retentive CLIP Adapter Tuning for Improved Video Recognition
Xie, Z., Xu, M., Zhang, S., & Zhou, L. M. (2024) · Electronics · doi:10.3390/electronics13050965 · Cited by 2
A Novel Pretrained General-purpose Vision Language Model for the Vietnamese Language
Vũ, Đ. N., Pham, Q. N. M., & Tran, G. S. (2024) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3654796 · Cited by 2
Metadata
- Authors
- Su, W., Zhu, X., Cao, Y., Li, B., Lu, L., Wei, F., & Dai, J.
- Affiliations
- University of Science and Technology of China (CN)
- Microsoft (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1908.08530
- OpenAlex
- W2969876226
- Cited by
- 950
- References
- 45
- Citations by year
- 2026: 2 · 2025: 47 · 2024: 170 · 2023: 255 · 2022: 194 · 2021: 197 · 2020: 80 · 2019: 5
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- pre-training, multimodal transformer, region of interest features, visual commonsense reasoning, visual question answering, referring expression comprehension
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
