The DOI matches its DataCite record; 46 of the 46 references it cites were found in a registry or an index.
VL-BERT: Pre-training of Generic Visual-Linguistic Representations
Open accessAbstract
We introduce a new pre-trainable generic representation for visual-linguistic tasks, called Visual-Linguistic BERT (VL-BERT for short). VL-BERT adopts the simple yet powerful Transformer model as the backbone, and extends it to take both visual and linguistic embedded features as input. In it, each element of the input is either of a word from the input sentence, or a region-of-interest (RoI) from the input image. It is designed to fit for most of the visual-linguistic downstream tasks. To better exploit the generic representation, we pre-train VL-BERT on the massive-scale Conceptual Captions dataset, together with text-only corpus. Extensive empirical analysis demonstrates that the pre-training procedure can better align the visual-linguistic clues and benefit the downstream tasks, such as visual commonsense reasoning, visual question answering and referring expression comprehension. It is worth noting that VL-BERT achieved the first place of single model on the leaderboard of the VCR benchmark. Code is released at \url{https://github.com/jackroos/VL-BERT}.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Vision Transformers are commonly evaluated using the sliding window procedure from MMSegmentation [59] (e.g., [10, 101, 103]).”
From MultiMAE: Multi-modal Multi-task Masked Autoencoders (2022) · cites this paper as [59]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Flagged first, then by citations
Research works printed but not linked: 4
- Ross Girshick. Fast r-cnn. In Proceedings of the IEEE international conference on computer vision , pp. 1440-1448, 2015.
- Ryan Kiros, Yukun Zhu, Ruslan R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Skip-thought vectors. In Advances in neural information processing systems , pp. 3294-3302, 2015.
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems , pp. 1097-1105, 2012.
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAI Blog , 1(8), 2019.
Cited by
Sorted by citations
Items 801–900 of 957
VMMP: Verifiable privacy-preserving multi-modal multi-task prediction
Bian, M., Ren, Y., He, G., Feng, G., & Zhang, X. (2024) · Information Sciences · doi:10.1016/j.ins.2024.120547 · Cited by 2
Anastasakis, Z., Mallis, D., Diomataris, M., Alexandridis, G., Kollias, S., & Pitsikalis, V. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00124 · Cited by 2
Improving Vision-and-Language Reasoning via Spatial Relations Modeling
Yang, C., Xu, R., Guo, Y., Huang, P., Chen, Y., Ding, W., Wang, Z., & Zhou, H. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00082 · Cited by 4
GDP: Generic Document Pretraining to Improve Document Understanding
Trivedi, A., Upadhyay, A., Mukhopadhyay, R., & Chaudhury, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70533-5_13 · Cited by 2
Multi-Channel Graph Fusion Representation for Tabular Data Imputation
Yu, N., Xu, K., Chen, K., Liu, S., Zheng, T., & Song, M. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10651425 · Cited by 2
InViG: Benchmarking Open-Ended Interactive Visual Grounding with 500K Dialogues
Zhang, H., Xu, J., Mo, Y., & Kong, T. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00560 · Cited by 2
A novel deep high-level concept-mining jointing hashing model for unsupervised cross-modal retrieval
Dong, C.-R., Zhang, J., Zhang, F., Hua, Q., & Xu, D. (2024) · High-Confidence Computing · doi:10.1016/j.hcc.2024.100274 · Cited by 2
3D scene generation for zero-shot learning using ChatGPT guided language prompts
Ahmadi, S., Cheraghian, A., Chowdhury, T. F., Saberi, M., & Rahman, S. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.104211 · Cited by 2
Kays, H. M. I., Momin, K. A., Muraleetharan, K. K., & Sadri, A. M. (2025) · Transportation Research Interdisciplinary Perspectives · doi:10.1016/j.trip.2025.101361 · Cited by 2
Understanding ME? Multimodal Evaluation for Fine-grained Visual Commonsense
Wang, Z., You, H., He, Y., Li, W., Chang, K.-W., & Chang, S. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.626 · Cited by 3
Hierarchical Transformer Encoder With Structured Representation for Abstract Reasoning
An, J., & Cho, S. (2020) · IEEE Access · doi:10.1109/access.2020.3035463 · Cited by 4
Reinforcement Stacked Learning with Semantic-Associated Attention for Visual Question Answering
Xiao, X., Zhang, C., Xiang, S., & Pan, C. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414636 · Cited by 2
Ding, H., Li, L. E., Hu, Z., Xu, Y., Hakkani-Tur, D. Z., Du, Z., & Zeng, B. (2021) · doi:10.18653/v1/2021.maiworkshop-1.11 · Cited by 2
Recognizing Multimodal Entailment
Magalhaes, C. I., Shirazi, A., Gopalan, A., Nagrani, A., Bratanič, B., Bregler, C., Funk, C., Ferreira, F., Barcik, G. F., Ilharco, G., et al. (2021) · doi:10.18653/v1/2021.acl-tutorials.6 · Cited by 3
Research on Image Caption Based on Multiple Word Embedding Representations
Lin, Z.-X., Qu, J.-X., Yang, X., Hou, Z., & Ding, C. (2021) · doi:10.1109/icnlp52887.2021.00050 · Cited by 2
Learning Hierarchical Reasoning for Text-Based Visual Question Answering
Li, C., Du, Q., Wang, Q., & Jin, Y. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86365-4_25 · Cited by 2
Nebbia, G., & Kovashka, A. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00510 · Cited by 2
PAC-MAN: Multi-Relation Network in Social Community for Personalized Hashtag Recommendation
Padungkiatwattana, U., & Maneeroj, S. (2022) · IEEE Access · doi:10.1109/access.2022.3229082 · Cited by 2
VReBERT: A Simple and Flexible Transformer for Visual Relationship Detection
Cui, Y. J., & Farazi, M. R. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956261 · Cited by 2
Boosting Generic Visual-Linguistic Representation With Dynamic Contexts
Ma, G., Bai, Y., Zhang, W., Yao, T., Shihada, B., & Mei, T. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3237164 · Cited by 2
TraVL: Transferring Pre-trained Visual-Linguistic Models for Cross-Lingual Image Captioning
Zhang, Z., Lu, P., Jiang, D., & Chen, G. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25198-6_26 · Cited by 2
A More Robust Model to Answer Noisy Questions in KBQA
Wang, Z., Xu, X., Li, X., Li, H., Zhu, L., & Wei, X. (2023) · IEEE Access · doi:10.1109/access.2023.3252608 · Cited by 2
Multi-Modal Food Classification in a Diet Tracking System with Spoken and Visual Inputs
Gowda, S., Hu, Y., & Korpusik, M. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095762 · Cited by 3
Attention-free based dual-encoder mechanism for Aspect-based Multimodal Sentiment Recognition
Gupta, P., Pandey, A., Kumar, A., & Vishwakarma, D. K. (2023) · International Conference in Advances in Power, Signal, and Information Technology (APSIT) · doi:10.1109/apsit58554.2023.10201711 · Cited by 2
Counterfactual Mix-Up for Visual Question Answering
Cho, J. W., Kim, D.-J., Jung, Y., & Kweon, I. S. (2023) · IEEE Access · doi:10.1109/access.2023.3303891 · Cited by 3
MetaReVision: Meta-Learning with Retrieval for Visually Grounded Compositional Concept Acquisition
Xu, G., Kordjamshidi, P., & Chai, J. Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.818 · Cited by 2
He, L., Liu, F., Liu, J., Duan, J., & Wang, H. (2024) · Applied Sciences · doi:10.3390/app14041375 · Cited by 2
Mapping Natural Language Intents to User Interfaces through Vision-Language Models
Abukadah, H., Fereidouni, M., & Siddique, A. B. (2024) · IEEE International Conference on Semantic Computing (ICSC) · doi:10.1109/icsc59802.2024.00045 · Cited by 3
ArtQuest: Countering Hidden Language Biases in ArtVQA
Bleidt, T., Eslami, S., & de Melo, G. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00716 · Cited by 4
Latent Alignment of Procedural Concepts in Multimodal Recipes
Faghihi, H. R., Mirzaee, R., Paliwal, S., & Kordjamshidi, P. (2020) · Workshop on Advances in Language and Vision Research (ALVR) · doi:10.18653/v1/2020.alvr-1.5 · Cited by 2
Samaran, J., García, N., Otani, M., Chu, C., & Nakashima, Y. (2021) · doi:10.18653/v1/2021.acl-srw.8 · Cited by 4
Fine-Grained Chemical Entity Typing with Multimodal Knowledge Representation
Sun, C., Li, W., Xiao, J., Parulian, N. N., Zhai, C., & Ji, H. (2021) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm52615.2021.9669360 · Cited by 5
MVLP: Multi-perspective Vision-Language Pre-training Model for Ethically Aligned Meme DetectionPreprint
Zhang, N., Feng, X., Gu, T., & Chang, L. (2023) · doi:10.36227/techrxiv.23689437 · Cited by 3
Bridging the Gap between Recognition-level Pre-training and Commonsensical Vision-language Tasks
Wan, Y. F., Ma, Y., You, H., Wang, Z., & Chang, S. (2022) · doi:10.18653/v1/2022.csrr-1.4 · Cited by 1
Tree-Structured Trajectory Encoding for Vision-and-Language Navigation
Zhou, X., & Mu, Y. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i3.25494 · Cited by 1
Wang, Z., Xu, X., Song, X., Li, H., Wei, X., & Huang, D. (2023) · International Journal of Intelligent Systems · doi:10.1155/2023/1202315 · Cited by 1
CLEVR-Implicit: A Diagnostic Dataset for Implicit Reasoning in Referring Expression Comprehension
Zhang, J., Wu, X., & Cai, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.791 · Cited by 2
An Accuracy Enhanced Vision Language Grounding Method Fused with Gaze Intention
Zhang, J., Tu, L., Zhang, Y., Xie, L., Xu, M., Ming, D., Yan, Y., & Yin, E. (2023) · Electronics · doi:10.3390/electronics12245007 · Cited by 1
Meng, L., Huang, Q., Yu, X., Guo, X., & Guo, T. (2023) · IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA) · doi:10.1109/ispa-bdcloud-socialcom-sustaincom59178.2023.00122 · Cited by 1
RehabGPT: a MaaS-based solution to large model building for digital rehabilitation
Wang, H., Zhong, Z., Dai, L., Wu, F. H., Ma, X., Xue, J., Pavel, S., Lam, P. L. K. M., Zhang, Q., & Lin, F. (2024) · International Workshop on Advanced Imaging Technology (IWAIT) · doi:10.1117/12.3019532 · Cited by 1
DecoupleCLIP: A Novel Cross-Modality Decouple Model for Painting Captioning
Zhang, M., Hou, X., Yan, Y., & Sun, M. (2024) · Electronics · doi:10.3390/electronics13214207 · Cited by 1
Attention head purification: A new perspective to harness CLIP for domain generalization
Wang, Y., & Kang, G. (2025) · Image and Vision Computing · doi:10.1016/j.imavis.2025.105511 · Cited by 3
Towards Identity-Aware Cross-Modal Retrieval: A Dataset and a Baseline
Messina, N., Vadicamo, L., Maltese, L., & Gennaro, C. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-88708-6_28 · Cited by 1
Multi-modal data augmentation based on masked modeling for image–text retrieval
Wang, M., Chao, G., Wang, X., Chen, Y., Xie, X., & Chu, D. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113821 · Cited by 1
MultiHGPT: Multi-task heterogeneous graph prompt tuning
Wang, Y., Cao, Y., Li, Z., Dong, B., Ye, G., & Chai, H. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104236 · Cited by 1
Self-citationProgressive Confident Masking Attention Network for Audio–Visual Segmentation
Wang, Y., Zhu, J., Dong, F. D., & Zhu, S. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113992 · Cited by 1
Backdoor threats in large language models—a survey
Liu, S., Pan, Y. F., Hong, K., Fei, R., Lin, C., Li, Q., & Shen, C. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4351-3 · Cited by 1
VQA Model Based on Image Descriptive Paragraph and Deep Integration of BERT
Wu, Z., Zhang, J., Zhang, H., & Chen, Y. (2020) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1624/2/022014 · Cited by 2
Which Apple Keeps Which Doctor Away? Colorful Word Representations With Visual Oracles
Zhang, Z., Yu, H., Zhao, H., & Utiyama, M. (2021) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2021.3130972 · Cited by 1
OSCAR and ActivityNet: an Image Captioning model can effectively learn a Video Captioning dataset
Byrd, E., & González-Mendoza, M. (2021) · LatinX in AI at Computer Vision and Pattern Recognition Conference (LXAI) · doi:10.52591/lxai202106257 · Cited by 1
Effectuating Communication for the Deaf and Hard-of-Hearing: An Ethnographic Review
Kamath, S. S., Martin, A., & Poojary, R. (2022) · International Conference on Electrical and Computing Technologies and Applications (ICECTA) · doi:10.1109/icecta57148.2022.9990196 · Cited by 1
Enhanced Video BERT for Fast Video Advertisement Retrieval
Yang, Y., Yu, T., Liu, J., Jin, Z., Jiao, X., Li, Y., Li, S., & Li, P. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020823 · Cited by 1
Visually-Enhanced Phrase Understanding
Hsu, T.-Y., Li, C.-A., Huang, C.-W., & Chen, Y.-N. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.363 · Cited by 1
An Anchor-based Relative Position Embedding Method for Cross-Modal Tasks
Wang, Y., Sun, X., Fengzong, L., Kang, Z., & Xu, C. X. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.362 · Cited by 1
Understanding Cross-modal Interactions in V&L Models that Generate Scene Descriptions
Cafagna, M., van Deemter, K., & Albert Gatt, A. (2022) · doi:10.18653/v1/2022.umios-1.6 · Cited by 1
CK-Transformer: Commonsense Knowledge Enhanced Transformers for Referring Expression Comprehension
Zhang, Z., Yannakoudakis, H., Zhen, X., & Shutova, E. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.196 · Cited by 3
Fu, Y., Gao, D., Liu, T., Zheng, H., Hao, D., & Pan, Z. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3312910 · Cited by 2
Dataset Bias Mitigation in Multiple-Choice Visual Question Answering and Beyond
Wang, Z., Chen, L., You, H., Xu, K., He, Y., Li, W., Codella, N., Chang, K.-W., & Chang, S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.576 · Cited by 4
Dönmez, E., Tilli, P., Yang, H.-Y., Vu, N. T., & Silberer, C. (2023) · Conference on Computational Natural Language Learning · doi:10.18653/v1/2023.conll-1.24 · Cited by 1
Tao, Y., Jiang, Y., Li, M., & Zhang, Y. (2023) · International Conference on Asian Language Processing · doi:10.1109/ialp61005.2023.10337089 · Cited by 2
Look Before You Drive: Boosting Trajectory Forecasting via Imagining Future
Fan, Y., Liu, X., Li, Y., & Wang, S. (2023) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros55552.2023.10341509 · Cited by 1
How to Use Language Expert to Assist Inference for Visual Commonsense Reasoning
Song, Z., Hu, W., Ye, H., & Hong, R. (2023) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw60847.2023.00074 · Cited by 1
Structure-Aware Multimodal Sequential Learning for Visual Dialog
Kim, Y., Kim, M.-J., An, K., Ahn, J., Kim, J., Heo, Y., Chang, D.-S., & Kim, E. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i12.29219 · Cited by 1
Fang, J. Z., Zheng, S., Sharma, V., & Piramuthu, R. (2024) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw60836.2024.00062 · Cited by 1
Implicit Modality Mining: An End-to-End Method for Multimodal Information Extraction
Lu, J., & Guo, Q. (2024) · Journal of Electronic Research and Application · doi:10.26689/jera.v8i2.6397 · Cited by 1
Modal-aware prompt tuning with deep adaptive feature enhancement
Wang, H., Shao, M., Tan, X., & Zhang, L. (2024) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2024.109270 · Cited by 1
Open-category referring expression comprehension via multi-modal knowledge transfer
Mi, W., Wang, J., Zhuang, F., An, Z., & Guo, W. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128063 · Cited by 1
OHiFormer: Object-Wise Hierarchical Dependency-Based Transformer for Screen Summarization
Han, Y. J., Lee, S., Kim, J. S., Lee, B., & Han, S. W. (2024) · IEEE Access · doi:10.1109/access.2024.3431711 · Cited by 1
Guo, W., Kong, X., & Huang, H. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129003 · Cited by 1
A System of Multimodal Image‐Text Retrieval Based on Pre‐Trained Models Fusion
Li, Q., Zhao, F., Zhao, L., Liu, M., Wang, Y., Zhang, S., Guo, Y., Wang, S., & Wang, W. (2024) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.8345 · Cited by 1
A multi-view projection-based object-aware graph network for dense captioning of point clouds
Ma, Z., Yang, Z. Q., Mao, A., Wen, S., Yi, R., & Liu, Y. (2024) · Computers & Graphics · doi:10.1016/j.cag.2024.104156 · Cited by 2
Shaon, M. S. H., Sultan, M. F., Karim, T., Alshan, M. S. H., Cuzzocrea, A., & Akter, M. S. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825409 · Cited by 1
Arshad, A., Moetesum, M., Hasan, A. U., & Faisal Shafait, B. (2024) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta63115.2024.00087 · Cited by 1
Liu, F., Hu, Z., Yang, P., & Liu, X. (2025) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2025.104515 · Cited by 1
A Region Descriptive Pre-training Approach with Self-attention Towards Visual Question Answering
Kolawole, B. B., & Lee, M. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-92310-5_9 · Cited by 1
Hsu, C.-J., Lee, H.-Y., & Tsao, Y. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-short.52 · Cited by 1
Most and Least Retrievable Images in Visual-Language Query Systems
Zhu, L., Ning, R., Jiang, L., Xin, C., & Wu, H. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_1 · Cited by 1
Probing Contextualized Sentence Representations with Visual AwarenessPreprint
Zhang, Z., Wang, R., Chen, K., Utiyama, M., Sumita, E., & Zhao, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.02971 · Cited by 3
Exceeding the Limits of Visual-Linguistic Multi-Task LearningPreprint
Wolfe, C. R., & Lundgaard, K. T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.13054 · Cited by 2
Human-centric Relation Segmentation: Dataset and SolutionPreprint
Liu, S., Wang, Z., Gao, Y., Ren, L., Liao, Y., Ren, G., Li, B., & Yan, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.11168 · Cited by 1
Self-citationNarrationBot and InfoBot: A Hybrid System for Automated Video DescriptionPreprint
Ihorn, S. M., Siu, Y.-T., Bodi, A., Narins, L. D., Castanon, J. M., Kant, Y., Das, A., Yoon, I., & Fazli, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.03994 · Cited by 8
Vision-and-Language Pretraining: Methods, Applications, and Future ChallengesPreprint
Nguyen, T., Nguyen, C.-D. T., Wu, X., Ng, S.-K., & Luu, A.-T. (2023) · Research Square · doi:10.21203/rs.3.rs-3120051/v1 · Cited by 1
Progressive Confident Masking Attention Network for Audio-Visual SegmentationPreprint
Wang, Y., Zhu, J., & Dong, F. D. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4656630 · Cited by 1
Kays, H. M. I., Momin, K. A., Muraleetharan, K. K., & Sadri, A. M. (2024) · Research Square · doi:10.21203/rs.3.rs-4536245/v1 · Cited by 1
Improving Calibration in Deep Metric Learning With Cross-Example SoftmaxPreprint
Veit, A., & Wilber, M. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.08824 · Cited by 3
Accurate Word Representations with Universal Visual GuidancePreprint
Zhang, Z., Yu, H., Zhao, H., Wang, R., & Utiyama, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.15086
T-EMDE: Sketching-based global similarity for cross-modal retrievalPreprint
Rychalska, B., Wieczorek, M., & Dabrowski, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.04242
Case Relation Transformer: A Crossmodal Language Generation Model for Fetching InstructionsPreprint
Kambara, M., & Sugiura, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.00789
Contextual Label Transformation For Scene Graph Generation
Lee, W., Kim, S., & Kim, G. (2021) · Proceedings - International Conference on Image Processing · doi:10.1109/icip42928.2021.9506213
RDMMFET: Representation of Dense Multimodality Fusion Encoder Based on Transformer
Zhang, X., Han, D., & Chang, C. (2021) · Mobile Information Systems · doi:10.1155/2021/2662064
Hong, H., Jin, X. H., Zhang⋆, Y., Hu, Y., Zhang, J., He, Y., & Xue, H. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475702
DSAMT: Dual-Source Aligned Multimodal Transformers for TextCaps
Liao, C., Liu, R., & Gao, S. (2021) · IEEE International Conference on Network Infrastructure and Digital Content (IC-NIDC) · doi:10.1109/ic-nidc54101.2021.9660575
An Augmented Embedding Spaces approach for Text-based Image Captioning
Bui, D. C., Trinh, T., Vo, N. D., & Nguyen, K. T. T. M. (2021) · NAFOSTED Conference on Information and Computer Science (NICS) · doi:10.1109/nics54270.2021.9701576
Key-Region and Layout Learning for Contract Intelligent Identification
Guo, Q., Hu, Z., Li, C., Liu, K., & Guo, X. (2021) · IEEE International Conference on Emergency Science and Information Technology (ICESIT) · doi:10.1109/icesit53460.2021.9696614
Critical Analysis of Deconfounded Pretraining to Improve Visio-Linguistic Models
Cornille, N., Laenen, K., & Moens, M.-F. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.736791 · Cited by 1
Multi-modal Universal Embedding Representations for Language Understanding
Luo, X., Cao, C., & Wang, L. (2022) · Communications in computer and information science · doi:10.1007/978-981-19-0523-0_7
Symbolizing Visual Features for Pre-training with Unlabeled Images
Kamata, Y., Yamada, M., Kato, K., Nakagawa, A., & Okatani, T. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-02444-3_37
Metadata
- Authors
- Su, W., Zhu, X., Cao, Y., Li, B., Lu, L., Wei, F., & Dai, J.
- Affiliations
- University of Science and Technology of China (CN)
- Microsoft (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1908.08530
- OpenAlex
- W2969876226
- Cited by
- 950
- References
- 45
- Citations by year
- 2026: 2 · 2025: 47 · 2024: 170 · 2023: 255 · 2022: 194 · 2021: 197 · 2020: 80 · 2019: 5
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- pre-training, multimodal transformer, region of interest features, visual commonsense reasoning, visual question answering, referring expression comprehension
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
