M3P: Learning Universal Representations via Multitask Multilingual Multimodal Pre-training
Abstract
We present M3P, a Multitask Multilingual Multimodal Pre-trained model that combines multilingual pre-training and multimodal pre-training into a unified framework via multitask pre-training. Our goal is to learn universal representations that can map objects occurred in different modalities or texts expressed in different languages into a common semantic space. In addition, to explicitly encourage fine-grained alignment between images and non-English languages, we also propose Multimodal Code-switched Training (MCT) to combine monolingual pre-training and multimodal pre-training via a code-switch strategy. Experiments are performed on the multilingual image retrieval task across two benchmark datasets, including MSCOCO and Multi30K. M3P can achieve comparable results for English and new state-of-the-art results for non-English languages.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Self-supervised learning does not rely on large amounts of manually labeled data and can be trained on unlabeled data.
“Self-supervised learning does not rely on large amounts of manually labeled data and can be trained using unlabeled data [184].”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [184]
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019) · arXiv · doi:10.48550/arxiv.1907.11692 · Cited by 16,515 · see citing works
Unsupervised Cross-lingual Representation Learning at Scale
Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzmán, F., Grave, E., Ott, M., Zettlemoyer, L., & Stoyanov, V. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.747 · Cited by 5,637 · see citing works
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107 · see citing works
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902 · see citing works
Kudo, T., & Richardson, J. (2018) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-2012 · Cited by 4,454 · see citing works
Young, P., Lai, A., Hodosh, M., & Hockenmaier, J. (2014) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00166 · Cited by 2,439 · see citing works
ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks
Lu, J., Batra, D., Parikh, D., & Lee, S. (2019) · arXiv · doi:10.48550/arxiv.1908.02265 · Cited by 2,109 · see citing works
Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning
Sharma, P., Ding, N., Goodman, S., & Soricut, R. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1238 · Cited by 1,879 · see citing works
Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Li, X., Yin, X. P., Li, C., Zhang, P., Hu, X., Zhang, L., Wang, L., Hu, H., Li, D., Wei, F., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58577-8_8 · Cited by 1,736 · see citing works
Self-citationMicrosoft COCO Captions: Data Collection and Evaluation Server
Chen, X., Fang, H., Lin, T.-Y., Vedantam, R., Gupta, S., Dollar, P., & Zitnick, C. L. (2015) · arXiv · doi:10.48550/arxiv.1504.00325 · Cited by 1,614 · see citing works
Cross-lingual Language Model Pretraining
Lample, G., & Conneau, A. (2019) · arXiv · doi:10.48550/arxiv.1901.07291 · Cited by 1,586 · see citing works
XNLI: Evaluating Cross-lingual Sentence Representations
Conneau, A., Rinott, R., Lample, G., Williams, A., Bowman, S. R., Schwenk, H., & Stoyanov, V. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1269 · Cited by 949 · see citing works
Im2Text: Describing Images Using 1 Million Captioned Photographs
Ordóñez, V., Kulkarni, G. A., & Berg, T. L. (2011) · Cited by 879 · see citing works
Unified Vision-Language Pre-Training for Image Captioning and VQA
Zhou, L., Palangi, H., Zhang, L., Hu, H., Corso, J., & Gao, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.7005 · Cited by 864 · see citing works
Unicoder-VL: A Universal Encoder for Vision and Language by Cross-Modal Pre-Training
Li, G., Duan, N., Fang, Y., Gong, M., & Jiang, D. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6795 · Cited by 795 · see citing works
Self-citationLearning Two-Branch Neural Networks for Image-Text Matching Tasks
Wang, L., Li, Y., Huang, J. L., & Lazebnik, S. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2797921 · Cited by 538 · see citing works
Self-citationLarge-Scale Adversarial Training for Vision-and-Language Representation LearningPreprint
Gan, Z., Chen, Y.-C., Li, L., Zhu, C., Cheng, Y., & Liu, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.06195 · Cited by 343 · see citing works
UNITER: Learning UNiversal Image-TExt Representations
Chen, Y.-C., Li, L., Yu, L., Kholy, A. E., Ahmed, F., Gan, Z., Cheng, Y., & Liu, J. (2019) · Cited by 280 · see citing works
XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation
Liang, Y., Duan, N., Gong, Y., Wu, N., Guo, F., Qi, W., Gong, M., Shou, L., Jiang, D., Cao, G., et al. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.484 · Cited by 270 · see citing works
Self-citationUnicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual Tasks
Huang, H., Liang, Y., Duan, N., Gong, M., Shou, L., Jiang, D., & Zhou, M. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1252 · Cited by 170 · see citing works
Self-citationCOCO-CN for Cross-Lingual Image Tagging, Captioning, and Retrieval
Li, X., Xu, C., Wang, X., Lan, W., Jia, Z., Yang, G., & Xu, J. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2896494 · Cited by 163 · see citing works
CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLP
Qin, L., Ni, M., Zhang, Y., & Che, W. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/533 · Cited by 140 · see citing works
Self-citationMulti30K: Multilingual English-German Image Descriptions
Elliott, D., Frank, S., Sima’an, K., & Specia, L. (2016) · Workshop on Vision and Language (VL) · doi:10.18653/v1/w16-3210 · Cited by 129 · see citing works
STAIR Captions: Constructing a Large-Scale Japanese Image Caption Dataset
Yoshikawa, Y., Shigeto, Y., & Takeuchi, A. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-2066 · Cited by 115 · see citing works
ERNIE-ViL: Knowledge Enhanced Vision-Language Representations Through Scene GraphPreprint
Yu, F. R., Tang, J., Yin, W., Sun, Y., Tian, H., Wu, H. R., & Wang, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.16934 · Cited by 106 · see citing works
Cross-Lingual Image Caption Generation
Miyazaki, T., & Shimizu, N. (2016) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p16-1168 · Cited by 95 · see citing works
Unicoder: A Universal Language Encoder by Pre-training with Multiple Cross-lingual TasksPreprint
Huang, H., Liang, Y., Duan, N., Gong, M., Shou, L., Jiang, D., & Zhou, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.00964 · Cited by 87 · see citing works
Self-citationCSP:Code-Switching Pre-training for Neural Machine Translation
Yang, Z. C., Hu, B., Han, A., Huang, S., & Ju, Q. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.208 · Cited by 59 · see citing works
Language-Agnostic Visual-Semantic Embeddings
Wehrmann, J., Lopes, M. A., Souza, D. M., & Barros, R. C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00590 · Cited by 54 · see citing works
MULE: Multimodal Universal Language Embedding
Kim, D., Saito, K., Saenko, K., Sclaroff, S., & Plummer, B. A. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6785 · Cited by 36 · see citing works
Learning to Scale Multilingual Representations for Vision-Language Tasks
Burns, A., Kim, D., Wijaya, D. T., Saenko, K., & Plummer, B. A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58548-8_12 · Cited by 34 · see citing works
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805
Flagged first, then by citations
7 of the 43 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072 · see citing works
Pre-Trained Language Models and Their Applications
Wang, H., Li, J., Wu, H. R., Hovy, E. H., & Sun, Y. (2022) · Engineering · doi:10.1016/j.eng.2022.04.024 · Cited by 404 · see citing works
Large-scale Multi-modal Pre-trained Models: A Comprehensive Survey
Wang, X., Chen, G., Qian, G., Gao, P., Wei, X.-Y., Wang, Y., Tian, Y., & Gao, W. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1410-8 · Cited by 245 · see citing works
Vision-Language Pre-Training: Basics, Recent Advances, and Future Trends
Gan, Z., Li, L., Li, C., Wang, L., Liu, Z., & Gao, J. (2022) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000105 · Cited by 155 · see citing works
Self-citationZeng, Z., Yao, Y., Liu, Z., & Sun, M. (2022) · Nature Communications · doi:10.1038/s41467-022-28494-3 · Cited by 124 · see citing works
Multimodal generative AI for medical image interpretation
Rao, V. M., Hla, M., Moor, M., Adithan, S., Kwak, S., Topol, E. J., & Rajpurkar, P. (2025) · Nature · doi:10.1038/s41586-025-08675-y · Cited by 128 · see citing works
A Survey of Multi-modal Knowledge Graphs: Technologies and Trends
Liang, W., Meo, P. D., Tang, Y. Z., & Zhu, J. (2024) · ACM Computing Surveys · doi:10.1145/3656579 · Cited by 99 · see citing works
X2-VLM: All-in-One Pre-Trained Model for Vision-Language Tasks
Zeng, Y., Zhang, X., Li, H., Wang, J., Zhang, J., & Zhou, W. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3339661 · Cited by 91 · see citing works
MVP: Multimodality-Guided Visual Pre-training
Wei, L., Xie, L., Zhou, W., Li, H., & Tian, Q. C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20056-4_20 · Cited by 69 · see citing works
Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications
Manzoor, M. A., Albarri, S., Xian, Z., Meng, Z., Nakov, P., & Liang, S. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3617833 · Cited by 76 · see citing works
Pre-training Methods in Information Retrieval
Fan, Y., Xie, X., Cai, Y., Chen, J., Ma, X., Li, X., Zhang, R., & Guo, J. (2022) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000100 · Cited by 64 · see citing works
Vision + X: A Survey on Multimodal Learning in the Light of Data
Zhu, Y., Wu, Y., Sebe, N., & Yan, Y. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3420239 · Cited by 46 · see citing works
Fine-grained depression analysis based on Chinese micro-blog reviews
Yang, T., Li, F., Ji, D., Liang, X., Xie, T., Tian, S., Li, B., & Liang, P. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102681 · Cited by 44 · see citing works
Multimodal fusion and vision–language models: A survey for robot vision
Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103652 · Cited by 41 · see citing works
“These datasets not only serve as training resources but also form the basis for the performance benchmarks discussed in Section 5.”
Toward the unification of generative and discriminative visual foundation model: a survey
Liu, X., Zhou, T., Wang, C., Wang, Y., Wang, Y., Cao, Q., Du, W., Yang, Y., He, J., Qiao, Y., et al. (2024) · The Visual Computer · doi:10.1007/s00371-024-03608-8 · Cited by 35 · see citing works
Multimodal LLM Enhanced Cross-lingual Cross-modal Retrieval
Wang, Y., Wang, L., Zhou, Q., Wang, Z., Li, H., Hua, G., & Tang, W. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680886 · Cited by 28 · see citing works
Self-citationCross-Lingual Cross-Modal Retrieval with Noise-Robust Learning
Wang, Y., Dong, J., Liang, T., Zhang, M., Cai, R., & Wang, X. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548003 · Cited by 27 · see citing works
Product-oriented Machine Translation with Cross-modal Cross-lingual Pre-training
Song, Y., Chen, S., Jin, Q., Wei, L., Xie, J., & Huang, F. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475303 · Cited by 22 · see citing works
Dual-View Curricular Optimal Transport for Cross-Lingual Cross-Modal Retrieval
Wang, Y., Wang, S., Luo, H., Dong, J., Wang, F., Han, M., Wang, X., & Wang, M. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3365248 · Cited by 21 · see citing works
Wang, X., Li, L., Li, Z., Wang, X., Zhu, X., Wang, C., Huang, J., & Xiao, Y. (2023) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3539597.3570481 · Cited by 18 · see citing works
Sun, X., Yao, F., & Ding, C. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3252359 · Cited by 19 · see citing works
Wang, J., Lu, G., Xu, H., Li, Z., Xu, C., & Fu, Y. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01044 · Cited by 19 · see citing works
Temporally Language Grounding With Multi-Modal Multi-Prompt Tuning
YaWen, Z., Han, N., Pan, K., & Jin, Q. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3310282 · Cited by 18 · see citing works
Token Embeddings Alignment for Cross-Modal Retrieval
Xie, C.-W., Wu, J., Zheng, Y. P., Pan, P., & Hua, X. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548107 · Cited by 17 · see citing works
Cross-Modality Pyramid Alignment for Visual Intention Understanding
Ye, M., Shi, Q., Su, K., & Du, B. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3261743 · Cited by 17 · see citing works
Enhanced CatBoost with Stacking Features for Social Media Prediction
Mao, S., Xi, W.-D., Yu, L., Lu, G., Xing, X., Zhou, X., & Wan, W. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612839 · Cited by 13 · see citing works
Multimodal large model pretraining, adaptation and efficiency optimization
Ji, L., Xiao, S., Feng, J., Gao, W., & Zhang, H. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.129138 · Cited by 14 · see citing works
MMSFT: Multilingual Multimodal Summarization by Fine-Tuning Transformers
Phani, S., Abdul, A., Prasad, M. K. S., & Sarma, H. K. D. (2024) · IEEE Access · doi:10.1109/access.2024.3454382 · Cited by 10 · see citing works
Zero-Shot Cross-Lingual Knowledge Transfer in VQA via Multimodal Distillation
Weng, Y., Dong, J., He, W., Chaomurilige, C., Liu, X., Liu, Z., & Gao, H. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3402270 · Cited by 9 · see citing works
Adapting Grounded Visual Question Answering Models to Low Resource Languages
Wang, Y., Pfeiffer, J., Carion, N., LeCun, Y., & Kamath, A. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00258 · Cited by 8 · see citing works
U-BERT for Fast and Scalable Text-Image Retrieval
Yu, T., Fei, H., & Li, P. (2022) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3539813.3545148 · Cited by 6 · see citing works
Cross-Lingual Adaptation for Vision-Language Model via Multimodal Semantic Distillation
Weng, Y., He, W., Dong, J., Chaomurilige, C., Liu, X., & Liu, Z. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3557678 · Cited by 5 · see citing works
A Survey of Full-Cycle Cross-Modal Retrieval: From a Representation Learning Perspective
Wang, S., Zhu, L., Shi, L., Mo, H., & Tan, S. (2023) · Applied Sciences · doi:10.3390/app13074571 · Cited by 5 · see citing works
Cross-Lingual Cross-Modal Retrieval With Noise-Robust Fine-Tuning
Cai, R., Dong, J., Liang, T., Liang, Y., Wang, Y., Yang, X., Wang, X., & Wang, M. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3400060 · Cited by 5 · see citing works
Improving the Consistency in Cross-Lingual Cross-Modal Retrieval with 1-to-K Contrastive Learning
Nie, Z., Zhang, R., Feng, Z., Huang, H., & Liu, X. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671787 · Cited by 5 · see citing works
Self-citationVisual Topic Semantic Enhanced Machine Translation for Multi-Modal Data Efficiency
Wang, C., Cai, S., Shi, B.-X., & Chong, Z. (2023) · Journal of Computer Science and Technology · doi:10.1007/s11390-023-1302-6 · Cited by 4 · see citing works
cViL: Cross-Lingual Training of Vision-Language Models using Knowledge Distillation
Gupta, K., Gautam, D., & Mamidi, R. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956259 · Cited by 2 · see citing works
Possibilities of Using the Architecture of Neural Networks "Transformer" in Transport Engineering
Tarasova, A. V., Gematudinov, R. А., Yahimovich, M. E., Selezneva, P. A., & Dzhabrailov, K. A. (2023) · Systems of Signals Generating and Processing in the Field of on Board Communications · doi:10.1109/ieeeconf56737.2023.10092182 · Cited by 2 · see citing works
MuMUR: Multilingual Multimodal Universal Retrieval
Madasu, A., Aflalo, E., Stan, G. B. M., Rosenman, S., Tseng, S.-Y., Bertasius, G., & Lal, V. (2023) · Information Retrieval · doi:10.1007/s10791-023-09422-5 · Cited by 2 · see citing works
A Novel Pretrained General-purpose Vision Language Model for the Vietnamese Language
Vũ, Đ. N., Pham, Q. N. M., & Tran, G. S. (2024) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3654796 · Cited by 2 · see citing works
SCAG: Semantic Co-occurring Attention Guided Alignment for Knowledge-based Visual Question Answering
Liu, Z., Yang, K., Weng, Y., He, Z., Liu, X., & Gao, H. (2025) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3734220 · Cited by 2 · see citing works
CSDNet: Contrastive Similarity Distillation Network for Multi-lingual Image-Text Retrieval
Lu, S., Guo, L., He, X., Zhu, X., Liu, J., & Liu, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-46311-2_32 · Cited by 2 · see citing works
Empowering Vision-Language Tuning with Machine Translation for Driving Scene Caption Generation
Zhou, L., Sasano, R., & Takeda, K. (2025) · IEICE Transactions on Information and Systems · doi:10.1587/transinf.2024edp7126 · Cited by 1 · see citing works
Adaptive Biofeedback for Digital Physiotherapy Using Sakoe-Chiba Constrained Pose Matching
Mueller, J. L., Weiss, A., & Eskofier, B. M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-00652-3_17 · Cited by 1 · see citing works
Strategies for improving the accuracy of English translation through multimodal fusion deep learning
Lan, S. H. E. N., Bai, Y., & Duan, J. (2026) · Proceedings of the Indian National Science Academy · doi:10.1007/s43538-025-00685-8 · Cited by 1 · see citing works
A Library-Oriented Large Language Model Approach to Cross-Lingual and Cross-Modal Document Retrieval
Wang, Y., Cai, X., Ma, H., Fu, Z., & Zhan, Y. (2025) · Electronics · doi:10.3390/electronics14153145 · Cited by 1 · see citing works
Adaptive Transfer Learning via Fine-grained Multi-task Pre-training
Ma, Y., Xu, H., He, J.-Z., Qian, K., & Li, T. (2021) · International Conference on Algorithms, Computing and Artificial Intelligence (ACAI) · doi:10.1145/3508546.3508604 · Cited by 1 · see citing works
Pre-training Methods for Neural Machine Translation
Wang, M., & Li, L. (2021) · doi:10.18653/v1/2021.acl-tutorials.4
Wang, Y., Guo, X., Zhang, Y., Ren, Y., Huang, W., Liu, Z., Feng, Y., Dai, X., Zhang, W., & Che, H. (2023) · International Conference on Advanced Computational Intelligence (ICACI) · doi:10.1109/icaci58115.2023.10146184
Learning Representations for Bipartite Graphs Using Multi-task Self-supervised Learning
Sethi, A., Gupta, S., Malhotra, A., & Asthana, S. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43418-1_2
MEGA: Multi-encoder GNN Architecture for Stronger Task Collaboration and Generalization
Khoshbakhtian, F., Oberoi, G., Aleman, D. M., & Asthana, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70368-3_12
Asymmetric Language-Aware Feature Learning for Low-Resource Cross-Lingual Image Caption
Lyu, M., Chen, J., & Zhang, M. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-6599-0_20
Hu, P., Cao, Z., Yin, C., Xu, M., He, J., & Zhou, L. (2025) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3743093.3771037
Extending Visual Dialog Beyond English: An Analysis of Monolingual and Multilingual Models
Adão, M. M., Guimarães, S. J. F., & Patrocínio, Z. K. G. (2025) · International Symposium on Multimedia · doi:10.1109/ism66958.2025.00052
Harrag, F., Ayat, N., Redjechta, A. A., & Rouabhi, A. L. (2026) · Open Information Science · doi:10.1515/opis-2025-0044
DOKU: Distribution-Optimized Knowledge Update for Lifelong Person Re-identification
Hu, X., Yang, R., Hui, Y., Shen, C., & Wang, S. (2026) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-026-01386-w
Adão, M. M., Guimarães, S. J. F., & Patrocínio, Z. K. G. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-23161-1_16
- Authors
- Ni, M., Huang, H., Su, L., Cui, E., Bharti, T., Wang, L., Zhang, D., & Duan, N.
- Affiliations
- Harbin Institute of Technology (CN)
- Microsoft (United States) (US)
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 3976-3985
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3177654849
- Cited by
- 57
- References
- 43
- Citations by year
- 2026: 4 · 2025: 10 · 2024: 12 · 2023: 17 · 2022: 10 · 2021: 4
- FWCI
- 3.52
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- universal representation, multilingual pre-training, multimodal pre-training, multi-task learning, code-switching, image-text alignment, non-English languages, cross-lingual representation learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
