CoChat
Sign Up
Cited by 8Multimodal Machine Learning ApplicationsTopic ModelingAdvanced Neural Network Applications

Enhancing Large Vision Language Models with Self-Training on Image Comprehension

Open access
Deng, Yihe, Lu, Pan, Yin, Fan, Hu, Ziniu, Shen, Sheng, Gu, Quanquan, Zou, James, Chang, Kai-Wei +1 more·2024·Advances in Neural Information Processing Systems 37
DOI 10.52202/079017-4175
Open this paper in CoChat — free accountOpen PDF

Abstract

Large vision language models (LVLMs) integrate large language models (LLMs) with pre-trained vision encoders, thereby activating the perception capability of the model to understand image inputs for different queries and conduct subsequent reasoning. Improving this capability requires high-quality vision-language data, which is costly and labor-intensive to acquire. Self-training approaches have been effective in single-modal settings to alleviate the need for labeled data by leveraging model's own generation. However, effective self-training remains a challenge regarding the unique visual perception and reasoning capability of LVLMs. To address this, we introduce Self-Training on Image Comprehension (STIC), which emphasizes a self-training approach specifically for image comprehension. First, the model self-constructs a preference dataset for image descriptions using unlabeled images. Preferred responses are generated through a step-by-step prompt, while dis-preferred responses are generated from either corrupted images or misleading prompts. To further self-improve reasoning on the extracted visual information, we let the model reuse a small portion of existing instruction-tuning data and append its self-generated image descriptions to the prompts. We validate the effectiveness of STIC across seven different benchmarks, demonstrating substantial performance gains of 4.0% on average while using 70% less supervised fine-tuning data than the current method. Further studies investigate various components of STIC and highlight its potential to leverage vast quantities of unlabeled images for self-training. Code and data are made publicly available.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.52202/079017-4175

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Deng et al. (2024) demonstrates how vision-language models can improve iteratively by training on their own generated image descriptions and visual reasoning chains.

    “Deng et al. (2024) demonstrates how vision-language models can improve iteratively by training on their own generated image descriptions and visual reasoning chains.”

    From A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence (2025) · cites this paper as Deng et al., 2024

    Section 4 (p. 5) describes a two-stage method in which the model learns from its own image descriptions, and the Abstract (p. 1) reports performance gains. But the paper says the reasoning stage reuses existing instruction-tuning data with self-generated descriptions appended to the prompts; it does not say the model trains on its own generated visual reasoning chains or demonstrate repeated iterative improvement. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

  • The approach leverages the model's existing visual understanding to generate detailed image descriptions, which are subsequently used to fine-tune the model's visual perception in a bootstrapping manner.

    “The approach leverages the model's existing visual understanding to generate detailed image descriptions, which are subsequently used to fine-tune the model's visual perception in a bootstrapping manner.”

    From A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence (2025) · cites this paper as Deng et al., 2024

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Still being checked.

Research works printed but not linked: 75

  1. ADOLPHS, L., GAO, T., XU, J., SHUSTER, K., SUKHBAATAR, S. and WESTON, J. (2023). The cringe loss: Learning what language not to model. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) .
  2. AHMADIAN, A., CREMER, C., GALLÉ, M., FADAEE, M., KREUTZER, J., PIETQUIN, O., ÜSTÜN, A. and HOOKER, S. (2024). Back to basics: Revisiting reinforce style optimization for learning from human feedback in llms. arXiv preprint arXiv:2402.14740 . · arXiv
  3. ALAYRAC, J.-B., DONAHUE, J., LUC, P., MIECH, A., BARR, I., HASSON, Y., LENC, K., MENSCH, A., MILLICAN, K., REYNOLDS, M. ET AL. (2022). Flamingo: a visual language model for few-shot learning. In Advances in Neural Information Processing Systems .
  4. AZAR, M. G., GUO, Z. D., PIOT, B., MUNOS, R., ROWLAND, M., VALKO, M. and CALANDRIELLO, D. (2024). A general theoretical paradigm to understand learning from human preferences. In International Conference on Artificial Intelligence and Statistics . PMLR.
  5. BAI, J., BAI, S., YANG, S., WANG, S., TAN, S., WANG, P., LIN, J., ZHOU, C. and ZHOU, J. (2023). Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966 . · arXiv
  6. BAI, Y., JONES, A., NDOUSSE, K., ASKELL, A., CHEN, A., DASSARMA, N., DRAIN, D., FORT, S., GANGULI, D., HENIGHAN, T. ET AL. (2022). Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862 . · arXiv
  7. BAO, H., WANG, W., DONG, L., LIU, Q., MOHAMMED, O. K., AGGARWAL, K., SOM, S., PIAO, S. and WEI, F. (2022). Vlmo: Unified vision-language pre-training with mixture-of-modality-experts. In Advances in Neural Information Processing Systems .
  8. CASPER, S., DAVIES, X., SHI, C., GILBERT, T. K., SCHEURER, J., RANDO, J., FREEDMAN, R., KORBAK, T., LINDNER, D., FREIRE, P. ET AL. (2023). Open problems and fundamental limitations of reinforcement learning from human feedback. arXiv preprint arXiv:2307.15217 . · arXiv
  9. CHEN, S., NIU, G., GONG, C., LI, J., YANG, J. and SUGIYAMA, M. (2021). Large-margin contrastive learning with distance polarization regularizer. In International Conference on Machine Learning . PMLR.
  10. CHEN, X. and HE, K. (2021). Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition .
  11. CHEN, X., WANG, X., CHANGPINYO, S., PIERGIOVANNI, A., PADLEWSKI, P., SALZ, D., GOODMAN, S., GRYCNER, A., MUSTAFA, B., BEYER, L. ET AL. (2022). Pali: A jointly-scaled multilingual language-image model. arXiv preprint arXiv:2209.06794 . · arXiv
  12. CHEN, Z., DENG, Y., LI, Y. and GU, Q. (2023a). Understanding transferable representation learning and zero-shot transfer in clip. arXiv preprint arXiv:2310.00927 . · arXiv
  13. CHEN, Z., DENG, Y., YUAN, H., JI, K. and GU, Q. (2024). Self-play fine-tuning converts weak language models to strong language models. arXiv preprint arXiv:2401.01335 . · arXiv
  14. CHEN, Z., WU, J., WANG, W., SU, W., CHEN, G., XING, S., MUYAN, Z., ZHANG, Q., ZHU, X., LU, L. ET AL. (2023b). Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. arXiv preprint arXiv:2312.14238 . · arXiv
  15. DAI, W., LI, J., LI, D., TIONG, A. M. H., ZHAO, J., WANG, W., LI, B., FUNG, P. N. and HOI, S. (2024). Instructblip: Towards general-purpose vision-language models with instruction tuning. Advances in Neural Information Processing Systems 36 .
  16. DENG, Y., ZHANG, W., CHEN, Z. and GU, Q. (2023). Rephrase and respond: Let large language models ask better questions for themselves. arXiv preprint arXiv:2311.04205 . · arXiv
  17. ETHAYARAJH, K., XU, W., MUENNIGHOFF, N., JURAFSKY, D. and KIELA, D. (2024). Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306 . · arXiv
  18. FRÄNKEN, J.-P., ZELIKMAN, E., RAFAILOV, R., GANDHI, K., GERSTENBERG, T. and GOODMAN, N. D. (2024). Self-supervised alignment with mutual information: Learning to follow principles without preference labels. arXiv preprint arXiv:2404.14313 . · arXiv
  19. GAO, L., SCHULMAN, J. and HILTON, J. (2023a). Scaling laws for reward model overoptimization. In International Conference on Machine Learning . PMLR.
  20. GAO, P., HAN, J., ZHANG, R., LIN, Z., GENG, S., ZHOU, A., ZHANG, W., LU, P., HE, C., YUE, X., LI, H. and QIAO, Y. (2023b). Llama-adapter v2: Parameter-efficient visual instruction model. arXiv preprint arXiv:2304.15010 . · arXiv
  21. GAO, P., ZHANG, R., LIU, C., QIU, L., HUANG, S., LIN, W., ZHAO, S., GENG, S., LIN, Z., JIN, P., ZHANG, K., SHAO, W., XU, C., HE, C., HE, J., SHAO, H., LU, P., LI, H. and QIAO, Y. (2024). Sphinx-x: Scaling data and parameters for a family of multi-modal large language models. In International Conference on Machine Learning (ICML) .
  22. GHIASI, G., ZOPH, B., CUBUK, E. D., LE, Q. V. and LIN, T.-Y. (2021). Multi-task self-training for learning general representations. In Proceedings of the IEEE/CVF International Conference on Computer Vision .
  23. GOEL, S., BANSAL, H., BHATIA, S., ROSSI, R., VINAY, V. and GROVER, A. (2022). Cyclip: Cyclic contrastive language-image pretraining. Advances in Neural Information Processing Systems 35 6704-6719.
  24. HE, J., GU, J., SHEN, J. and RANZATO, M. (2019). Revisiting self-training for neural sequence generation. arXiv preprint arXiv:1909.13788 . · arXiv
  25. HU, E. J., WALLIS, P., ALLEN-ZHU, Z., LI, Y., WANG, S., WANG, L., CHEN, W. ET AL. (2021). Lora: Low-rank adaptation of large language models. In International Conference on Learning Representations .
  26. JIA, C., YANG, Y., XIA, Y., CHEN, Y., PAREKH, Z., PHAM, H., LE, Q. V., SUNG, Y., LI, Z. and DUERIG, T. (2021a). Scaling up visual and vision-language representation learning with noisy text supervision. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event (M. Meila and T. Zhang, eds.), vol. 139 of Proceedings of Machine Learning Research . PMLR.
  27. JIA, C., YANG, Y., XIA, Y., CHEN, Y.-T., PAREKH, Z., PHAM, H., LE, Q., SUNG, Y.-H., LI, Z. and DUERIG, T. (2021b). Scaling up visual and vision-language representation learning with noisy text supervision. In International Conference on Machine Learning . PMLR.
  28. JIANG, A. Q., SABLAYROLLES, A., MENSCH, A., BAMFORD, C., CHAPLOT, D. S., CASAS, D. D. L., BRESSAND, F., LENGYEL, G., LAMPLE, G., SAULNIER, L. ET AL. (2023). Mistral 7b. arXiv preprint arXiv:2310.06825 . · arXiv
  29. JOSIFOSKI, M., SAKOTA, M., PEYRARD, M. and WEST, R. (2023). Exploiting asymmetry for synthetic training data generation: Synthie and the case of information extraction. arXiv preprint arXiv:2303.04132 . · arXiv
  30. KANG, G.-C., KIM, S., KIM, J.-H., KWAK, D. and ZHANG, B.-T. (2023). The dialog must go on: Improving visual dialog via generative self-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition .
  31. KIM, D., PARK, C., KIM, S., LEE, W., SONG, W., KIM, Y., KIM, H., KIM, Y., LEE, H., KIM, J. ET AL. (2023). Solar 10.7 b: Scaling large language models with simple yet effective depth up-scaling. arXiv preprint arXiv:2312.15166 . · arXiv
  32. KIM, W., SON, B. and KIM, I. (2021). ViLT: Vision-and-language transformer without convolution or region supervision. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event (M. Meila and T. Zhang, eds.), vol. 139 of Proceedings of Machine Learning Research . PMLR.
  33. LI, J., LI, D., SAVARESE, S. and HOI, S. (2023a). Blip-2: Bootstrapping language-image pretraining with frozen image encoders and large language models. In International conference on machine learning . PMLR.
  34. LI, J., LI, D., SAVARESE, S. and HOI, S. (2023b). Blip-2: Bootstrapping language-image pretraining with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597 . · arXiv
  35. LI, L. H., YATSKAR, M., YIN, D., HSIEH, C. and CHANG, K. (2019). Visualbert: A simple and performant baseline for vision and language. CoRR abs/1908.03557 . · arXiv
  36. LI, L. H., ZHANG, P., ZHANG, H., YANG, J., LI, C., ZHONG, Y., WANG, L., YUAN, L., ZHANG, L., HWANG, J.-N. ET AL. (2022). Grounded language-image pre-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition .
  37. LI, X., YIN, X., LI, C., ZHANG, P., HU, X., ZHANG, L., WANG, L., HU, H., DONG, L., WEI, F., CHOI, Y. and GAO, J. (2020). Oscar: Object-semantics aligned pre-training for vision-language tasks. In Computer Vision - ECCV 2020 - 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXX (A. Vedaldi, H. Bischof, T. Brox and J. Frahm, eds.), vol. 12375 of Lecture Notes in Computer Science . Springer.
  38. LIN, T.-Y., MAIRE, M., BELONGIE, S., HAYS, J., PERONA, P., RAMANAN, D., DOLLÁR, P. and ZITNICK, C. L. (2014). Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13 . Springer.
  39. LIU, H., LI, C., LI, Y. and LEE, Y. J. (2023a). Improved baselines with visual instruction tuning. arXiv preprint arXiv:2310.03744 . · arXiv
  40. LIU, H., LI, C., WU, Q. and LEE, Y. J. (2023b). Visual instruction tuning. Advances in Neural Information Processing Systems (NeurIPS) 36 .
  41. LIU, Y., DUAN, H., ZHANG, Y., LI, B., ZHANG, S., ZHAO, W., YUAN, Y., WANG, J., HE, C., LIU, Z. ET AL. (2023c). Mmbench: Is your multi-modal model an all-around player? arXiv preprint arXiv:2307.06281 . · arXiv
  42. LU, P., BANSAL, H., XIA, T., LIU, J., LI, C., HAJISHIRZI, H., CHENG, H., CHANG, K.-W., GALLEY, M. and GAO, J. (2024). Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Representations (ICLR) .
  43. LU, P., MISHRA, S., XIA, T., QIU, L., CHANG, K.-W., ZHU, S.-C., TAFJORD, O., CLARK, P. and KALYAN, A. (2022). Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems 35 2507-2521.
  44. MASRY, A., DO, X. L., TAN, J. Q., JOTY, S. and HOQUE, E. (2022). Chartqa: A benchmark for question answering about charts with visual and logical reasoning. In Findings of the Association for Computational Linguistics: ACL 2022 .
  45. MCKINZIE, B., GAN, Z., FAUCONNIER, J.-P., DODGE, S., ZHANG, B., DUFTER, P., SHAH, D., DU, X., PENG, F., WEERS, F. ET AL. (2024). Mm1: Methods, analysis & insights from multimodal llm pre-training. arXiv preprint arXiv:2403.09611 . · arXiv
  46. OUYANG, L., WU, J., JIANG, X., ALMEIDA, D., WAINWRIGHT, C., MISHKIN, P., ZHANG, C., AGARWAL, S., SLAMA, K., RAY, A. ET AL. (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35 2773027744.
  47. PANG, R. Y., YUAN, W., CHO, K., HE, H., SUKHBAATAR, S. and WESTON, J. (2024). Iterative reasoning preference optimization. arXiv preprint arXiv:2404.19733 . · arXiv
  48. PI, R., HAN, T., XIONG, W., ZHANG, J., LIU, R., PAN, R. and ZHANG, T. (2024). Strengthening multimodal large language model with bootstrapped preference optimization. arXiv preprint arXiv:2403.08730 . · arXiv
  49. PRASAD, A., STENGEL-ESKIN, E. and BANSAL, M. (2023). Rephrase, augment, reason: Visual grounding of questions for vision-language models. arXiv preprint arXiv:2310.05861 . · arXiv
  50. RADFORD, A., KIM, J. W., HALLACY, C., RAMESH, A., GOH, G., AGARWAL, S., SASTRY, G., ASKELL, A., MISHKIN, P., CLARK, J. ET AL. (2021). Learning transferable visual models from natural language supervision. In International Conference on Machine Learning (ICML) . PMLR.
  51. RAFAILOV, R., SHARMA, A., MITCHELL, E., ERMON, S., MANNING, C. D. and FINN, C. (2023). Direct preference optimization: Your language model is secretly a reward model. arXiv preprint arXiv:2305.18290 . · arXiv
  52. ROSSET, C., CHENG, C.-A., MITRA, A., SANTACROCE, M., AWADALLAH, A. and XIE, T. (2024). Direct nash optimization: Teaching language models to self-improve with general preferences. arXiv preprint arXiv:2404.03715 . · arXiv
  53. SCHULMAN, J., WOLSKI, F., DHARIWAL, P., RADFORD, A. and KLIMOV, O. (2017). Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 . · arXiv
  54. SHEN, S., LI, L. H., TAN, H., BANSAL, M., ROHRBACH, A., CHANG, K.-W., YAO, Z. and KEUTZER, K. (2022). How much can clip benefit vision-and-language tasks? In ICLR .
  55. SINGH, A., HU, R., GOSWAMI, V., COUAIRON, G., GALUBA, W., ROHRBACH, M. and KIELA, D. (2021). FLAVA: A foundational language and vision alignment model. CoRR abs/2112.04482 . · arXiv
  56. SINGH, A., NATARJAN, V., SHAH, M., JIANG, Y., CHEN, X., BATRA, D., PARIKH, D. and ROHRBACH, M. (2019). Towards vqa models that can read. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition .
  57. SOHN, K., BERTHELOT, D., CARLINI, N., ZHANG, Z., ZHANG, H., RAFFEL, C. A., CUBUK, E. D., KURAKIN, A. and LI, C.-L. (2020). Fixmatch: Simplifying semi-supervised learning with consistency and confidence. Advances in neural information processing systems 33 596-608.
  58. SUN, Z., SHEN, S., CAO, S., LIU, H., LI, C., SHEN, Y., GAN, C., GUI, L.-Y., WANG, Y.-X., YANG, Y. ET AL. (2023). Aligning large multimodal models with factually augmented rlhf. arXiv preprint arXiv:2309.14525 . · arXiv
  59. TAN, H. and BANSAL, M. (2019). LXMERT: Learning cross-modality encoder representations from transformers. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019 (K. Inui, J. Jiang, V. Ng and X. Wan, eds.). Association for Computational Linguistics.
  60. TEAM, G., ANIL, R., BORGEAUD, S., WU, Y., ALAYRAC, J.-B., YU, J., SORICUT, R., SCHALKWYK, J., DAI, A. M., HAUTH, A. ET AL. (2023). Gemini: A family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 . · arXiv
  61. TOUVRON, H., MARTIN, L., STONE, K., ALBERT, P., ALMAHAIRI, A., BABAEI, Y., BASHLYKOV, N., BATRA, S., BHARGAVA, P., BHOSALE, S. ET AL. (2023a). Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 . · arXiv
  62. TOUVRON, H., MARTIN, L., STONE, K., ALBERT, P., ALMAHAIRI, A., BABAEI, Y., BASHLYKOV, N., BATRA, S., BHARGAVA, P., BHOSALE, S. ET AL. (2023b). Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 . · arXiv
  63. WANG, P., YANG, A., MEN, R., LIN, J., BAI, S., LI, Z., MA, J., ZHOU, C., ZHOU, J. and YANG, H. (2022a). Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework. CoRR abs/2202.03052 . · arXiv
  64. WANG, Z., YU, J., YU, A. W., DAI, Z., TSVETKOV, Y. and CAO, Y. (2022b). SimVLM: Simple visual language model pretraining with weak supervision. In ICLR .
  65. WEI, C., SHEN, K., CHEN, Y. and MA, T. (2020). Theoretical analysis of self-training with deep networks on unlabeled data. arXiv preprint arXiv:2010.03622 . · arXiv
  66. WU, T., YANG, G., LI, Z., ZHANG, K., LIU, Z., GUIBAS, L., LIN, D. and WETZSTEIN, G. (2024). Gpt-4v (ision) is a human-aligned evaluator for text-to-3d generation. arXiv preprint arXiv:2401.04092 . · arXiv
  67. XIE, Q., LUONG, M.-T., HOVY, E. and LE, Q. V. (2020). Self-training with noisy student improves imagenet classification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition .
  68. XIONG, W., DONG, H., YE, C., ZHONG, H., JIANG, N. and ZHANG, T. (2023). Gibbs sampling from human feedback: A provable kl-constrained framework for rlhf. arXiv preprint arXiv:2312.11456 . · arXiv
  69. XU, J., LEE, A., SUKHBAATAR, S. and WESTON, J. (2023). Some things are more cringe than others: Preference optimization with the pairwise cringe loss. arXiv preprint arXiv:2312.16682 . · arXiv
  70. YU, W., YANG, Z., LI, L., WANG, J., LIN, K., LIU, Z., WANG, X. and WANG, L. (2023). Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308.02490 . · arXiv
  71. YUAN, W., PANG, R. Y., CHO, K., SUKHBAATAR, S., XU, J. and WESTON, J. (2024). Selfrewarding language models. arXiv preprint arXiv:2401.10020 . · arXiv
  72. ZHAO, Y., JOSHI, R., LIU, T., KHALMAN, M., SALEH, M. and LIU, P. J. (2023). Slic-hf: Sequence likelihood calibration with human feedback. arXiv preprint arXiv:2305.10425 . · arXiv
  73. ZHENG, C., WANG, Z., JI, H., HUANG, M. and PENG, N. (2024). Weak-to-strong extrapolation expedites alignment. arXiv preprint arXiv:2404.16792 . · arXiv
  74. ZHOU, Y., CUI, C., RAFAILOV, R., FINN, C. and YAO, H. (2024). Aligning modalities in vision large language models via preference fine-tuning. arXiv preprint arXiv:2402.11411 . · arXiv
  75. ZOPH, B., GHIASI, G., LIN, T.-Y., CUI, Y., LIU, H., CUBUK, E. D. and LE, Q. (2020). Rethinking pre-training and self-training. Advances in neural information processing systems 33 3833-3845.

Cited by

Sorted by citations

Related

Authors
Deng, Y., Lu, P., Yin, F., Hu, Z., Shen, S., Gu, Q., Zou, J., Chang, K.-W., & Wang, W.
ORCID
Venue
Advances in Neural Information Processing Systems 37
Publisher
Neural Information Processing Systems Foundation, Inc. (NeurIPS)
Published
2024
Pages
131369-131397
ISSN
1049-5258 (ISSN-L)
DOI
10.52202/079017-4175
OpenAlex
W4415800540
Cited by
7
References
0
Citations by year
2026: 5 · 2025: 2
FWCI
1.44
Type
conference paper
Language
en
Open access
Locations
  • neural information processing systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
large vision-language models, self-training, image understanding, vision-language models, self-supervised learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review