CoChat
Sign Up
Cited by 3Multimodal Machine Learning ApplicationsGenerative Adversarial Networks and Image SynthesisNatural Language Processing Techniques

The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.

Cycle Consistency as Reward: Learning Image-Text Alignment Without Human Preferences

Open access
Bahng, Hyojin, Chan, Caroline, Durand, Frédo, Isola, Phillip·2025·2025 IEEE/CVF International Conference on Computer Vision (ICCV)
DOI 10.1109/iccv51701.2025.02129
Open this paper in CoChat — free accountOpen PDF

Abstract

Measuring alignment between language and vision is a fundamental challenge, especially as multimodal data becomes increasingly detailed and complex. Existing methods often rely on collecting human or AI preferences, which can be costly and time-intensive. We propose an alternative approach that leverages cycle consistency as a supervisory signal. Given an image and generated text, we map the text back to image space using a text-to-image model and compute the similarity between the original image and its reconstruction. Analogously, for text-to-image generation, we measure the textual similarity between an input caption and its reconstruction through the cycle. We use the cycle consistency score to rank candidates and construct a preference dataset of 866K comparison pairs. The reward model trained on our dataset, CycleReward, outperforms state-of-the-art alignment metrics on detailed captioning, with superior inference-time scalability when used as a verifier for Best-of-N sampling, while maintaining speed and differentiability. Furthermore, performing DPO and Diffusion DPO using our dataset enhances performance across a wide range of vision-language tasks and text-to-image generation. Our dataset, model, and code are publicly released at https://cyclereward.github.io.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/iccv51701.2025.02129

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 106

  1. 10.1007/978-3-319-46454-1_24 · DOI · Paper page
  2. Bai, 2023, Qwen-vl: A frontier large vision-language model with versatile abilities, arXiv preprint, 1
  3. Betker, 2023, Improving image generation with better captions, Computer Science, 2
  4. Black, 2024, Training diffusion models with reinforcement learning
  5. 2024, Announcing black forest labs
  6. 10.1177/135910457000100301 · DOI · Paper page
  7. David, 2023, Clair: Evaluating image captions with large language models
  8. 10.1007/978-3-031-72643-9_22 · DOI
  9. 10.1007/s11432-024-4231-5 · DOI
  10. Cobbe, 2021, Training verifiers to solve math word problems, arXiv preprint
  11. 10.52202/075280-2142 · DOI
  12. Deng, 2025, Less is more: Improving llm alignment via preference data selection, arXiv preprint
  13. 10.18653/v1/2023.emnlp-main.798 · DOI
  14. 10.18653/vl/N19-142 · DOI
  15. 10.18653/v1/2024.findings-acl.212 · DOI · Paper page
  16. Dong, 2024, Benchmarking and improving detail image caption, arXiv preprint
  17. Dosovitskiy, 2021, An image is worth 16x16 words: Transformers for image recognition at scale, ICLR
  18. 10.52202/075280-3327 · DOI
  19. Dubey, 2024, The llama 3 herd of models, arXiv preprint
  20. Esser, 2024, Scaling rectified flow transformers for high-resolution image synthesis
  21. Fu, 2023, Mme: A comprehensive evaluation benchmark for multimodal large language models, arXiv preprint
  22. Fu, 2023, Dreamsim: Learning new dimensions of human visual similarity using synthetic data, arXiv preprint
  23. 10.18653/v1/2024.emnlp-main.6 · DOI
  24. 2023, Gemini: a family of highly capable multimodal models
  25. Godard, 2017, Unsupervised monocular depth estimation with leftright consistency, 270
  26. Krishna Gorti, 2018, Text-to-image-totext translation using cycle consistent adversarial networks, arXiv preprint
  27. 10.1609/aaai.v38i16.29771 · DOI · Paper page
  28. He, 2016, Dual learning for machine translation, Advances in neural information processing systems
  29. 10.18653/v1/2021.emnlp-main.595 · DOI · Paper page
  30. Hoffman, 2018, Cycada: Cycle-consistent adversarial domain adaptation, 1989
  31. 10.1109/cvpr52733.2024.01354 · DOI · Paper page
  32. 10.1007/978-981-96-2071-5_30 · DOI
  33. 10.52202/075280-3443 · DOI · Paper page
  34. 10.1111/cgf.12184 · DOI
  35. Huh, 2024, The platonic representation hypothesis
  36. Jabri, 2020, Space-time correspondence as a contrastive random walk, Advances in neural information processing systems, 33, 19545
  37. 10.1109/CVPR52729.2023.00263 · DOI
  38. Jinnai, 2024, Regularized best-of-n sampling to mitigate reward hacking for language model alignment
  39. 10.1109/ICPR.2010.675 · DOI
  40. 10.52202/075280-1594 · DOI
  41. 10.3115/1626355.1626389 · DOI · Paper page
  42. 10.18653/v1/2024.acl-long.205 · DOI
  43. 10.1109/cvprw63382.2024.00538 · DOI
  44. Li, 2024, Llava-onevision: Easy visual task transfer, arXiv preprint
  45. Li, 2017, Alice: Towards understanding adversarial learning for joint distribution matching, Advances in neural information processing systems, 30
  46. Li, 2022, Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation, 12888
  47. Li, 2023, Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language.models, 19730
  48. 10.18653/v1/2024.emnlp-main.358 · DOI
  49. Li, 2024, Leveraging unpaired data for vision-language generative models via cycle consistency
  50. 10.18653/v1/2023.emnlp-main.20 · DOI
  51. 10.1007/978-3-319-10602-1_48 · DOI · Paper page
  52. 10.1007/978-3-031-72673-6_20 · DOI
  53. 10.52202/075280-1516 · DOI · Paper page
  54. 10.1109/CVPR52733.2024.02484 · DOI · Paper page
  55. Liu, 2024, Visual instruction tuning, Advances in neural information processing systems, 36
  56. Loshchilov, 2019, Decoupled weight decay regularization
  57. Loshchilov, 2019, Decoupled weight decay regularization
  58. Ma, 2025, Inference-time scaling for diffusion models beyond scaling denoising steps, arXiv preprint
  59. 10.1109/LRA.2022.3145053 · DOI
  60. Nakano, 2021, Webgpt: Browser-assisted question-answering with human feedback, arXiv preprint
  61. 2024, Hello gpt-4o
  62. 2023, Gpt-4v(ision) system card
  63. 2024, OpenGVLab
  64. Oquab, 2024, Dinov2: Learning robust visual features without supervision, Transactions on Machine Learning Research
  65. 10.3115/1073083.1073135 · DOI · Paper page
  66. Podell, 2023, Sdxl: Improving latent diffusion models for high-resolution image synthesis, arXiv preprint
  67. Prabhudesai, 2023, Aligning text-to-image diffusion models with reward backpropagation
  68. Radford, 2021, Learning transferable visual models from natural language supervision, 8748
  69. 10.52202/075280-2338 · DOI · Paper page
  70. Saketh Rambhatla, 2025, Selfeval: Leveraging discriminative nature of generative models for evaluation, Transactions on Machine Learning Research
  71. 10.18653/v1/D19-1410 · DOI · Paper page
  72. 10.1109/CVPR52688.2022.01042 · DOI · Paper page
  73. 10.52202/068431-2643 · DOI · Paper page
  74. Sauer, 2023, Adversarial diffusion distillation, arXiv preprint
  75. Schulman, 2017, Proximal policy optimization algorithms, arXiv preprint
  76. 10.1109/CVPR.2019.00681 · DOI
  77. Shao, 2024, Deepseekmath: Pushing the limits of mathematical reasoning in open language models, arXiv preprint
  78. Sharifzadeh, 2024, Synth2: Boosting visual-language models with synthetic captions and image embeddings, arXiv preprint
  79. Stiennon, 2020, Christiano. Learning to summarize with human feedback, Advances in neural information processing systems, 33, 3008
  80. 10.18653/v1/2024.findings-acl.775 · DOI
  81. 10.1007/978-3-642-15549-9_32 · DOI
  82. Team, 2024, Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context, arXiv preprint
  83. 10.1109/cvpr52733.2024.02521 · DOI
  84. 10.1109/CVPR.2015.7299087 · DOI · Paper page
  85. 10.1109/CVPR52733.2024.00786 · DOI
  86. 10.1109/iccv.2013.110 · DOI
  87. Wang, 2023, Cogvlm: Visual expert for pretrained language models
  88. Wu, 2023, Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis, arXiv preprint
  89. 10.1109/ICCV51070.2023.00200 · DOI
  90. 10.1609/aaai.v40i13.38107 · DOI
  91. Xu, 2024, Imagereward: Learning and evaluating human.preferences for text-to-image generation, Advances in Neural Information Processing Systems, 36
  92. Ye, 2025, Painting with words: Elevating detailed image captioning with benchmark and alignment learning
  93. Yeh, 2024, How reliable is human feedback for aligning large language models?, arXiv preprint
  94. 10.1109/ICCV.2017.310 · DOI
  95. Yu, 2022, Scaling autoregressive models for content-rich text-to-image generation, Transactions on Machine Learning Research
  96. 10.1109/CVPR52733.2024.01310 · DOI
  97. 10.1109/cvpr52734.2025.01861 · DOI
  98. Yuksekgonul, 2023, When and why visionlanguage.models behave like bags-of-words, and what to.do about it?
  99. 10.1109/CVPR.2010.5539801 · DOI
  100. 10.1109/cvpr.2018.00068 · DOI · Paper page
  101. Zhang, 2019, Bertscore: Evaluating text generation with bert, arXiv preprint
  102. Zhao, 2023, Beyond hallucinations: Enhancing lvlms through hallucination-aware direct preference optimization, arXiv preprint
  103. 10.1109/cvpr.2015.7298723 · DOI
  104. 10.1109/cvpr.2016.20 · DOI
  105. Zhou, 2024, Aligning modalities in vision large language models via preference fine-tuning
  106. 10.1109/iccv.2017.244 · DOI · Paper page

Cited by

Sorted by citations

See all 3

Metadata

Authors
Bahng, H., Chan, C., Durand, F., & Isola, P.
Affiliations
MIT Computer Science and Artificial Intelligence Laboratory (US)
ORCID
Venue
2025 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher
IEEE
Published
2025-10-19
Pages
22934-22946
DOI
10.1109/iccv51701.2025.02129
OpenAlex
W4415130663
Cited by
3
References
1
Citations by year
2026: 3
FWCI
2.27
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
image-text alignment, cycle consistency, reward modeling, preference dataset, text-to-image generation, captioning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review