Adding Conditional Control to Text-to-Image Diffusion Models
Abstract
We present ControlNet, a neural network architecture to add spatial conditioning controls to large, pretrained text-to-image diffusion models. ControlNet locks the production-ready large diffusion models, and reuses their deep and robust encoding layers pretrained with billions of images as a strong backbone to learn a diverse set of conditional controls. The neural architecture is connected with "zero convolutions" (zero-initialized convolution layers) that progressively grow the parameters from zero and ensure that no harmful noise could affect the finetuning. We test various conditioning controls, e.g., edges, depth, segmentation, human pose, etc., with Stable Diffusion, using single or multiple conditions, with or without prompts. We show that the training of ControlNets is robust with small (1m) datasets. Extensive results show that ControlNet may facilitate wider applications to control image diffusion models.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Image annotations, including prompts and spatial cues, are essential for training and evaluating image generation models as they provide localized and semantically rich supervision that help models learn meaningful visual representations.
“Image annotations, including prompts and spatial cues [92, 121], are essential for training and evaluating image generation models as they provide localized and semantically rich supervision that help models learn meaningful visual representations.”
From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [121]
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126 · see citing works
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · see citing works
A Computational Approach to Edge Detection
Canny, J. (1986) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1986.4767851 · Cited by 30,004 · see citing works
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602 · see citing works
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930 · see citing works
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 15,880 · see citing works
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · see citing works
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881 · see citing works
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,869 · see citing works
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714 · see citing works
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365 · see citing works
OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., & Sheikh, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2929257 · Cited by 5,536 · see citing works
Progressive Growing of GANs for Improved Quality, Stability, and Variation
Karras, T., Aila, T., Laine, S., & Lehtinen, J. (2017) · arXiv · doi:10.48550/arxiv.1710.10196 · Cited by 4,727 · see citing works
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
Wang, T.-C., Liu, M.-Y., Zhu, J.-Y., Tao, A., Kautz, J., & Catanzaro, B. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00917 · Cited by 4,514 · see citing works
StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation
Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., & Choo, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00916 · Cited by 4,163 · see citing works
Holistically-Nested Edge Detection
Xie, S., & Tu, Z. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.164 · Cited by 3,401 · see citing works
Diffusion Models Beat GANs on Image Synthesis
Dhariwal, P., & Nichol, A. (2021) · arXiv · doi:10.48550/arxiv.2105.05233 · Cited by 3,362 · see citing works
Scene Parsing through ADE20K Dataset
Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., & Torralba, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.544 · Cited by 3,326 · see citing works
Semantic Image Synthesis With Spatially-Adaptive Normalization
Park, T., Liu, M.-Y., Wang, T.-C., & Zhu, J.-Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00244 · Cited by 2,974 · see citing works
Deep Unsupervised Learning using Nonequilibrium Thermodynamics
Sohl-Dickstein, J., Weiss, E. A., Maheswaranathan, N., & Ganguli, S. (2015) · arXiv · doi:10.48550/arxiv.1503.03585 · Cited by 2,637 · see citing works
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E., Ghasemipour, S. K. S., Ayan, B. K., Mahdavi, S. S., Lopes, R. G., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.11487 · Cited by 2,528 · see citing works
LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 2,523 · see citing works
Taming Transformers for High-Resolution Image Synthesis
Esser, P., Rombach, R., & Ommer, B. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01268 · Cited by 2,514 · see citing works
Hierarchical Text-Conditional Image Generation with CLIP Latents
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M. (2022) · arXiv · doi:10.48550/arxiv.2204.06125 · Cited by 2,511 · cited 2× · see citing works
“The image diffusion process directly provides some control over color variation [53] and inpainting [66, 7].”
BackgroundMethodZero-shot text-to-image generation
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., & Sutskever, I. (2021) · arXiv · doi:10.48550/arxiv.2102.12092 · Cited by 2,417 · see citing works
Pre-Trained Image Processing Transformer
Chen, H., Wang, Y., Guo, T., Xu, C., Deng, Y., Liu, Z., Ma, S., Xu, C., Xu, C., & Gao, W. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01212 · Cited by 2,308 · see citing works
DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation
Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., & Aberman, K. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.02155 · Cited by 2,295 · see citing works
Parameter-efficient transfer learning for NLP
Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., de Laroussilhe, Q., Gesmundo, A., Attariyan, M., & Gelly, S. (2019) · arXiv · doi:10.48550/arxiv.1902.00751 · Cited by 2,270 · see citing works
Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer
Ranftl, R., Lasinger, K., Hafner, D., Schindler, K., & Koltun, V. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3019967 · Cited by 2,047 · see citing works
Palette: Image-to-Image Diffusion Models
Saharia, C., Chan, W., Chang, H., Lee, C., Ho, J., Salimans, T., Fleet, D., & Norouzi, M. (2022) · Special Interest Group on Computer Graphics and Interactive Techniques Conference Proceedings · doi:10.1145/3528233.3530757 · Cited by 1,717 · see citing works
Improved Denoising Diffusion Probabilistic Models
Nichol, A., & Dhariwal, P. (2021) · arXiv · doi:10.48550/arxiv.2102.09672 · Cited by 1,550 · see citing works
Instructpix2pix: Learning to follow image editing instructions
Brooks, T., Holynski, A., & Efros, A. A. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01764 · Cited by 1,466 · see citing works
GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I., & Chen, M. (2021) · arXiv · doi:10.48550/arxiv.2112.10741 · Cited by 1,357 · see citing works
CLIP-Adapter: Better Vision-Language Models with Feature Adapters
Gao, P., Geng, S., Zhang, R., Ma, T., Fang, R., Zhang, Y., Li, H., & Qiao, Y. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01891-x · Cited by 1,341 · see citing works
PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning
Mallya, A., & Lazebnik, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00810 · Cited by 1,258 · see citing works
Toward Multimodal Image-to-Image TranslationPreprint
Zhu, J.-Y., Zhang, R., Pathak, D., Darrell, T., Efros, A. A., Wang, O., & Shechtman, E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.11586 · Cited by 1,159 · see citing works
Classifier-Free Diffusion Guidance
Ho, J., & Salimans, T. (2022) · arXiv · doi:10.48550/arxiv.2207.12598 · Cited by 1,126 · see citing works
Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation
Richardson, E., Alaluf, Y., Patashnik, O., Nitzan, Y., Azar, Y., Shapiro, S., & Cohen‐Or, D. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00232 · Cited by 1,120 · see citing works
StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery
Patashnik, O., Wu, Z., Shechtman, E., Cohen‐Or, D., & Lischinski, D. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00209 · Cited by 1,079 · see citing works
Mou, C., Wang, X., Xie, L., Wu, Y., Zhang, J., Qi, Z., & Shan, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28226 · Cited by 1,026 · see citing works
Vision Transformer Adapter for Dense PredictionsPreprint
Chen, Z., Duan, Y., Wang, W., He, J.-J., Lu, T., Dai, J.-F., & Qiao, Y. (2022) · International Conference on Learning Representations · doi:10.48550/arxiv.2205.08534 · Cited by 959 · see citing works
Exploring Plain Vision Transformer Backbones for Object Detection
Li, Y., Mao, H., Girshick, R., & He, K. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20077-9_17 · Cited by 903 · see citing works
Imagic: Text-Based Real Image Editing with Diffusion Models
Kawar, B., Zada, S., Lang, O., Tov, O., Chang, H., Dekel, T., Mosseri, I., & Irani, M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00582 · Cited by 813 · see citing works
Blended Diffusion for Text-driven Editing of Natural Images
Avrahami, O., Lischinski, D., & Fried, O. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01767 · Cited by 799 · see citing works
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2021) · arXiv · doi:10.48550/arxiv.2112.10752 · Cited by 737 · see citing works
Piggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights
Mallya, A., Davis, D., & Lazebnik, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01225-0_5 · Cited by 644 · see citing works
Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation
Tumanyan, N., Geyer, M., Bagon, S., & Dekel, T. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00191 · Cited by 644 · see citing works
An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion
Gal, R., Alaluf, Y., Atzmon, Y., Patashnik, O., Bermano, A. H., Chechik, G., & Cohen-Or, D. (2022) · arXiv · doi:10.48550/arxiv.2208.01618 · Cited by 611 · see citing works
GLIGEN: Open-Set Grounded Text-to-Image Generation
Li, Y., Liu, H., Wu, Q., Mu, F., Yang, J., Gao, J., Li, C., & Lee, Y. J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02156 · Cited by 610 · see citing works
Measuring the Intrinsic Dimension of Objective LandscapesPreprint
Li, C.-Y., Farkhoor, H., Liu, R., & Yosinski, J. (2018) · International Conference on Learning Representations · doi:10.48550/arxiv.1804.08838 · Cited by 607 · see citing works
DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation
Kim, G., Kwon, T., & Ye, J. C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00246 · Cited by 570 · see citing works
Overcoming catastrophic forgetting with hard attention to the taskPreprint
Serr\`a, J., Surís, D., Miron, M., & Karatzoglou, A. (2018) · arXiv (Cornell University) · Cited by 564 · see citing works
OneFormer: One Transformer to Rule Universal Image Segmentation
Jain, J., Li, J., Chiu, M. T., Hassani, A., Orlov, N., & Shi, H. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00292 · Cited by 550 · see citing works
SDEdit: Guided Image Synthesis and Editing with Stochastic Differential EquationsPreprint
Meng, C., He, Y., Song, Y., Song, J., Wu, J., Zhu, J.-Y., & Ermon, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.01073 · Cited by 528 · see citing works
Gal, R., Patashnik, O., Maron, H., Bermano, A. H., Chechik, G., & Cohen‐Or, D. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530164 · Cited by 503 · see citing works
Prompt-to-Prompt Image Editing with Cross Attention ControlPreprint
Hertz, A., Mokady, R., Tenenbaum, J., Aberman, K., Pritch, Y., & Cohen‐Or, D. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2208.01626 · Cited by 496 · see citing works
Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning
Aghajanyan, A., Gupta, S., & Zettlemoyer, L. (2021) · doi:10.18653/v1/2021.acl-long.568 · Cited by 429 · see citing works
Zero-shot Image-to-Image Translation
Parmar, G., Singh, K. K., Zhang, R., Li, Y., Lu, J., & Zhu, J.-Y. (2023) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3588432.3591513 · Cited by 383 · see citing works
Kingma, D. P., Salimans, T., Poole, B., & Ho, J. (2021) · arXiv · doi:10.48550/arxiv.2107.00630 · Cited by 374 · see citing works
Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors
Gafni, O., Polyak, A., Ashual, O., Sheynin, S., Parikh, D., & Taigman, Y. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19784-0_6 · Cited by 345 · see citing works
VL-ADAPTER: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks
Sung, Y.-L., Cho, J., & Bansal, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00516 · Cited by 329 · see citing works
Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language ModelingPreprint
Zhang, R., Fang, R., Zhang, W., Gao, P., Li, K., Dai, J., Qiao, Y., & Li, H. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.03930 · Cited by 285 · see citing works
HyperStyle: StyleGAN Inversion with HyperNetworks for Real Image Editing
Alaluf, Y., Tov, O., Mokady, R., Gal, R., & Bermano, A. H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01796 · Cited by 270 · see citing works
Incremental Learning Through Deep Adaptation
Rosenfeld, A., & Tsotsos, J. K. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2884462 · Cited by 260 · see citing works
Cross-Domain Correspondence Learning for Exemplar-Based Image Translation
Zhang, P., Zhang, B., Chen, D., Yuan, L., & Wen, F. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00519 · Cited by 255 · see citing works
Sketch-Guided Text-to-Image Diffusion Models
Voynov, A. S., Aberman, K., & Cohen‐Or, D. (2023) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3588432.3591560 · Cited by 191 · see citing works
Side-Tuning: A Baseline for Network Adaptation via Additive Side Networks
Zhang, J. O., (Sasha), A. S., Zamir, A., Guibas, L. J., & Malik, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58580-8_41 · Cited by 190 · see citing works
SpaText: Spatio-Textual Representation for Controllable Image Generation
Avrahami, O., Hayes, T., Gafni, O., Gupta, S., Taigman, Y., Parikh, D., Lischinski, D., Fried, O., & Yin, X. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01762 · Cited by 171 · see citing works
DIODE: A Dense Indoor and Outdoor DEpth DatasetPreprint
Vasiljevic, I., Kolkin, N., Zhang, S., Luo, R., Wang, H., Dai, F. Z., Daniele, A. F., Mostajabi, M., Basart, S., Walter, M. R., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.00463 · Cited by 169 · see citing works
Alaluf, Y., Patashnik, O., & Cohen‐Or, D. (2021) · ACM Transactions on Graphics · doi:10.1145/3450626.3459805 · Cited by 161 · see citing works
Pretraining is All You Need for Image-to-Image TranslationPreprint
Wang, T., Zhang, T., Zhang, B., Ouyang, H., Chen, D., Chen, Q., & Wen, F. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.12952 · Cited by 158 · see citing works
MultiDiffusion: Fusing Diffusion Paths for Controlled Image GenerationPreprint
Bar-Tal, O., Yariv, L., Lipman, Y., & Dekel, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.08113 · Cited by 137 · see citing works
Composer: Creative and Controllable Image Synthesis with Composable ConditionsPreprint
Huang, L., Chen, D., Liu, Y., Shen, Y., Zhao, D., & Zhou, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.09778 · Cited by 120 · see citing works
HyperInverter: Improving StyleGAN Inversion via Hypernetwork
Dinh, T. M., Tran, A. T., Nguyen, R. M. H., & Hua, B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01110 · Cited by 117 · see citing works
CoCosNet v2: Full-Resolution Correspondence Learning for Image Translation
Zhou, X., Zhang, B., Zhang, T., Zhang, P., Bao, J., Chen, D., Zhang, Z. M., & Wen, F. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01130 · Cited by 116 · see citing works
BERT and PALs: Projected Attention Layers for Efficient Adaptation in Multi-Task LearningPreprint
Stickland, A. C., & Murray, I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.02671 · Cited by 109 · see citing works
Towards Light-Weight and Real-Time Line Segment Detection
Gu, G., Ko, B., Go, S., Lee, S., Lee, J., & Shin, M. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i1.19953 · Cited by 104 · see citing works
Benchmarking Detection Transfer Learning with Vision TransformersPreprint
Li, Y., Xie, S., Chen, X., Dollár, P., He, K., & Girshick, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.11429 · Cited by 89 · see citing works
Nitzan, Y., Aberman, K., He, Q., Liba, O., Yarom, M., Gandelsman, Y., Mosseri, I., Pritch, Y., & Cohen‐Or, D. (2022) · ACM Transactions on Graphics · doi:10.1145/3550454.3555436 · Cited by 73 · see citing works
MaskSketch: Unpaired Structure-guided Masked Image Generation
Bashkirova, D., Lezama, J., Sohn, K., Saenko, K., & Essa, I. A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00187 · Cited by 32 · see citing works
Self-Distilled StyleGAN: Towards Generation from Internet Photos
Mokady, R., Tov, O., Yarom, M., Lang, O., Mosseri, I., Dekel, T., Cohen‐Or, D., & Irani, M. (2022) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3528233.3530708 · Cited by 29 · see citing works
Region-Aware Diffusion for Zero-shot Text-driven Image EditingPreprint
Huang, N., Tang, F., Dong, W., Lee, T., & Xu, C. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2302.11797 · Cited by 16 · see citing works
Multi-level Latent Space Structuring for Generative ControlPreprint
Katzir, O., Perepelook, V., Lischinski, D., & Cohen‐Or, D. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.05910 · Cited by 3 · see citing works
ZerO Initialization: Initializing Residual Networks with only Zeros and Ones
Zhao, J., Schäfer, F., & Anandkumar, A. (2021) · CaltechAUTHORS (California Institute of Technology) · Cited by 3 · see citing works
Flagged first, then by citations
5 of the 91 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Diffusion Models: A Comprehensive Survey of Methods and Applications
Yang, L., Zhang, Z., Song, Y., Hong, S., Xu, R., Zhao, Y., Zhang, W., Cui, B., & Yang, M.-H. (2024) · ACM Computing Surveys · doi:10.1145/3626235 · Cited by 1,704 · see citing works
Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data
Yang, L., Kang, B., Huang, Z., Xu, X., Feng, J., & Zhao, H. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00987 · Cited by 1,296 · see citing works
Mou, C., Wang, X., Xie, L., Wu, Y., Zhang, J., Qi, Z., & Shan, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28226 · Cited by 1,026 · see citing works
A Survey on Generative Diffusion Models
Cao, H., Tan, C., Gao, Z., Xu, Y., Chen, G., Heng, P., & Li, S. Z. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3361474 · Cited by 679 · see citing works
Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation
Wu, J. Z., Ge, Y., Wang, X., Lei, S. W., Gu, Y., Shi, Y., Hsu, W., Shan, Y., Qie, X., & Shou, M. Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00701 · Cited by 622 · see citing works
A generative model for inorganic materials design
Zeni, C., Pinsler, R., Zügner, D., Fowler, A., Horton, M., Fu, X., Wang, Z., Shysheya, A., Crabbé, J., Ueda, S., et al. (2025) · Nature · doi:10.1038/s41586-025-08628-5 · Cited by 518 · see citing works
Exploiting Diffusion Prior for Real-World Image Super-Resolution
Wang, J., Yue, Z., Zhou, S., Chan, K. C. K., & Loy, C. C. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02168-7 · Cited by 488 · see citing works
MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing
Cao, M., Wang, X., Qi, Z., Shan, Y., Qie, X., & Zheng, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02062 · Cited by 443 · see citing works
Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation
Ke, B., Obukhov, A., Huang, S., Metzger, N., Daudt, R. C., & Schindler, K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00907 · Cited by 444 · see citing works
Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators
Khachatryan, L., Movsisyan, A., Tadevosyan, V., Henschel, R. D., Wang, Z., Navasardyan, S., & Shi, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01462 · Cited by 426 · see citing works
A Survey on Multimodal Large Language Models for Autonomous Driving
Cui, C., Ma, Y., Cao, X., Ye, W., Zhou, Y., Liang, K., Chen, J., Lu, J., Yang, Z., Liao, K.-D., et al. (2024) · 2024 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw60836.2024.00106 · Cited by 369 · see citing works
Prompting AI Art: An Investigation into the Creative Skill of Prompt Engineering
Oppenlaender, J., Linder, R., & Silvennoinen, J. M. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2431761 · Cited by 308 · see citing works
Lightweight Deep Learning for Resource-Constrained Environments: A Survey
Liu, H.-I., Galindo, M., Xie, H., Wong, L.-K., Shuai, H.-H., Li, Y., & Cheng, W.-H. (2024) · ACM Computing Surveys · doi:10.1145/3657282 · Cited by 296 · see citing works
Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection
Wang, S., Liu, Y., Wang, T., Li, Y., & Zhang, X. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00335 · Cited by 292 · see citing works
Proceedings of the 32nd ACM International Conference on Multimedia
Pan, Q., Zhao, Z. W., Wang, Z., Long, S., Wu, Y., Ji, W., Liang, H., & Liang, R. (2024) · doi:10.1145/3664647 · Cited by 290 · see citing works
DiffBIR: Toward Blind Image Restoration with Generative Diffusion Prior
Lin, X., He, J., Chen, Z., Lyu, Z., Dai, B., Yu, F., Qiao, Y., Ouyang, W., & Dong, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73202-7_25 · Cited by 283 · see citing works
On the Opportunities and Challenges of Foundation Models for GeoAI (Vision Paper)
Mai, G., Huang, W., Sun, J., Song, S., Mishra, D. R., Liu, N., Gao, S., Liu, T., Cong, G., Hu, Y., et al. (2024) · ACM Transactions on Spatial Algorithms and Systems · doi:10.1145/3653070 · Cited by 275 · see citing works
Recent advances in 3D Gaussian splatting
Wu, T., Yuan, Y.-J., Zhang, L., Yang, J., Cao, Y., Yan, L., & Gao, L. (2024) · Computational Visual Media · doi:10.1007/s41095-024-0436-y · Cited by 270 · see citing works
Self-citationFateZero: Fusing Attentions for Zero-shot Text-based Video Editing
Qi, C., Cun, X., Zhang, Y., Lei, C., Wang, X., Shan, Y., & Chen, Q. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01460 · Cited by 256 · see citing works
The Stable Signature: Rooting Watermarks in Latent Diffusion Models
Fernandez, P., Couairon, G., Jégou, H., Douze, M., & Furon, T. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02053 · Cited by 251 · see citing works
Effective Whole-body Pose Estimation with Two-stages Distillation
Yang, Z., Zeng, A., Yuan, C., & Li, Y. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00455 · Cited by 242 · see citing works
A Diffusion-Based Framework for Multi-Class Anomaly Detection
He, H., Zhang, J., Chen, H., Chen, X., Li, Z., Chen, X., Wang, Y., Wang, C., & Xie, L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i8.28690 · Cited by 242 · see citing works
Multimodal Foundation Models: From Specialists to General-Purpose Assistants
Li, C., Gan, Z., Yang, Z., Yang, J., Li, L., Wang, L., & Gao, J. (2024) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000110 · Cited by 225 · see citing works
SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution
Wu, R., Yang, T., Sun, L., Zhang, Z., Li, S., & Zhang, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02405 · Cited by 223 · see citing works
SVDiff: Compact Parameter Space for Diffusion Fine-Tuning
Han, L., Li, Y., Zhang, H., Milanfar, P., Metaxas, D., & Yang, F. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00673 · Cited by 224 · see citing works
LaVie: High-Quality Video Generation with Cascaded Latent Diffusion Models
Wang, Y., Chen, X., Ma, X., Zhou, S., Huang, Z., Yang, C., Yang, C., He, Y., Yu, J., Yang, P., et al. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02295-1 · Cited by 218 · see citing works
ECON: Explicit Clothed humans Optimized via Normal integration
Xiu, Y., Yang, J., Cao, X., Tzionas, D., & Black, M. J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00057 · Cited by 206 · see citing works
Brade, S., Wang, B. K., Sousa, M., Oore, S., & Grossman, T. (2023) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3586183.3606725 · Cited by 198 · see citing works
Generative AI models for different steps in architectural design: A literature review
Li, C., Zhang, T., Du, X., Zhang, Y., & Xie, H. (2024) · Frontiers of Architectural Research · doi:10.1016/j.foar.2024.10.001 · Cited by 198 · see citing works
InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuning
Shi, J., Xiong, W., Lin, Z., & Jung, H. J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00816 · Cited by 192 · see citing works
FastComposer: Tuning-Free Multi-subject Image Generation with Localized Attention
Xiao, G., Yin, T., Freeman, W. T., Durand, F., & Han, S. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02227-z · Cited by 189 · see citing works
Parameter-efficient fine-tuning in large language models: a survey of methodologies
Wang, L., Chen, S., Jiang, L., Pan, S., Cai, R., Yang, S., & Yang, F. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11236-4 · Cited by 186 · see citing works
Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation
Li, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00779 · Cited by 184 · see citing works
DriveDreamer: Towards Real-World-Drive World Models for Autonomous Driving
Wang, X., Zhu, Z. H., Huang, G., Chen, X., Zhu, J., & Lu, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73195-2_4 · Cited by 184 · see citing works
MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model
Xu, Z., Zhang, J., Liew, J. H., Yan, H., Liu, J., Zhang, C., Feng, J., & Shou, M. Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00147 · Cited by 182 · see citing works
Training-Free Layout Control with Cross-Attention Guidance
Chen, M., Laina, I., & Vedaldi, A. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00526 · Cited by 180 · see citing works
Text2Tex: Text-driven Texture Synthesis via Diffusion Models
Chen, D. Z., Siddiqui, Y., Lee, H.-Y., Tulyakov, S., & Nießner, M. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01701 · Cited by 178 · see citing works
All-in-One Weather-Degraded Image Restoration Via Adaptive Degradation-Aware Self-Prompting Model
Wen, Y., Gao, T., Li, Z., Zhang, J., Zhang, K., & Chen, T. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3535316 · Cited by 178 · see citing works
Pix2Video: Video Editing using Image Diffusion
Ceylan, D., Huang, C.-H. P., & Mitra, N. J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02121 · Cited by 174 · see citing works
BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusion
Xie, J., Li, Y., Huang, Y., Liu, H., Zhang, W., Zheng, Y., & Shou, M. Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00685 · Cited by 174 · see citing works
CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets
Zhang, L., Wang, Z., Zhang, Q., Qiu, Q., Pang, A., Jiang, H., Yang, W., Xu, L., & Jingyi Yu, A. (2024) · ACM Transactions on Graphics · doi:10.1145/3658146 · Cited by 173 · see citing works
Self-citationSemantic Image Synthesis via Diffusion Models
Zhou, W., Wang, W., Bao, J., Chen, D., Chen, D., Yuan, L., & Li, H. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3680594 · Cited by 170 · see citing works
Diffusion Models for Image Restoration and Enhancement: A Comprehensive Survey
Li, X., Ren, Y., Jin, X., Lan, C., Wang, X., Zeng, W., Wang, X., & Chen, Z. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02570-9 · Cited by 170 · see citing works
Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models
Höllein, L., Cao, A., Owens, A., Johnson, J., & Nießner, M. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00727 · Cited by 167 · see citing works
DreamBooth3D: Subject-Driven Text-to-3D Generation
Raj, A., Kaza, S., Poole, B., Niemeyer, M., Ruiz, N., Mildenhall, B., Zada, S., Aberman, K., Rubinstein, M., Barron, J. T., et al. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00223 · Cited by 167 · see citing works
Diffusion Model-Based Image Editing: A Survey
Huang, Y., Huang, J., Liu, Y., Yan, M., Lv, J., Liu, J., Xiong, W., Zhang, H., Cao, L., & Chen, S. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3541625 · Cited by 168 · see citing works
Evaluating the subjective perceptions of streetscapes using street-view images
Ogawa, Y., Oki, T., Zhao, C., Sekimoto, Y., & Shimizu, C. (2024) · Landscape and Urban Planning · doi:10.1016/j.landurbplan.2024.105073 · Cited by 165 · see citing works
Multimodal Image Synthesis and Editing: The Generative AI Era
Zhan, F., Yu, Y., Wu, R., Zhang, J., Lu, S., Liu, L., Kortylewski, A., Theobalt, C., & Xing, E. P. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3305243 · Cited by 162 · see citing works
PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding
Li, Z., Cao, M., Wang, X., Qi, Z., Cheng, M., & Shan, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00825 · Cited by 161 · see citing works
Lumiere: A Space-Time Diffusion Model for Video Generation
Bar-Tal, O., Chefer, H., Tov, O., Herrmann, C., Paiss, R., Zada, S., Ephrat, A., Hur, J., Liu, G., Raj, A., et al. (2024) · SIGGRAPH Asia Conference Papers · doi:10.1145/3680528.3687614 · Cited by 161 · see citing works
A Survey on Video Diffusion Models
Xing, Z., Feng, Q., Chen, H., Dai, Q., Hu, H., Xu, H., Wu, Z., & Jiang, Y. (2024) · ACM Computing Surveys · doi:10.1145/3696415 · Cited by 160 · see citing works
FreeU: Free Lunch in Diffusion U-Net
Si, C., Huang, Z., Jiang, Y., & Liu, Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00453 · Cited by 158 · see citing works
Zero-Reference Low-Light Enhancement via Physical Quadruple Priors
Wang, W., Yang, H., Fu, J., & Liu, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02462 · Cited by 159 · see citing works
State of the Art on Diffusion Models for Visual Computing
Po, R., Yifan, W., Golyanik, V., Aberman, K., Barron, J. T., Bermano, A. H., Chan, E. R., Dekel, T., Holynski, A., Kanazawa, A., et al. (2024) · Computer Graphics Forum · doi:10.1111/cgf.15063 · Cited by 157 · see citing works
Generative-AI-Driven Human Digital Twin in IoT Healthcare: A Comprehensive Survey
Chen, J., Shi, Y., Yi, C., Du, H., Kang, J., & Niyato, D. T. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3421918 · Cited by 155 · see citing works
HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Models
Ruiz, N., Li, Y., Jampani, V., Wei, W., Hou, T., Pritch, Y., Wadhwa, N., Rubinstein, M., & Aberman, K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00624 · Cited by 155 · see citing works
Style Aligned Image Generation via Shared Attention
Hertz, A., Voynov, A. S., Fruchter, S., & Cohen‐Or, D. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00457 · Cited by 153 · see citing works
Follow Your Pose: Pose-Guided Text-to-Video Generation Using Pose-Free Videos
Ma, Y., He, Y., Cun, X., Wang, X., Chen, S., Li, X., & Chen, Q. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28206 · Cited by 153 · see citing works
MotionCtrl: A Unified and Flexible Motion Controller for Video Generation
Wang, Z., Yuan, Z., Wang, X., Li, Y., Chen, T., Xia, M., Luo, P., & Shan, Y. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657518 · Cited by 153 · see citing works
Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation
Yang, S., Zhou, Y., Liu, Z., & Loy, C. C. (2023) · SIGGRAPH Asia Conference Papers · doi:10.1145/3610548.3618160 · Cited by 151 · see citing works
Versatile Diffusion: Text, Images and Variations All in One Diffusion Model
Xu, X., Wang, Z., Zhang, E., Wang, K., & Shi, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00713 · Cited by 150 · see citing works
Ablating Concepts in Text-to-Image Diffusion Models
Kumari, N., Zhang, B., Wang, S.-Y., Shechtman, E., Zhang, R., & Zhu, J.-Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02074 · Cited by 150 · see citing works
Generative AI for visualization: State of the art and future directions
Ye, Y., Hao, J., Hou, Y., Wang, Z., Xiao, S., Luo, Y., & Zeng, W. (2024) · Visual Informatics · doi:10.1016/j.visinf.2024.04.003 · Cited by 147 · see citing works
LLMR: Real-time Prompting of Interactive Worlds using Large Language Models
Torre, F. D. L., Fang, C. M., Huang, H., Banburski, A., Amores, J., & Lanier, J. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642579 · Cited by 145 · see citing works
LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching
Liang, Y., Yang, X., Lin, J., Li, H., Xu, X., & Chen, Y.-C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00623 · Cited by 147 · see citing works
Pixel-Aware Stable Diffusion for Realistic Image Super-Resolution and Personalized Stylization
Yang, T., Wu, R., Ren, P., Xie, X., & Lei Zhang, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73247-8_5 · Cited by 144 · see citing works
3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities
Bao, Y., Ding, T., Huo, J., Liu, Y., Li, Y., Li, W., Gao, Y., & Luo, J. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3538684 · Cited by 142 · see citing works
A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)
Deldjoo, Y., He, Z., McAuley, J., Korikov, A., Sanner, S., Arnau Ramisa, A., Vidal, R., Sathiamoorthy, M., Kasirzadeh, A., & Milano, S. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671474 · Cited by 142 · see citing works
PromptMagician: Interactive Prompt Engineering for Text-to-Image Creation
Feng, Y., Wang, X., Kam-Kwai, W., Wang, S., Lu, Y., Zhu, M., Wang, B., & Chen, W. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3327168 · Cited by 140 · see citing works
A comprehensive survey on deep active learning in medical image analysis
Wang, H., Jin, Q., Li, S., Liu, S., Wang, M., & Song, Z. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103201 · Cited by 139 · see citing works
Sapiens: Foundation for Human Vision Models
Khirodkar, R., Bagautdinov, T., Martinez, J., Zhaoen, S., James, A., Selednik, P., Anderson, S., & Saito, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73235-5_12 · Cited by 139 · see citing works
Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models
Ge, S., Nah, S., Liu, G., Poon, T., Tao, A., Catanzaro, B., Jacobs, D. W., Huang, J., Liu, M.-Y., & Balaji, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02096 · Cited by 138 · see citing works
DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion
Karras, J., Holynski, A., Wang, T.-C., & Kemelmacher-Shlizerman, I. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02073 · Cited by 137 · see citing works
Advances in deep concealed scene understanding
Fan, D.-P., Ji, G.-P., Xu, P., Cheng, M., Sakaridis, C., & Gool, L. V. (2023) · Visual Intelligence · doi:10.1007/s44267-023-00019-6 · Cited by 137 · see citing works
Diffusion Augmentation for Sequential Recommendation
Liu, Q., Fan, Y. B., Zhao, X., Du, Z., Guo, H., Tang, R., & Tian, F. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3615134 · Cited by 137 · see citing works
FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model
Yu, J., Wang, Y., Zhao, C., Ghanem, B., & Zhang, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02118 · Cited by 131 · see citing works
DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models
Cao, Y., Cao, Y., Han, K., Shan, Y., & Wong, K.-Y. K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00097 · Cited by 130 · see citing works
Stable VITON: Learning Semantic Correspondence with Latent Diffusion Model for Virtual Try-On
Kim, J., Gu, G., Park, M.-H., Park, S., & Choo, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00781 · Cited by 131 · see citing works
CreativeConnect: Supporting Reference Recombination for Graphic Design Ideation with Generative AI
Choi, D., Hong, S., Park, J., Chung, J. J. Y., & Kim, J. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642794 · Cited by 124 · see citing works
From Matching to Generation: A Survey on Generative Information Retrieval
Li, X., Jin, J., Zhou, Y., Zhang, Y., Zhang, P., Zhu, Y., & Dou, Z. (2025) · ACM Transactions on Information Systems · doi:10.1145/3722552 · Cited by 124 · see citing works
StableVideo: Text-driven Consistency-aware Diffusion Video Editing
Chai, W., Guo, X., Wang, G., & Lu, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02106 · Cited by 122 · see citing works
Evaluating Text-to-Visual Generation with Image-to-Text Generation
Lin, Z., Pathak, D., Li, B., Li, J., Xia, X., Neubig, G., Zhang, P., & Ramanan, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72673-6_20 · Cited by 124 · see citing works
Diffusion Models, Image Super-Resolution, and Everything: A Survey
Moser, B. B., Shanbhag, A. S., Raue, F., Frolov, S., Palacio, S., & Dengel, A. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3476671 · Cited by 117 · see citing works
On the design fundamentals of diffusion models: A survey
Chang, Z., Koulieris, G. A., Chang, H. J., & Shum, H. P. H. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111934 · Cited by 117 · see citing works
Deep learning for cross-domain data fusion in urban computing: Taxonomy, advances, and outlook
Zou, X., Yan, Y., Hao, X., Hu, Y., Wen, H., Liu, E., Zhang, J., Li, Y., Li, T., Zheng, Y., et al. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102606 · Cited by 117 · see citing works
Dense Text-to-Image Generation with Attention Modulation
Kim, Y., Lee, J., Kim, J.-H., Ha, J.-W., & Zhu, J.-Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00708 · Cited by 115 · see citing works
Beyond Reality: The Pivotal Role of Generative AI in the Metaverse
Chamola, V., Bansal, G., Das, T. K., Hassija, V., Sai, S., Wang, J., Zeadally, S., Hussain, A., Yu, F. R., Guizani, M., et al. (2024) · IEEE Internet of Things Magazine · doi:10.1109/iotm.001.2300174 · Cited by 108 · see citing works
EditGuard: Versatile Image Watermarking for Tamper Localization and Copyright Protection
Zhang, X., Li, R., Yu, J., Xu, Y., Li, W., & Zhang, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01137 · Cited by 108 · see citing works
WaterCycleDiffusion: Visual–textual fusion empowered underwater image enhancement
Wang, H., Zhang, W., Xu, Y., Li, H., & Ren, P. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103693 · Cited by 108 · see citing works
Expressive 3D Facial Animation Generation Based on Local-to-Global Latent Diffusion
Song, W., Wang, X., Jiang, Y., Li, S., Hao, A., Hou, X., & Qin, H. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3456213 · Cited by 107 · see citing works
Disco: Disentangled Control for Realistic Human Dance Generation
Wang, T., Li, L., Lin, K., Zhai, Y., Lin, C.-C., Yang, Z., Zhang, H., Liu, Z., & Wang, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00891 · Cited by 107 · see citing works
A Survey on Diffusion Models for Time Series and Spatio-Temporal Data
Yang, Y., Jin, M., Wen, H., Zhang, C., Liang, Y., Ma, L., Wang, Y., Liu, C., Yang, B., Xu, Z., et al. (2025) · ACM Computing Surveys · doi:10.1145/3783986 · Cited by 107 · see citing works
BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion
Ju, X., Liu, X., Wang, X., Bian, Y., Shan, Y., & Xu, Q. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72661-3_9 · Cited by 107 · see citing works
Face Generation and Editing With StyleGAN: A Survey
Melnik, A., Miasayedzenkau, M., Makaravets, D., Pirshtuk, D., Akbulut, E., Holzmann, D., Renusch, T., Reichert, G., & Ritter, H. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3350004 · Cited by 105 · see citing works
LayoutLLM-T2I: Eliciting Layout Guidance from LLM for Text-to-Image Generation
Qu, L., Wu, S., Fei, H., Nie, L., & Chua, T. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612012 · Cited by 104 · see citing works
RoSteALS: Robust Steganography using Autoencoder Latent Space
Bui, T., Agarwal, S., Yu, N., & Collomosse, J. P. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00100 · Cited by 103 · see citing works
Taming the Power of Diffusion Models for High-Quality Virtual Try-On with Appearance Flow
Gou, J., Sun, S., Zhang, J., Si, J., Qian, C., & Zhang, L. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612255 · Cited by 102 · see citing works
Self-citationPromptPaint: Steering Text-to-Image Generation Through Paint Medium-like Interactions
Chung, J. J. Y., & Adar, E. (2023) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3586183.3606777 · Cited by 101 · see citing works
Improving Diffusion Models for Authentic Virtual Try-on in the Wild
Choi, Y., Kwak, S., Lee, K., Choi, H., & Shin, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73016-0_13 · Cited by 102 · see citing works
Localizing Object-level Shape Variations with Text-to-Image Diffusion Models
Patashnik, O., Garibi, D., Azuri, I., Averbuch‐Elor, H., & Cohen‐Or, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02107 · Cited by 100 · see citing works
- Authors
- Zhang, L., Rao, A., & Agrawala, M.
- Affiliations
- Stanford University (US)
- Venue
- 2023 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2023-10-01
- Pages
- 3813-3824
- OpenAlex
- W4390873054
- Cited by
- 4,547
- References
- 91
- Citations by year
- 2026: 1290 · 2025: 1847 · 2024: 1255 · 2023: 155
- FWCI
- 486.28
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Funders
- Brown Institute for Media Innovation
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- ControlNet, text-to-image diffusion models, spatial conditioning, conditional image generation, Stable Diffusion, parameter-efficient fine-tuning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review