The DOI matches its Crossref record; 88 of the 91 references it cites were found in a registry or an index.
Adding Conditional Control to Text-to-Image Diffusion Models
Abstract
We present ControlNet, a neural network architecture to add spatial conditioning controls to large, pretrained text-to-image diffusion models. ControlNet locks the production-ready large diffusion models, and reuses their deep and robust encoding layers pretrained with billions of images as a strong backbone to learn a diverse set of conditional controls. The neural architecture is connected with "zero convolutions" (zero-initialized convolution layers) that progressively grow the parameters from zero and ensure that no harmful noise could affect the finetuning. We test various conditioning controls, e.g., edges, depth, segmentation, human pose, etc., with Stable Diffusion, using single or multiple conditions, with or without prompts. We show that the training of ControlNets is robust with small (1m) datasets. Extensive results show that ControlNet may facilitate wider applications to control image diffusion models.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Image annotations, including prompts and spatial cues [92, 121], are essential for training and evaluating image generation models as they provide localized and semantically rich supervision that help models learn meaningful visual representations.”
From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [121]
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
A Computational Approach to Edge Detection
Canny, J. (1986) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1986.4767851 · Cited by 30,004
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881
Research works printed but not linked: 13
- 2022, Disco diffusion
- 2022, Protogen x3.4 (photorealism) official release
- Ha, Hypernetworks
- 2022, Hypernetwork style training, a tiny guide, stable-diffusion-webui
- 10.1109/TPAMI.2020.2970919 · DOI · Paper page
- 2022, Novelai improvements on stable diffusion
- Lehtinen, Noise2noise: Learning image restoration without clean data
- 2023, Midjourney
- 2022, Comic-diffusion v2, trained on 6 styles at once
- 2023, Dall-e-2
- Schuhmann, LAION-5b: An open large-scale dataset for training next generation image-text models
- 2022, Stable diffusion v1.5 model card
- 2022, Stable diffusion v2 model card, stable-diffusion-2-depth
Cited by
Sorted by citations
Diffusion Models: A Comprehensive Survey of Methods and Applications
Yang, L., Zhang, Z., Song, Y., Hong, S., Xu, R., Zhao, Y., Zhang, W., Cui, B., & Yang, M.-H. (2024) · ACM Computing Surveys · doi:10.1145/3626235 · Cited by 1,704
Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data
Yang, L., Kang, B., Huang, Z., Xu, X., Feng, J., & Zhao, H. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00987 · Cited by 1,296
Mou, C., Wang, X., Xie, L., Wu, Y., Zhang, J., Qi, Z., & Shan, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28226 · Cited by 1,477
A Survey on Generative Diffusion Models
Cao, H., Tan, C., Gao, Z., Xu, Y., Chen, G., Heng, P., & Li, S. Z. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3361474 · Cited by 834
Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation
Wu, J. Z., Ge, Y., Wang, X., Lei, S. W., Gu, Y., Shi, Y., Hsu, W., Shan, Y., Qie, X., & Shou, M. Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00701 · Cited by 799
A generative model for inorganic materials design
Zeni, C., Pinsler, R., Zügner, D., Fowler, A., Horton, M., Fu, X., Wang, Z., Shysheya, A., Crabbé, J., Ueda, S., et al. (2025) · Nature · doi:10.1038/s41586-025-08628-5 · Cited by 518
Exploiting Diffusion Prior for Real-World Image Super-Resolution
Wang, J., Yue, Z., Zhou, S., Chan, K. C. K., & Loy, C. C. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02168-7 · Cited by 608
MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing
Cao, M., Wang, X., Qi, Z., Shan, Y., Qie, X., & Zheng, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02062 · Cited by 603
Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation
Ke, B., Obukhov, A., Huang, S., Metzger, N., Daudt, R. C., & Konrad Schindler, B. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00907 · Cited by 595
Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators
Khachatryan, L., Movsisyan, A., Tadevosyan, V., Henschel, R. D., Wang, Z., Navasardyan, S., & Shi, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01462 · Cited by 620
A Survey on Multimodal Large Language Models for Autonomous Driving
Cui, C., Ma, Y., Cao, X., Ye, W., Zhou, Y., Liang, K., Chen, J., Lu, J., Yang, Z., Liao, K.-D., et al. (2024) · 2024 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw60836.2024.00106 · Cited by 369
Prompting AI Art: An Investigation into the Creative Skill of Prompt Engineering
Oppenlaender, J., Linder, R., & Silvennoinen, J. M. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2431761 · Cited by 376
Lightweight Deep Learning for Resource-Constrained Environments: A Survey
Liu, H.-I., Galindo, M., Xie, H., Wong, L.-K., Shuai, H.-H., Li, Y., & Cheng, W.-H. (2024) · ACM Computing Surveys · doi:10.1145/3657282 · Cited by 332
Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection
Wang, S., Liu, Y., Wang, T., Li, Y., & Zhang, X. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00335 · Cited by 292
Proceedings of the 32nd ACM International Conference on Multimedia
Pan, Q., Zhao, Z. W., Wang, Z., Long, S., Wu, Y., Ji, W., Liang, H., & Liang, R. (2024) · doi:10.1145/3664647 · Cited by 290
DiffBIR: Toward Blind Image Restoration with Generative Diffusion Prior
Lin, X., He, J., Chen, Z., Lyu, Z., Dai, B., Yu, F., Qiao, Y., Ouyang, W., & Dong, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73202-7_25 · Cited by 313
On the Opportunities and Challenges of Foundation Models for GeoAI (Vision Paper)
Mai, G., Huang, W., Sun, J., Song, S., Mishra, D. R., Liu, N., Gao, S., Liu, T., Cong, G., Hu, Y., et al. (2024) · ACM Transactions on Spatial Algorithms and Systems · doi:10.1145/3653070 · Cited by 384
Recent advances in 3D Gaussian splatting
Wu, T., Yuan, Y.-J., Zhang, L., Yang, J., Cao, Y., Yan, L., & Gao, L. (2024) · Computational Visual Media · doi:10.1007/s41095-024-0436-y · Cited by 315
Self-citationFateZero: Fusing Attentions for Zero-shot Text-based Video Editing
Qi, C., Cun, X., Zhang, Y., Lei, C., Wang, X., Shan, Y., & Chen, Q. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01460 · Cited by 388
The Stable Signature: Rooting Watermarks in Latent Diffusion Models
Fernandez, P., Couairon, G., Jégou, H., Douze, M., & Furon, T. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02053 · Cited by 251
Effective Whole-body Pose Estimation with Two-stages Distillation
Yang, Z., Zeng, A., Yuan, C., & Li, Y. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00455 · Cited by 273
A Diffusion-Based Framework for Multi-Class Anomaly Detection
He, H., Zhang, J., Chen, H., Chen, X., Li, Z., Chen, X., Wang, Y., Wang, C., & Xie, L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i8.28690 · Cited by 271
Multimodal Foundation Models: From Specialists to General-Purpose Assistants
Li, C., Gan, Z., Yang, Z., Yang, J., Li, L., Wang, L., & Gao, J. (2024) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000110 · Cited by 331
SeeSR: Towards Semantics-Aware Real-World Image Super-Resolution
Wu, R., Yang, T., Sun, L., Zhang, Z., Li, S., & Zhang, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02405 · Cited by 279
SVDiff: Compact Parameter Space for Diffusion Fine-Tuning
Han, L., Li, Y., Zhang, H., Milanfar, P., Metaxas, D., & Yang, F. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00673 · Cited by 302
LaVie: High-Quality Video Generation with Cascaded Latent Diffusion Models
Wang, Y., Chen, X., Ma, X., Zhou, S., Huang, Z., Yang, C., Yang, C., He, Y., Yu, J., Yang, P., et al. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02295-1 · Cited by 436
ECON: Explicit Clothed humans Optimized via Normal integration
Xiu, Y., Yang, J., Cao, X., Tzionas, D., & Black, M. J. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00057 · Cited by 241
Brade, S., Wang, B., Sousa, M., Oore, S., & Grossman, T. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606725 · Cited by 198
Generative AI models for different steps in architectural design: A literature review
Li, C., Zhang, T., Du, X., Zhang, Y., & Xie, H. (2024) · Frontiers of Architectural Research · doi:10.1016/j.foar.2024.10.001 · Cited by 236
InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuning
Shi, J., Xiong, W., Lin, Z., & Jung, H. J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00816 · Cited by 303
FastComposer: Tuning-Free Multi-subject Image Generation with Localized Attention
Xiao, G., Yin, T., Freeman, W. T., Durand, F., & Han, S. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02227-z · Cited by 297
Parameter-efficient fine-tuning in large language models: a survey of methodologies
Wang, L., Chen, S., Jiang, L., Pan, S., Cai, R., Yang, S., & Yang, F. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11236-4 · Cited by 186
Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation
Li, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00779 · Cited by 361
DriveDreamer: Towards Real-World-Drive World Models for Autonomous Driving
Wang, X., Zhu, Z. H., Huang, G., Chen, X., Zhu, J., & Lu, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73195-2_4 · Cited by 365
MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model
Xu, Z., Zhang, J., Liew, J. H., Yan, H., Liu, J., Zhang, C., Feng, J., & Shou, M. Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00147 · Cited by 262
Training-Free Layout Control with Cross-Attention Guidance
Chen, M., Laina, I., & Vedaldi, A. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00526 · Cited by 248
Text2Tex: Text-driven Texture Synthesis via Diffusion Models
Chen, D. Z., Siddiqui, Y., Lee, H.-Y., Tulyakov, S., & Nießner, M. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01701 · Cited by 247
All-in-One Weather-Degraded Image Restoration Via Adaptive Degradation-Aware Self-Prompting Model
Wen, Y., Gao, T., Li, Z., Zhang, J., Zhang, K., & Chen, T. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3535316 · Cited by 195
Pix2Video: Video Editing using Image Diffusion
Ceylan, D., Huang, C.-H. P., & Mitra, N. J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02121 · Cited by 234
BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusion
Xie, J., Li, Y., Huang, Y., Liu, H., Zhang, W., Zheng, Y., & Shou, M. Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00685 · Cited by 218
CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets
Zhang, L., Wang, Z., Zhang, Q., Qiu, Q., Pang, A., Jiang, H., Yang, W., Xu, L., & Jingyi Yu, A. (2024) · ACM Transactions on Graphics · doi:10.1145/3658146 · Cited by 173
Self-citationFrom Matching to Generation: A Survey on Generative Information Retrieval
Li, X., Jin, J., Zhou, Y., Zhang, Y., Zhang, P., Zhu, Y., & Dou, Z. (2025) · ACM Transactions on Information Systems · doi:10.1145/3722552 · Cited by 172
Semantic Image Synthesis via Diffusion Models
Zhou, W., Wang, W., Bao, J., Chen, D., Chen, D., Yuan, L., & Li, H. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3680594 · Cited by 219
Diffusion Models for Image Restoration and Enhancement: A Comprehensive Survey
Li, X., Ren, Y., Jin, X., Lan, C., Wang, X., Zeng, W., Wang, X., & Chen, Z. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02570-9 · Cited by 219
Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models
Höllein, L., Cao, A., Owens, A., Johnson, J., & Nießner, M. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00727 · Cited by 167
DreamBooth3D: Subject-Driven Text-to-3D Generation
Raj, A., Kaza, S., Poole, B., Niemeyer, M., Ruiz, N., Mildenhall, B., Zada, S., Aberman, K., Rubinstein, M., Barron, J. T., et al. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00223 · Cited by 167
Diffusion Model-Based Image Editing: A Survey
Huang, Y., Huang, J., Liu, Y., Yan, M., Lv, J., Liu, J., Xiong, W., Zhang, H., Cao, L., & Chen, S. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3541625 · Cited by 278
Evaluating the subjective perceptions of streetscapes using street-view images
Ogawa, Y., Oki, T., Zhao, C., Sekimoto, Y., & Shimizu, C. (2024) · Landscape and Urban Planning · doi:10.1016/j.landurbplan.2024.105073 · Cited by 165
Multimodal Image Synthesis and Editing: The Generative AI Era
Zhan, F., Yu, Y., Wu, R., Zhang, J., Lu, S., Liu, L., Kortylewski, A., Theobalt, C., & Xing, E. P. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3305243 · Cited by 197
PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding
Li, Z., Cao, M., Wang, X., Qi, Z., Cheng, M., & Shan, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00825 · Cited by 233
Lumiere: A Space-Time Diffusion Model for Video Generation
Bar-Tal, O., Chefer, H., Tov, O., Herrmann, C., Paiss, R., Zada, S., Ephrat, A., Hur, J., Liu, G., Raj, A., et al. (2024) · SIGGRAPH Asia Conference Papers · doi:10.1145/3680528.3687614 · Cited by 215
A Survey on Video Diffusion Models
Xing, Z., Feng, Q., Chen, H., Dai, Q., Hu, H., Xu, H., Wu, Z., & Jiang, Y. (2024) · ACM Computing Surveys · doi:10.1145/3696415 · Cited by 228
FreeU: Free Lunch in Diffusion U-Net
Si, C., Huang, Z., Jiang, Y., & Liu, Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00453 · Cited by 202
Zero-Reference Low-Light Enhancement via Physical Quadruple Priors
Wang, W., Yang, H., Fu, J., & Liu, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02462 · Cited by 185
State of the Art on Diffusion Models for Visual Computing
Po, R., Yifan, W., Golyanik, V., Aberman, K., Barron, J. T., Bermano, A. H., Chan, E. R., Dekel, T., Holynski, A., Kanazawa, A., et al. (2024) · Computer Graphics Forum · doi:10.1111/cgf.15063 · Cited by 198
Generative-AI-Driven Human Digital Twin in IoT Healthcare: A Comprehensive Survey
Chen, J., Shi, Y., Yi, C., Du, H., Kang, J., & Niyato, D. T. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3421918 · Cited by 168
HyperDreamBooth: HyperNetworks for Fast Personalization of Text-to-Image Models
Ruiz, N., Li, Y., Jampani, V., Wei, W., Hou, T., Pritch, Y., Wadhwa, N., Rubinstein, M., & Aberman, K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00624 · Cited by 155
Style Aligned Image Generation via Shared Attention
Hertz, A., Voynov, A. S., Fruchter, S., & Cohen‐Or, D. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00457 · Cited by 204
Follow Your Pose: Pose-Guided Text-to-Video Generation Using Pose-Free Videos
Ma, Y., He, Y., Cun, X., Wang, X., Chen, S., Li, X., & Chen, Q. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28206 · Cited by 222
MotionCtrl: A Unified and Flexible Motion Controller for Video Generation
Wang, Z., Yuan, Z., Wang, X., Li, Y., Chen, T., Xia, M., Luo, P., & Shan, Y. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657518 · Cited by 280
Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation
Yang, S., Zhou, Y., Liu, Z., & Loy, C. C. (2023) · SIGGRAPH Asia Conference Papers · doi:10.1145/3610548.3618160 · Cited by 208
Versatile Diffusion: Text, Images and Variations All in One Diffusion Model
Xu, X., Wang, Z., Zhang, E., Wang, K., & Shi, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00713 · Cited by 212
Ablating Concepts in Text-to-Image Diffusion Models
Kumari, N., Zhang, B., Wang, S.-Y., Shechtman, E., Zhang, R., & Zhu, J.-Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02074 · Cited by 207
Generative AI for visualization: State of the art and future directions
Ye, Y., Hao, J., Hou, Y., Wang, Z., Xiao, S., Luo, Y., & Zeng, W. (2024) · Visual Informatics · doi:10.1016/j.visinf.2024.04.003 · Cited by 167
LLMR: Real-time Prompting of Interactive Worlds using Large Language Models
Torre, F. D. L., Fang, C. M., Huang, H., Banburski, A., Amores, J., & Lanier, J. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642579 · Cited by 145
LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching
Liang, Y., Yang, X., Lin, J., Li, H., Xu, X., & Chen, Y.-C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00623 · Cited by 244
Pixel-Aware Stable Diffusion for Realistic Image Super-Resolution and Personalized Stylization
Yang, T., Wu, R., Ren, P., Xie, X., & Zhang, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73247-8_5 · Cited by 159
3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities
Bao, Y., Ding, T., Huo, J., Liu, Y., Li, Y., Li, W., Gao, Y., & Luo, J. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3538684 · Cited by 187
A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)
Deldjoo, Y., He, Z., McAuley, J., Korikov, A., Sanner, S., Arnau Ramisa, A., Vidal, R., Sathiamoorthy, M., Kasirzadeh, A., & Milano, S. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671474 · Cited by 177
PromptMagician: Interactive Prompt Engineering for Text-to-Image Creation
Feng, Y., Wang, X., Wong, K. K., Wang, S., Lu, Y., Zhu, M., Wang, B., & Chen, W. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3327168 · Cited by 124
A comprehensive survey on deep active learning in medical image analysis
Wang, H., Jin, Q., Li, S., Liu, S., Wang, M., & Song, Z. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103201 · Cited by 157
Sapiens: Foundation for Human Vision Models
Khirodkar, R., Bagautdinov, T., Martinez, J., Zhaoen, S., James, A., Selednik, P., Anderson, S., & Saito, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73235-5_12 · Cited by 213
Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models
Ge, S., Nah, S., Liu, G., Poon, T., Tao, A., Catanzaro, B., Jacobs, D. W., Huang, J., Liu, M.-Y., & Balaji, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02096 · Cited by 176
DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion
Karras, J., Holynski, A., Wang, T.-C., & Kemelmacher-Shlizerman, I. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02073 · Cited by 137
Advances in deep concealed scene understanding
Fan, D.-P., Ji, G.-P., Xu, P., Cheng, M., Sakaridis, C., & Gool, L. V. (2023) · Visual Intelligence · doi:10.1007/s44267-023-00019-6 · Cited by 137
Diffusion Augmentation for Sequential Recommendation
Liu, Q., Fan, Y. B., Zhao, X., Du, Z., Guo, H., Tang, R., & Tian, F. (2023) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3583780.3615134 · Cited by 147
FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model
Yu, J., Wang, Y., Zhao, C., Ghanem, B., & Zhang, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02118 · Cited by 183
DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models
Cao, Y., Cao, Y., Han, K., Shan, Y., & Wong, K.-Y. K. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00097 · Cited by 179
Stable VITON: Learning Semantic Correspondence with Latent Diffusion Model for Virtual Try-On
Kim, J., Gu, G., Park, M.-H., Park, S., & Choo, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00781 · Cited by 173
CreativeConnect: Supporting Reference Recombination for Graphic Design Ideation with Generative AI
Choi, D., Hong, S., Park, J., Chung, J. J. Y., & Kim, J. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642794 · Cited by 124
StableVideo: Text-driven Consistency-aware Diffusion Video Editing
Chai, W., Guo, X., Wang, G., & Lu, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02106 · Cited by 145
Evaluating Text-to-Visual Generation with Image-to-Text Generation
Lin, Z., Pathak, D., Li, B., Li, J., Xia, X., Neubig, G., Zhang, P., & Ramanan, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72673-6_20 · Cited by 281
Diffusion Models, Image Super-Resolution, and Everything: A Survey
Moser, B. B., Shanbhag, A. S., Raue, F., Frolov, S., Palacio, S., & Dengel, A. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3476671 · Cited by 117
On the design fundamentals of diffusion models: A survey
Chang, Z., Koulieris, G. A., Chang, H. J., & Shum, H. P. H. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111934 · Cited by 158
Deep learning for cross-domain data fusion in urban computing: Taxonomy, advances, and outlook
Zou, X., Yan, Y., Hao, X., Hu, Y., Wen, H., Liu, E., Zhang, J., Li, Y., Li, T., Zheng, Y., et al. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102606 · Cited by 127
Dense Text-to-Image Generation with Attention Modulation
Kim, Y., Lee, J., Kim, J.-H., Ha, J.-W., & Zhu, J.-Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00708 · Cited by 129
Beyond Reality: The Pivotal Role of Generative AI in the Metaverse
Chamola, V., Bansal, G., Das, T. K., Hassija, V., Sai, S., Wang, J., Zeadally, S., Hussain, A., Yu, F. R., Guizani, M., et al. (2024) · IEEE Internet of Things Magazine · doi:10.1109/iotm.001.2300174 · Cited by 108
EditGuard: Versatile Image Watermarking for Tamper Localization and Copyright Protection
Zhang, X., Li, R., Yu, J., Xu, Y., Li, W., & Zhang, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01137 · Cited by 133
WaterCycleDiffusion: Visual–textual fusion empowered underwater image enhancement
Wang, H., Zhang, W., Xu, Y., Li, H., & Ren, P. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103693 · Cited by 108
Expressive 3D Facial Animation Generation Based on Local-to-Global Latent Diffusion
Song, W., Wang, X., Jiang, Y., Li, S., Hao, A., Hou, X., & Qin, H. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3456213 · Cited by 110
Disco: Disentangled Control for Realistic Human Dance Generation
Wang, T., Li, L., Lin, K., Zhai, Y., Lin, C.-C., Yang, Z., Zhang, H., Liu, Z., & Wang, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00891 · Cited by 174
A Survey on Diffusion Models for Time Series and Spatio-Temporal Data
Yang, Y., Jin, M., Wen, H., Zhang, C., Liang, Y., Ma, L., Wang, Y., Liu, C., Yang, B., Xu, Z., et al. (2025) · ACM Computing Surveys · doi:10.1145/3783986 · Cited by 166
BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion
Ju, X., Liu, X., Wang, X., Bian, Y., Shan, Y., & Xu, Q. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72661-3_9 · Cited by 170
Face Generation and Editing With StyleGAN: A Survey
Melnik, A., Miasayedzenkau, M., Makaravets, D., Pirshtuk, D., Akbulut, E., Holzmann, D., Renusch, T., Reichert, G., & Ritter, H. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3350004 · Cited by 105
LayoutLLM-T2I: Eliciting Layout Guidance from LLM for Text-to-Image Generation
Qu, L., Wu, S., Fei, H., Nie, L., & Chua, T. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612012 · Cited by 118
RoSteALS: Robust Steganography using Autoencoder Latent Space
Bui, T., Agarwal, S., Yu, N., & Collomosse, J. P. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00100 · Cited by 103
Taming the Power of Diffusion Models for High-Quality Virtual Try-On with Appearance Flow
Gou, J., Sun, S., Zhang, J., Si, J., Qian, C., & Zhang, L. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612255 · Cited by 122
Self-citationPromptPaint: Steering Text-to-Image Generation Through Paint Medium-like Interactions
Chung, J. J. Y., & Adar, E. (2023) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3586183.3606777 · Cited by 131
Improving Diffusion Models for Authentic Virtual Try-on in the Wild
Choi, Y., Kwak, S., Lee, K., Choi, H., & Shin, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73016-0_13 · Cited by 162
Localizing Object-level Shape Variations with Text-to-Image Diffusion Models
Patashnik, O., Garibi, D., Azuri, I., Averbuch‐Elor, H., & Cohen‐Or, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02107 · Cited by 122
Metadata
- Authors
- Zhang, L., Rao, A., & Agrawala, M.
- Affiliations
- Stanford University (US)
- Venue
- 2023 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2023-10-01
- Pages
- 3813-3824
- OpenAlex
- W4390873054
- Cited by
- 4,547
- References
- 91
- Citations by year
- 2026: 1290 · 2025: 1847 · 2024: 1255 · 2023: 155
- FWCI
- 486.28
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Funders
- Brown Institute for Media Innovation
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- ControlNet, text-to-image diffusion models, spatial conditioning, conditional image generation, Stable Diffusion, parameter-efficient fine-tuning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
