Cited by 4,547Generative Adversarial Networks and Image SynthesisAdvanced Neural Network ApplicationsMultimodal Machine Learning Applications

The DOI matches its Crossref record; 88 of the 91 references it cites were found in a registry or an index.

Adding Conditional Control to Text-to-Image Diffusion Models

Zhang, Lvmin, Rao, Anyi, Agrawala, Maneesh·2023·2023 IEEE/CVF International Conference on Computer Vision (ICCV)

DOI 10.1109/iccv51070.2023.00355

Open this paper in CoChat — free account

Abstract

We present ControlNet, a neural network architecture to add spatial conditioning controls to large, pretrained text-to-image diffusion models. ControlNet locks the production-ready large diffusion models, and reuses their deep and robust encoding layers pretrained with billions of images as a strong backbone to learn a diverse set of conditional controls. The neural architecture is connected with "zero convolutions" (zero-initialized convolution layers) that progressively grow the parameters from zero and ensure that no harmful noise could affect the finetuning. We test various conditioning controls, e.g., edges, depth, segmentation, human pose, etc., with Stable Diffusion, using single or multiple conditions, with or without prompts. We show that the training of ControlNets is robust with small (1m) datasets. Extensive results show that ControlNet may facilitate wider applications to control image diffusion models.

Integrity signals

Registry

Crossref · 10.1109/iccv51070.2023.00355

Cites

88 of 88 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Deep learning
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
A Computational Approach to Edge Detection
Canny, J. (1986) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1986.4767851 · Cited by 30,004
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881

Show all 91 references

Research works printed but not linked: 13

2022, Disco diffusion
2022, Protogen x3.4 (photorealism) official release
Ha, Hypernetworks
2022, Hypernetwork style training, a tiny guide, stable-diffusion-webui
10.1109/TPAMI.2020.2970919 · DOI · Paper page
2022, Novelai improvements on stable diffusion
Lehtinen, Noise2noise: Learning image restoration without clean data
2023, Midjourney
2022, Comic-diffusion v2, trained on 6 styles at once
2023, Dall-e-2
Schuhmann, LAION-5b: An open large-scale dataset for training next generation image-text models
2022, Stable diffusion v1.5 model card
2022, Stable diffusion v2 model card, stable-diffusion-2-depth

Sorted by citations

Items 301–400 of 1,012

Pixel super-resolved virtual staining of label-free tissue using diffusion models
Zhang, Y., Huang, L., Pillar, N., Li, Y., Chen, H., & Özcan, A. (2025) · Nature Communications · doi:10.1038/s41467-025-60387-z · Cited by 46
An Overview of Image Generation of Industrial Surface Defects
Zhong, X., Zhu, J., Liu, W., Hu, C., Deng, Y., & Wu, Z. (2023) · Sensors · doi:10.3390/s23198160 · Cited by 44
Training-free Content Injection using h-space in Diffusion Models
Jeong, J., Kwon, M., & Uh, Y. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00507 · Cited by 60
Creative interior design matching the indoor structure generated through diffusion model with an improved control network
Chen, J., Zheng, X., Shao, Z., Ruan, M., Li, H., Dong, Z., & Liang, Y. (2024) · Frontiers of Architectural Research · doi:10.1016/j.foar.2024.08.003 · Cited by 48
EmoGen: Emotional Image Content Generation with Text-to-Image Diffusion Models
Yang, J., Feng, J., & Huang, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00608 · Cited by 54
ConsistNet: Enforcing 3D Consistency for Multi-View Images Diffusion
Yang, J., Cheng, Z., Duan, Y., Ji, P., & Li, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00676 · Cited by 56
DDPM-CD: Denoising Diffusion Probabilistic Models as Feature Extractors for Remote Sensing Change Detection
Bandara, W. G. C., Nair, N. G., & Patel, V. M. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00513 · Cited by 54
Synthetic Scientific Image Generation with VAE, GAN, and Diffusion Model Architectures
Sordo, Z., Chagnon, E., Hu, Z., Donatelli, J. J., Andeer, P. F., Nico, P., Northen, T. R., & Ushizima, D. M. (2025) · Journal of Imaging · doi:10.3390/jimaging11080252 · Cited by 51
Ctrl-Room: Controllable Text-to-3D Room Meshes Generation with Layout Constraints
Fang, C., Dong, Y., Luo, K., Hu, X., Shrestha, R., & Tan, P. (2025) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv66043.2025.00069 · Cited by 41
Diffusion-Based Visual Art Creation: A Survey and New Perspectives
Wang, B., Chen, Q., & Wang, Z. (2025) · ACM Computing Surveys · doi:10.1145/3728459 · Cited by 40
Diverse Hyperspectral Remote Sensing Image Synthesis With Diffusion Models
Liu, L., Chen, B., Chen, H., Zou, Z., & Shi, Z. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3335975 · Cited by 47
Diffusion-Guided Reconstruction of Everyday Hand-Object Interaction Clips
Ye, Y., Hebbar, P., Gupta, A., & Tulsiani, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01806 · Cited by 49
Weed database development: An updated survey of public weed datasets and cross-season weed detection adaptation
Deng, B., Lu, Y., & Xu, J. (2024) · Ecological Informatics · doi:10.1016/j.ecoinf.2024.102546 · Cited by 40
Inversion-Free Image Editing with Language-Guided Diffusion Models
Xu, S., Huang, Y., Pan, J., Ma, Z., & Chai, J. Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00903 · Cited by 100
AIGC Image Quality Assessment via Image-Prompt Correspondence
Peng, F., Fu, H., Ming, A., Wang, C., Ma, H., He, S., Dou, Z., & Chen, S. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00644 · Cited by 40
DGInStyle: Domain-Generalizable Semantic Segmentation with Image Diffusion Models and Stylized Semantic Control
Jia, Y., Hoyer, L., Huang, S., Wang, T., Gool, L. V., Konrad Schindler, B., & Obukhov, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72933-1_6 · Cited by 48
An Investigation on Utilizing Large Language Model for Industrial Computer-Aided Design Automation
Deng, H., Khan, S., & Erkoyuncu, J. A. (2024) · Procedia CIRP · doi:10.1016/j.procir.2024.07.049 · Cited by 44
Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generation
Xue, J., Deng, Y., Gao, Y., & Li, Y. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3485485 · Cited by 71
ToonCrafter: Generative Cartoon Interpolation
Xing, J., Liu, H., Xia, M., Zhang, Y., Wang, X., Shan, Y., & Wong, T. (2024) · ACM Transactions on Graphics · doi:10.1145/3687761 · Cited by 40
I-Design: Personalized LLM Interior Designer
Çelen, A., Guo, H., Schindler, K., Gool, L. V., Armeni, I., Obukhov, A., & Wang, X. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-92387-6_17 · Cited by 40
3D MedDiffusion: A 3D Medical Latent Diffusion Model for Controllable and High-Quality Medical Image Generation
Wang, H., Liu, Z. Q., Sun, K., Wang, X., Shen, D. G., & Cui, Z. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3585372 · Cited by 55
Image Augmentation with Controlled Diffusion for Weakly-Supervised Semantic Segmentation
Wu, W., Dai, T., Huang, X., Ma, F., & Xiao, J. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447893 · Cited by 44
Taming Stable Diffusion for Text to 360° Panorama Image Generation
Zhang, C., Wu, Q., Gambardella, C. C., Huang, X., Phung, D., Ouyang, W., & Cai, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00607 · Cited by 61
MatFuse: Controllable Material Generation with Diffusion Models
Vecchio, G., Sortino, R., Palazzo, S., & Spampinato, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00424 · Cited by 50
Texture-Preserving Diffusion Models for High-Fidelity Virtual Try-On
Yang, X., Ding, C., Hong, Z., Huang, J., Jin, T., & Xu, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00670 · Cited by 43
DGE: Direct Gaussian 3D Editing by Consistent Multi-view Editing
Chen, M., Laina, I., & Vedaldi, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72904-1_5 · Cited by 39
Control Color: Multimodal Diffusion-Based Interactive Image Colorization
Liang, Z., Li, Z., Zhou, S., Li, Y., & Loy, C. C. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02549-6 · Cited by 61
Exploring One-Shot Semi-supervised Federated Learning with Pre-trained Diffusion Models
Yang, M., Su, S., Li, B., & Xue, X. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i15.29568 · Cited by 53
RoomDreaming: Generative-AI Approach to Facilitating Iterative, Preliminary Interior Design Exploration
Wang, S.-Y., Su, W. L., Chen, S., Tsai, C.-Y., Misztal, M., Cheng, K. M., Lin, A., Chen, Y., & Chen, M. Y. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642901 · Cited by 48
Towards Realistic Scene Generation with LiDAR Diffusion Models
Ran, H., Guizilini, V., & Wang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01396 · Cited by 52
OpenBias: Open-Set Bias Detection in Text-to-Image Generative Models
Moreno D’Incà, Peruzzo, E., Mancini, M., Xu, D., Goe, V., Xu, X., Wang, Z., Shi, H., & Sebe, N. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01162 · Cited by 54
SMooDi: Stylized Motion Diffusion Model
Zhong, L., Xie, Y., Jampani, V., Sun, D., & Jiang, H. W. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73232-4_23 · Cited by 38
DINO-Tracker: Taming DINO for Self-supervised Point Tracking in a Single Video
Tumanyan, N., Singer, A., Bagon, S., & Dekel, T. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73347-5_21 · Cited by 38
Diffusion Models for 3D Generation: A Survey
Wang, C., Peng, H.-Y., Liu, Y.-T., Gu, J., & Hu, S. (2025) · Computational Visual Media · doi:10.26599/cvm.2025.9450452 · Cited by 55
Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Control
Gu, Z., Yan, R., Lu, J., Li, P., Dou, Z., Si, C., Dong, Z., Liu, Q., Lin, C. C., Liu, Z., et al. (2025) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3721238.3730607 · Cited by 113
RoomDreamer: Text-Driven 3D Indoor Scene Synthesis with Coherent Geometry and Texture
Song, L., Cao, L., Xu, H., Kang, K., Tang, F., Yuan, J., & Yang, Z. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611800 · Cited by 70
DreamStyler: Paint by Style Inversion with Text-to-Image Diffusion Models
Ahn, N., Lee, J., Lee, C., Kim, K., Kim, D., Nam, S.-H., & Hong, K. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i2.27824 · Cited by 37
TypeDance: Creating Semantic Typographic Logos from Image through Personalized Generation
Xiao, S., Wang, L., Ma, X., & Zeng, W. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642185 · Cited by 45
Enhancing interaction in virtual-real architectural environments: A comparative analysis of generative AI-driven reality approaches
Chen, X., Gao, W., Chu, Y., & Song, Y. (2024) · Building and Environment · doi:10.1016/j.buildenv.2024.112113 · Cited by 37
ControlRoom3D: Room Generation Using Semantic Proxy Rooms
Schult, J., Tsai, S. S., Höllein, L., Wu, B., Wang, J., Ma, C., Li, K., Wang, X., Wimbauer, F., He, Z., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00593 · Cited by 58
DiffusionLight: Light Probes for Free by Painting a Chrome Ball
Phongthawee, P., Chinchuthakun, W., Sinsunthithet, N., Jampani, V., Raj, A., Khungurn, P., & Suwajanakorn, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00018 · Cited by 50
MotionEditor: Editing Video Motion via Content-Aware Diffusion
Tu, S., Dai, Q., Cheng, Z.-Q., Hu, H., Han, X., Wu, Z., & Jiang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00753 · Cited by 52
Diffusion-based Blind Text Image Super-Resolution
Zhang, Y., Zhang, J., Li, H., Wang, Z., Hou, L., Zou, D., & Bian, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02440 · Cited by 42
Face-Adapter for Pre-trained Diffusion Models with Fine-Grained ID and Attribute Control
Han, Y., Zhu, J., He, K., Chen, X., Ge, Y., Li, W., Li, X., Zhang, J., Wang, C., & Liu, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72973-7_2 · Cited by 37
SpaDiT: diffusion transformer for spatial gene expression prediction using scRNA-seq
Li, X., Zhu, F., & Min, W. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae571 · Cited by 43
Inkspire: Supporting Design Exploration with Generative AI through Analogical Sketching
Lin, D. C.-E., Kang, H. B., Martelaro, N., Kittur, A., Chen, Y.-Y., & Hong, M. K. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713397 · Cited by 37
A comprehensive overview of Generative AI (GAI): Technologies, applications, and challenges
Zhang, Z., Zhang, J., Zhang, X., & Mai, W. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129645 · Cited by 45
AI-assisted architectural design studio (AI-a-ADS): How artificial intelligence join the architectural design studio?
Özorhon, G., Gelirli, D. N., Lekesiz, G., & Müezzinoğlu, C. (2025) · International Journal of Technology and Design Education · doi:10.1007/s10798-025-09975-0 · Cited by 46
Generative Semantic Communications With Foundation Models: Perception-Error Analysis and Semantic-Aware Power Allocation
Xu, C., Mashhadi, M. B., Ma, Y., Tafazolli, R., & Wang, J. (2025) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2025.3559120 · Cited by 50
DifIISR: A Diffusion Model with Gradient Guidance for Infrared Image Super-Resolution
Li, X. R., Wang, Z., Zou, Y., Chen, Z., Ma, J., Jiang, Z., Ma, L., & Liu, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00706 · Cited by 37
LoRA-Composer: Leveraging Low-Rank Adaptation for Multi-Concept Customization in Training-Free Diffusion Models
Yang, Y., Wen, W. Y., Peng, L., Song, C., Chen, Y., Li, H., Yang, X., Lu, Q., Cai, D., He, X., et al. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3633153 · Cited by 37
Organizational preparedness for the use of large language models in pathology informatics
Hart, S. N., Hoffman, N. G., Gershkovich, P., Christenson, C., McClintock, D. S., Miller, L. J., Jackups, R., Azimi, V., Spies, N. C., & Brodsky, V. (2023) · Journal of Pathology Informatics · doi:10.1016/j.jpi.2023.100338 · Cited by 38
Harnessing the Spatial-Temporal Attention of Diffusion Models for High-Fidelity Text-to-Image Synthesis
Wu, Q., Liu, Y., Zhao, H., Bui, T., Lin, Z., Zhang, Y., & Chang, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00714 · Cited by 40
Robust Text Image Tampering Localization via Forgery Traces Enhancement and Multiscale Attention
Dong, L., Liang, W., & Wang, R. (2024) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2024.3367947 · Cited by 37
DiffCLIP: Leveraging Stable Diffusion for Language Grounded 3D Classification
Shen, S., Zhu, Z., Fan, L., Zhang, H., & Wu, X. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00356 · Cited by 39
LOOSECONTROL: Lifting ControlNet for Generalized Depth Conditioning
Bhat, S. F., Mitra, N. J., & Wonka, P. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657525 · Cited by 36
RealtimeGen: An Intervenable AI Image Generation System for Commercial Digital Art Asset Creators
Li, Z., Zhang, Y., Zhou, S., Liu, Q., Zhang, J., Xu, H., Chen, S., Chen, X., & Sun, L. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2382508 · Cited by 39
HAIGEN: Towards Human-AI Collaboration for Facilitating Creativity and Style Generation in Fashion Design
Jiang, J., Wu, D., Deng, H., Long, Y., Tang, W., Li, X., Liu, C., Jin, Z., Zhang, W., & Qi, T. (2024) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3678518 · Cited by 36
360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Model
Wang, Q., Li, W., Mou, C., Cheng, X., & Zhang, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00660 · Cited by 48
Focus on Your Instruction: Fine-grained and Multi-instruction Image Editing by Attention Modulation
Guo, Q., & Lin, T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00667 · Cited by 52
DreamControl: Control-Based Text-to-3D Generation with 3D Self-Prior
Huang, T., Zeng, Y., Zhang, Z., Xu, W., Xu, H., Xu, S., Lau, R. W. H., & Zuo, W. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00513 · Cited by 47
Learning Transferable Negative Prompts for Out-of-Distribution Detection
Li, T., Pang, G., Bai, X. P., Miao, W., & Zheng, J. G. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01665 · Cited by 50
Generative and Malleable User Interfaces with Generative and Evolving Task-Driven Data Model
Cao, Y., Jiang, P., & Xia, H. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713285 · Cited by 92
Vision-Language Modeling Meets Remote Sensing: Models, datasets, and perspectives
Weng, X., Pang, C., & Xia, G.-S. (2025) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2025.3572702 · Cited by 47
Attention Distillation: A Unified Approach to Visual Characteristics Transfer
Zhou, Y., Gao, X., Chen, Z., & Huang, H. F. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01703 · Cited by 36
BiaS: Incorporating Biased Knowledge to Boost Unsupervised Image Anomaly Localization
Cao, Y., Xu, X., Sun, C., Gao, L., & Shen, W. (2024) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2023.3344383 · Cited by 35
AnimeDiffusion: Anime Diffusion Colorization
Cao, Y., Meng, X., Mok, P. Y., Lee, T., Liu, X., & Li, P. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3357568 · Cited by 40
Recent Trends in 3D Reconstruction of General Non‐Rigid Scenes
Yunus, R., Lenssen, J. E., Niemeyer, M., Liao, Y., Rupprecht, C., Theobalt, C., Pons‐Moll, G., Huang, J., Golyanik, V., & Ilg, E. (2024) · Computer Graphics Forum · doi:10.1111/cgf.15062 · Cited by 38
Exploring artificial intelligence generated content (AIGC) applications in the metaverse: Challenges, solutions, and future directions
Wang, X., Hong, Y., & He, X. (2024) · IET Blockchain · doi:10.1049/blc2.12076 · Cited by 38
SketchDream: Sketch-based Text-To-3D Generation and Editing
Liu, F., Fu, H., Lai, Y., & Gao, L. (2024) · ACM Transactions on Graphics · doi:10.1145/3658120 · Cited by 35
RAVE: Randomized Noise Shuffling for Fast and Consistent Video Editing with Diffusion Models
Kara, Ö., Kurtkaya, B., Yesiltepe, H., Rehg, J. M., & Yanardag, P. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00622 · Cited by 63
Control4D: Efficient 4D Portrait Editing With Text
Shao, R., Sun, J., Cheng, P., Zheng, Z., Zhou, B., Zhang, H., & Liu, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00436 · Cited by 65
VCoder: Versatile Vision Encoders for Multimodal Large Language Models
Jain, J., Yang, J., & Shi, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02644 · Cited by 53
GPT4Motion: Scripting Physical Motions in Text-to-Video Generation via Blender-Oriented GPT Planning
Lv, J., Huang, Y., Yan, M., Huang, J., Liu, J., Liu, Y., Wen, Y., Chen, X., & Chen, S. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00150 · Cited by 48
A Survey on Multimodal Large Language Models in Radiology for Report Generation and Visual Question Answering
Yi, Z., Xiao, T., & Albert, M. V. (2025) · Information · doi:10.3390/info16020136 · Cited by 43
DreamArtist: Controllable One-Shot Text-to-Image Generation via Positive-Negative Adapter
Dong, Z., Wei, P., & Lin, L. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02526-z · Cited by 35
XCreation: A Graph-based Crossmodal Generative Creativity Support Tool
Yan, Z., Yang, C., Liang, Q., & Chen, X. (2023) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3586183.3606826 · Cited by 42
TransC-GD-CD: Transformer-Based Conditional Generative Diffusion Change Detection Model
Wen, Y., Zhang, Z., Cao, Q., & Niu, G. (2024) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2024.3373201 · Cited by 34
CharacterGen: Efficient 3D Character Generation from Single Images with Multi-View Pose Canonicalization
Peng, H.-Y., Zhang, J.-P., Guo, M.-H., Cao, Y., & Hu, S. (2024) · ACM Transactions on Graphics · doi:10.1145/3658217 · Cited by 43
Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generation
Li, H., Shen, C., Torr, P. H. S., Volker Tresp, C., & Gu, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01141 · Cited by 56
Relightful Harmonization: Lighting-Aware Portrait Background Replacement
Ren, M., Xiong, W., Yoon, J. S., Shu, Z., Zhang, J., Jung, H., Gerig, G., & Zhang, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00617 · Cited by 52
DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatars
Kirschstein, T., Giebenhain, S., & Nießner, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00524 · Cited by 46
LaWa: Using Latent Space for In-Generation Image Watermarking
Rezaei, A., Akbari, M., Alvar, S. R., Fatemi, A., & Zhang, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73024-5_8 · Cited by 44
SignDiff: Diffusion Model for American Sign Language Production
Fang, S., Sui, C., Zhou, Y., Zhang, X., Zhong, H., Tian, Y., & Chen, C. (2025) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg61629.2025.11099440 · Cited by 34
TSD-SR: One-Step Diffusion with Target Score Distillation for Real-World Image Super-Resolution
Dong, L., Fan, Q., Yihong, G., Wang, Z., Zhang, Q., Chen, J., Luo, Y., & Zou, C. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02158 · Cited by 34
VideoGigaGAN: Towards Detail-rich Video Super-Resolution
Xu, Y., Park, T., Zhang, R., Zhou, Y., Shechtman, E., Liu, F., Huang, J., & Liu, D. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00205 · Cited by 34
LivelySpeaker: Towards Semantic-Aware Co-Speech Gesture Generation
Zhi, Y., Cun, X., Chen, X., Shen, X., Guo, W. Q., Huang, S., & Gao, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01902 · Cited by 44
Deep Learning Technique for Human Parsing: A Survey and Outlook
Yang, L., Jia, W., Li, S., & Song, Q. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02031-9 · Cited by 42
DiffMIC-v2: Medical Image Classification via Improved Diffusion Network
Yang, Y., Fu, H., Aviles-Rivero, A. I., Xing, Z., & Zhu, L. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3530399 · Cited by 41
Deep learning applications in games: a survey from a data perspective
Hu, Z., Ding, Y., Wu, R., Li, L., Zhang, R., Hu, Y., Qiu, F., Zhang, Z., Wang, K., Zhao, S., et al. (2023) · Applied Intelligence · doi:10.1007/s10489-023-05094-2 · Cited by 33
Audio Generation with Multiple Conditional Diffusion Model
Guo, Z., Mao, J., Tao, R. M., Yan, L., Ouchi, K., Liu, H., & Wang, X. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29773 · Cited by 36
GGHead: Fast and Generalizable 3D Gaussian Heads
Kirschstein, T., Giebenhain, S., Tang, J., Georgopoulos, M., & Nießner, M. (2024) · SIGGRAPH Asia Conference Papers · doi:10.1145/3680528.3687686 · Cited by 54
I2V-Adapter: A General Image-to-Video Adapter for Diffusion Models
Guo, X., Zheng, M., Hou, L., Gao, Y., Deng, Y., Wan, P., Zhang, D., Liu, Y., Hu, W., Zha, Z.-J., et al. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657407 · Cited by 71
BlockFusion: Expandable 3D Scene Generation using Latent Tri-plane Extrapolation
Wu, Z., Li, Y., Yan, H., Shang, T., Sun, W., Wang, S., Cui, R., Liu, W., Satō, H., Li, H., et al. (2024) · ACM Transactions on Graphics · doi:10.1145/3658188 · Cited by 54
One-dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applications
Lyu, M., Yang, Y., Hong, H., Chen, H., Jin, X., He, Y., Xue, H., Han, J., & Ding, G. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00722 · Cited by 67
Diff-Plugin: Revitalizing Details for Diffusion-Based Low-Level Tasks
Liu, Y., Ke, Z., Liu, F., Zhao, N., & Lau, R. W. H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00402 · Cited by 41
ArtAdapter: Text-to-Image Style Transfer using Multi-Level Style Encoder and Explicit Adaptation
Chen, D.-Y., Tennent, H., & Hsu, C.-W. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00823 · Cited by 41
ControlNet-XS: Rethinking the Control of Text-to-Image Diffusion Models as Feedback-Control Systems
Zavadski, D., Feiden, J.-F., & Rother, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73223-2_20 · Cited by 33
Receler: Reliable Concept Erasing of Text-to-Image Diffusion Models via Lightweight Erasers
Huang, C.-P., Chang, K.-P., Tsai, C.-T., Lai, Y.-H., Yang, F.-E., & Wang, Y.-C. F. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73661-2_20 · Cited by 70
CPDM: Content-preserving diffusion model for underwater image enhancement
Shi, X., & Wang, Y. (2024) · Scientific Reports · doi:10.1038/s41598-024-82803-y · Cited by 33

Authors: Zhang, L., Rao, A., & Agrawala, M.
Affiliations: Stanford University (US)
ORCID: Zhang, Lvmin
Rao, Anyi
Agrawala, Maneesh
Venue: 2023 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher: IEEE
Published: 2023-10-01
Pages: 3813-3824
DOI: 10.1109/iccv51070.2023.00355
OpenAlex: W4390873054
Cited by: 4,547
References: 91
Citations by year: 2026: 1290 · 2025: 1847 · 2024: 1255 · 2023: 155
FWCI: 486.28
Type: conference paper
Language: en
Open access: closed
Licence: doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
Funders: Brown Institute for Media Innovation
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords: ControlNet, text-to-image diffusion models, spatial conditioning, conditional image generation, Stable Diffusion, parameter-efficient fine-tuning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Adding Conditional Control to Text-to-Image Diffusion Models

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks