Cited by 4,547Generative Adversarial Networks and Image SynthesisAdvanced Neural Network ApplicationsMultimodal Machine Learning Applications

The DOI matches its Crossref record; 88 of the 91 references it cites were found in a registry or an index.

Adding Conditional Control to Text-to-Image Diffusion Models

Zhang, Lvmin, Rao, Anyi, Agrawala, Maneesh·2023·2023 IEEE/CVF International Conference on Computer Vision (ICCV)

DOI 10.1109/iccv51070.2023.00355

Open this paper in CoChat — free account

Abstract

We present ControlNet, a neural network architecture to add spatial conditioning controls to large, pretrained text-to-image diffusion models. ControlNet locks the production-ready large diffusion models, and reuses their deep and robust encoding layers pretrained with billions of images as a strong backbone to learn a diverse set of conditional controls. The neural architecture is connected with "zero convolutions" (zero-initialized convolution layers) that progressively grow the parameters from zero and ensure that no harmful noise could affect the finetuning. We test various conditioning controls, e.g., edges, depth, segmentation, human pose, etc., with Stable Diffusion, using single or multiple conditions, with or without prompts. We show that the training of ControlNets is robust with small (1m) datasets. Extensive results show that ControlNet may facilitate wider applications to control image diffusion models.

Integrity signals

Registry

Crossref · 10.1109/iccv51070.2023.00355

Cites

88 of 88 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Deep learning
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
A Computational Approach to Edge Detection
Canny, J. (1986) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1986.4767851 · Cited by 30,004
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881

Show all 91 references

Research works printed but not linked: 13

2022, Disco diffusion
2022, Protogen x3.4 (photorealism) official release
Ha, Hypernetworks
2022, Hypernetwork style training, a tiny guide, stable-diffusion-webui
10.1109/TPAMI.2020.2970919 · DOI · Paper page
2022, Novelai improvements on stable diffusion
Lehtinen, Noise2noise: Learning image restoration without clean data
2023, Midjourney
2022, Comic-diffusion v2, trained on 6 styles at once
2023, Dall-e-2
Schuhmann, LAION-5b: An open large-scale dataset for training next generation image-text models
2022, Stable diffusion v1.5 model card
2022, Stable diffusion v2 model card, stable-diffusion-2-depth

Sorted by citations

Items 401–500 of 1,012

DyArtbank: Diverse artistic style transfer via pre-trained stable diffusion and dynamic style prompt Artbank
Zhang, Z., Zhang, Q., Li, G., Luan, J., Yang, M., Wang, Y., & Zhao, L. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.112959 · Cited by 33
Weed image augmentation by ControlNet-added stable diffusion for multi-class weed detection
Deng, B., & Lu, Y. (2025) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2025.110123 · Cited by 35
Unsupervised industrial anomaly detection with diffusion models
Xu, H., Xu, S., & Yang, W. (2023) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2023.103983 · Cited by 36
Mask2Former with Improved Query for Semantic Segmentation in Remote-Sensing Images
Guo, S., Yang, Q., Xiang, S., Wang, S., & Wang, X. (2024) · Mathematics · doi:10.3390/math12050765 · Cited by 32
Smooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Models
Guo, J., Xu, X., Pu, Y., Ni, Z., Wang, C., Vasu, M., Song, S., Huang, G. M., & Shi, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00721 · Cited by 43
Boosting Diffusion Models with Moving Average Sampling in Frequency Domain
Qian, Y., Cai, Q., Pan, Y., Li, Y., Yao, T., Sun, Q., & Mei, T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00851 · Cited by 39
Generating Human Interaction Motions in Scenes with Text Control
Yi, H., Thies, J., Black, M. J., Peng, X. B., & Rempe, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73235-5_14 · Cited by 32
EchoScene: Indoor Scene Generation via Information Echo Over Scene Graph Diffusion
Zhai, G., Örnek, E. P., Chen, D. Z., Liao, R., Di, Y., Navab, N., Tombari, F., & Busam, B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72664-4_10 · Cited by 32
360-degree Human Video Generation with 4D Diffusion Transformer
Shao, R., Pang, Y., Zheng, Z., Sun, J., & Liu, Y. (2024) · ACM Transactions on Graphics · doi:10.1145/3687980 · Cited by 67
Preserving architectural heritage in urban renewal: a stable diffusion model framework for automated historical facade generation
Kuang, Z., Zhang, J., Li, Y., & Fukuda, T. (2025) · npj Heritage Science · doi:10.1038/s40494-025-01826-4 · Cited by 37
FoleyCrafter: Bring Silent Videos to Life with Lifelike and Synchronized Sounds
Zhang, Y., Gu, Y., Zeng, Y., Xing, Z., Wang, Y. Q., Wu, Z., Liu, B., & Chen, K. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-025-02649-3 · Cited by 117
Single-Image 3D Human Digitization with Shape-guided Diffusion
AlBahar, B., Saito, S., Tseng, H.-Y., Kim, C., Kopf, J., & Huang, J. (2023) · SIGGRAPH Asia Conference Papers · doi:10.1145/3610548.3618153 · Cited by 31
Generating Images of Rare Concepts Using Pre-trained Diffusion Models
Samuel, D., Ben‐Ari, R., Raviv, S., Darshan, N., & Chechik, G. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28270 · Cited by 51
DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Models
Zhang, Y., Liu, Y., Xie, Z., Yang, L., Liu, Z., Yang, M., Zhang, R., Kou, Q., Lin, C. C., Wang, W., et al. (2024) · ACM Transactions on Graphics · doi:10.1145/3658170 · Cited by 31
It's All About Your Sketch: Democratising Sketch Control in Diffusion Models
Koley, S., Bhunia, A. K., Sekhri, D., Sain, A., Chowdhury, P. N., Xiang, T., & Song, Y.-Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00688 · Cited by 35
DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception
Wang, Y., Gao, R., Chen, K., Zhou, K., Cai, Y., Hong, L., Li, Z., Jiang, L., Yeung, D., Xu, Q., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00692 · Cited by 47
Diffusion 3D Features (Diff3F) Decorating Untextured Shapes with Distilled Semantic Features
Dutt, N. S., Muralikrishnan, S., & Mitra, N. J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00430 · Cited by 31
Visual Anagrams: Generating Multi-View Optical Illusions with Diffusion Models
Geng, D., Park, I., & Owens, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02280 · Cited by 39
EfficientDreamer: High-Fidelity and Stable 3D Creation via Orthogonal-view Diffusion Priors
Hu, Z., Zhao, M., Zhao, C., Liang, X., Li, L., Zhao, Z., Fan, C., Zhou, X., & Yu, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00473 · Cited by 31
LN3Diff: Scalable Latent Neural Fields Diffusion for Speedy 3D Generation
Lan, Y., Hong, F., Yang, S., Zhou, S., Meng, X., Dai, B., Pan, X., & Loy, C. C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73235-5_7 · Cited by 42
DragAnything: Motion Control for Anything Using Entity Representation
Wu, W., Li, Z., Gu, Y., Zhao, R., He, Y., Zhang, D. J., Shou, M. Z., Li, Y., Gao, T., & Zhang, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72670-5_19 · Cited by 61
BK-SDM: A Lightweight, Fast, and Cheap Version of Stable Diffusion
Kim, B.-K., Song, H.-K., Castells, T., & Choi, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72949-2_22 · Cited by 31
MLLM-Tool: A Multimodal Large Language Model for Tool Agent Learning
Wang, C., Luo, W., Dong, S., Xuan, X., Li, Z., Ma, L., & Gao, S. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00650 · Cited by 74
Improving AI models for rare thyroid cancer subtype by text guided diffusion models
Dai, F., Yao, S., Wang, M., Zhu, Y., Qiu, X., Sun, P., Qiu, C., Yin, J., Shen, G., Sun, J., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-59478-8 · Cited by 34
One-for-More: Continual Diffusion Model for Anomaly Detection
Li, X., Tan, X., Chen, Z., Zhang, Z., Zhang, R.-X., Guo, R., Jiang, G., Chen, Y., Qu, Y., Ma, L., et al. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00449 · Cited by 37
Peptide design through binding interface mimicry with PepMimic
(2025) · doi:10.1038/s41551-025-01507-4 · Cited by 30
Points-to-3D: Bridging the Gap between Sparse Points and Shape-Controllable Text-to-3D Generation
Yu, C., Zhou, Q., Li, J., Zhang, Z., Wang, Z., & Wang, F. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612232 · Cited by 46
Prompt learning in computer vision: a survey
Lei, Y., Li, J., Li, Z., Cao, Y., & Shan, H. (2024) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2300389 · Cited by 30
Diffusion models in text generation: a survey
Yi, Q., Chen, X., Zhang, C., Zhou, Z., Zhu, L., & Kong, X. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.1905 · Cited by 33
AltDiffusion: A Multilingual Text-to-Image Diffusion Model
Ye, F., Liu, G., Wu, X., & Wu, L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i7.28487 · Cited by 38
IntentTuner: An Interactive Framework for Integrating Human Intentions in Fine-tuning Text-to-Image Generative Models
Zeng, X., Gao, Z., Ye, Y., & Zeng, W. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642165 · Cited by 35
Stable diffusion for high-quality image reconstruction in digital rock analysis
Ma, Y., Liao, Q., Yan, Z., You, S., Song, X., Tian, S., & Li, G. (2024) · ADVANCES IN GEO-ENERGY RESEARCH · doi:10.46690/ager.2024.06.02 · Cited by 30
Knowledge-Driven and Diffusion Model-Based Methods for Generating Historical Building Facades: A Case Study of Traditional Minnan Residences in China
Xu, S., Zhang, J., & Li, Y. (2024) · Information · doi:10.3390/info15060344 · Cited by 35
Let's Think Outside the Box: Exploring Leap-of-Thought in Large Language Models with Creative Humor Generation
Zhong, S., Huang, Z., Gao, S., Wen, W., Lin, L., Žitnik, M., & Zhou, P. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01258 · Cited by 36
InteractDiffusion: Interaction Control in Text-to-Image Diffusion Models
Hoe, J. T., Jiang, X., Chan, C. S., Tan, Y., & Hu, W. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00591 · Cited by 37
FreeCustom: Tuning-Free Customized Image Generation for Multi-Concept Composition
Ding, G., Zhao, C., Wang, W., Yang, Z., Liu, Z., Chen, H., & Shen, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00868 · Cited by 37
SwiftBrush: One-Step Text-to-Image Diffusion Model with Variational Score Distillation
Nguyen, T. H., & Tran, A. N. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00746 · Cited by 53
Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion
Nunes, L., Marcuzzi, R., Mersch, B., Behley, J., & Stachniss, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01399 · Cited by 39
LightIt: Illumination Modeling and Control for Diffusion Models
Kocsis, P., Philip, J., Sunkavalli, K., Nießner, M., & Hold-Geoffroy, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00894 · Cited by 44
ControlPolypNet: Towards Controlled Colon Polyp Synthesis for Improved Polyp Segmentation
Sharma, V., Kumar, A., Jha, D., Bhuyan, M. K., Das, P. K., & Bağcı, U. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00238 · Cited by 34
Conditional Diffusion Model With Spatial-Frequency Refinement for SAR-to-Optical Image Translation
Qin, J., Wang, K. W., Zou, B., Zhang, L., & van de Weijer, J. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3491826 · Cited by 36
Self-citation
HeadStudio: Text to Animatable Head Avatars with 3D Gaussian Splatting
Zhou, Z., Ma, F., Fan, H., Yang, Z., & Yang, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73411-3_9 · Cited by 49
VideoStudio: Generating Consistent-Content and Multi-scene Videos
Long, F., Qiu, Z., Yao, T., & Mei, T. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73027-6_27 · Cited by 62
Be Yourself: Bounded Attention for Multi-subject Text-to-Image Generation
Dahary, O., Patashnik, O., Aberman, K., & Cohen‐Or, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72630-9_25 · Cited by 30
Visual Question Answering: A Survey of Methods, Datasets, Evaluation, and Challenges
Kim, B. S., Kim, J., Lee, D., & Jang, B. (2025) · ACM Computing Surveys · doi:10.1145/3728635 · Cited by 38
A Review of Deepfake and Its Detection: From Generative Adversarial Networks to Diffusion Models
Liu, B., Liu, B., Zhu, T., & Ding, M. (2025) · International Journal of Intelligent Systems · doi:10.1155/int/9987535 · Cited by 34
Quality Assessment for AI Generated Images With Instruction Tuning
Wang, J., Duan, H., Zhai, G., & Min, X. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3651009 · Cited by 30
Control3Diff: Learning Controllable 3D Diffusion Models from Single-view Images
Gu, J., Gao, Q., Zhai, S., Chen, B., Liu, L., & Susskind, J. (2024) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv62453.2024.00030 · Cited by 39
The Chosen One: Consistent Characters in Text-to-Image Diffusion Models
Avrahami, O., Hertz, A., Vinker, Y., Arar, M., Fruchter, S., Fried, O., Cohen-Or, D., & Lischinski, D. (2024) · Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers · doi:10.1145/3641519.3657430 · Cited by 29
DanXe: An extended artificial intelligence framework to analyze and promote dance heritage
Stacchio, L., Garzarella, S., Cascarano, P., Filippo, A. D., Cervellati, E., & Marfia, G. (2024) · Digital Applications in Archaeology and Cultural Heritage · doi:10.1016/j.daach.2024.e00343 · Cited by 29
Amodal Completion via Progressive Mixed Context Diffusion
Xu, K. H., Zhang, L., & Shi, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00869 · Cited by 40
Self-citation
A Recipe for Scaling up Text-to-Video Generation with Text-free Videos
Wang, X., Zhang, S., Yuan, H., Qing, Z., Gong, B., Zhang, Y., Shen, Y., Gao, C., & Sang, N. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00628 · Cited by 40
AutoSpark: Supporting Automobile Appearance Design Ideation with Kansei Engineering and Generative AI
Chen, L., Jing, Q., Tsang, Y., Wang, Q., Liu, R., Xia, D., Zhou, Y., & Sun, L. (2024) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3654777.3676337 · Cited by 34
WoVoGen: World Volume-Aware Diffusion for Controllable Multi-camera Driving Scene Generation
Lu, J., Huang, Z., Yang, Z., Zhang, J., & Zhang, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72989-8_19 · Cited by 29
Self-citation
TrailBlazer: Trajectory Control for Diffusion-Based Video Generation
Ma, W.-D. K., Lewis, J. P., & Kleijn, W. B. (2024) · SIGGRAPH Asia Conference Papers · doi:10.1145/3680528.3687652 · Cited by 29
End‐to‐End Crystal Structure Prediction from Powder X‐Ray Diffraction
Lai, Q., Xu, F., Yao, L., Gao, Z., Liu, S., Wang, H., Lu, S., He, D., Wang, L., Zhang, L., et al. (2025) · Advanced Science · doi:10.1002/advs.202410722 · Cited by 38
Self-citation
Medical image translation with deep learning: Advances, datasets and perspectives
Chen, J., Ye, Z., Zhang, R., Li, H., Fang, B., Zhang, L., & Wang, W. W. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103605 · Cited by 37
Self-citation
Underwater image enhancement by diffusion model with customized CLIP-classifier
Liu, S., Li, K., Ding, Y., & Qi, Q. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.112232 · Cited by 33
360-Degree Panorama Generation from Few Unregistered NFoV Images
Wang, J., Chen, Z., Ling, J., Xie, R., & Song, L. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612508 · Cited by 46
Alchemist: Parametric Control of Material Properties with Diffusion Models
Sharma, P., Jampani, V., Li, Y., Jia, X., Lagun, D., Durand, F., Freeman, B., & Matthews, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02278 · Cited by 38
Minimal data requirement for realistic endoscopic image generation with Stable Diffusion
Kaleta, J., Dall’Alba, D., Płotka, S., & Korzeniowski, P. (2023) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-023-03030-w · Cited by 28
Brush Your Text: Synthesize Any Scene Text on Images via Diffusion Model
Zhang, L., Chen, X., Wang, Y., Lu, Y., & Qiao, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i7.28550 · Cited by 36
Self-citation
Collage Diffusion
Sarukkai, V., Li, L., Ma, A., Ré, C., & Fatahalian, K. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00416 · Cited by 43
Gen2Sim: Scaling up Robot Learning in Simulation with Generative Models
Katara, P., Xian, Z., & Fragkiadaki, K. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610566 · Cited by 45
Ranni: Taming Text-to-Image Diffusion for Accurate Instruction Following
Feng, Y., Gong, B., Chen, D., Shen, Y., Liu, Y., & Zhou, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00454 · Cited by 54
Hierarchical Spatio-temporal Decoupling for Text-to- Video Generation
Qing, Z., Zhang, S., Wang, J., Wang, X., Wei, Y., Zhang, Y., Gao, C., & Sang, N. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00634 · Cited by 43
Text-to-Image Diffusion Models are Great Sketch-Photo Matchmakers
Koley, S., Bhunia, A. K., Sain, A., Chowdhury, P. N., Xiang, T., & Song, Y.-Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01592 · Cited by 31
MoE-AGIQA: Mixture-of-Experts Boosted Visual Perception-Driven and Semantic-Aware Quality Assessment for AI-Generated Images
Yang, J., Fu, J., Zhang, W., Cao, W., Liu, L., & Peng, H. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00641 · Cited by 33
GeoSynth: Contextually-Aware High-Resolution Satellite Image Synthesis
Sastry, S., Khanal, S., Dhakal, A., & Jacobs, N. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00051 · Cited by 38
Enhancing Spatiotemporal Disease Progression Models via Latent Diffusion and Prior Knowledge
Puglisi, L., Alexander, D. C., & Ravì, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72069-7_17 · Cited by 35
Customize-A-Video: One-Shot Motion Customization of Text-to-Video Diffusion Models
Ren, Y., Zhou, Y., Yang, J., Shi, J., Liu, D., Liu, F., Kwon, M., & Shrivastava, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73024-5_20 · Cited by 28
AI-Instruments: Embodying Prompts as Instruments to Abstract & Reflect Graphical Interface Commands as General-Purpose Tools
Riche, N. H., Offenwanger, A., Gmeiner, F., Brown, D., Romat, H., Pahud, M., Marquardt, N., Inkpen, K. M., & Hinckley, K. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714259 · Cited by 49
A Collaborative, Interactive and Context-Aware Drawing Agent for Co-Creative Design
Ibarrola, F., Lawton, T., & Grace, K. S. (2023) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3293853 · Cited by 31
Diffusing the Creator: Attributing Credit for Generative AI Outputs
Khosrowi, D., Finn, F., & Clark, E. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604716 · Cited by 28
Learning Spatial Adaptation and Temporal Coherence in Diffusion Models for Video Super-Resolution
Chen, Z., Long, F., Qiu, Z., Yao, T., Zhou, W., Luo, J., & Mei, T. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.00882 · Cited by 27
Using a Diffusion Model for Pedestrian Trajectory Prediction in Semi-Open Autonomous Driving Environments
Tang, Y., He, H., Wang, Y., & Wu, Y. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3382406 · Cited by 28
Lightweight diffusion models: a survey
Song, W., Ma, W., Zhang, M., Zhang, Y., & Zhao, X. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10800-8 · Cited by 34
Development of a Method for Commercial Style Transfer of Historical Architectural Facades Based on Stable Diffusion Models
Zhang, J., Huang, Y., Li, Z., Li, Y., Yu, Z., & Li, M. (2024) · Journal of Imaging · doi:10.3390/jimaging10070165 · Cited by 27
OMG: Towards Open-vocabulary Motion Generation via Mixture of Controllers
Liang, H., Bao, J., Zhang, R., Ren, S., Xu, Y., Yang, S., Chen, X., Jingyi Yu, A., & Xu, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00053 · Cited by 33
All in One Framework for Multimodal Re-Identification in the Wild
He, L., Ye, M., Zhang, M. X., & Du, B. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01653 · Cited by 34
Make-Your-Anchor: A Diffusion-based 2D Avatar Generation Framework
Huang, Z., Tang, F., Zhang, Y., Cun, X., Cao, J., Li, J., & Lee, T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00668 · Cited by 32
Block and Detail: Scaffolding Sketch-to-Image Generation
Sarukkai, V., Yuan, L., Tang, M., Agrawala, M., & Fatahalian, K. (2024) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3654777.3676444 · Cited by 27
Self-citation
GDSG: Graph Diffusion-Based Solution Generator for Optimization Problems in MEC Networks
Liang, R., Yang, B. F., Chen, P., Cao, X., Yu, Z., Debbah, M. A., Niyato, D. T., Poor, H. V., & Yuen, C. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3568248 · Cited by 27
LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generation
Yang, S., Tan, J., Zhang, M., Wu, T., Wetzstein, G., Liu, Z., & Lin, D. (2025) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3721238.3730643 · Cited by 43
Align3R: Aligned Monocular Depth Estimation for Dynamic Videos
Lu, J., Huang, T., Li, P., Dou, Z., Lin, C. C., Cui, Z., Dong, Z., Yeung, S.-K., Wang, W., & Liu, Y. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02125 · Cited by 27
FaithDiff: Unleashing Diffusion Priors for Faithful Image Super-resolution
Chen, J., Pan, J., & Dong, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02625 · Cited by 27
ACE++: Instruction-Based Image Creation and Editing via Context-Aware Content Filling
Mao, C., Zhang, J., Pan, Y., Jiang, Z., Han, Z., Liu, Y., & Zhou, J. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00207 · Cited by 101
AI in Museums
Thiel, S., Bernhardt, J., & Medien, B. D. B. F. K. U. (2023) · Edition Museum · doi:10.1515/9783839467107 · Cited by 36
Sketch-Guided Latent Diffusion Model for High-Fidelity Face Image Synthesis
Peng, Y., Zhao, C., Xie, H., Fukusato, T., & Miyata, K. (2023) · IEEE Access · doi:10.1109/access.2023.3346408 · Cited by 34
Controllable Image Synthesis of Industrial Data using Stable Diffusion
Valvano, G., Agostino, A., Magistris, G. D., Graziano, A., & Veneri, G. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00527 · Cited by 28
UDiffText: A Unified Framework for High-Quality Text Synthesis in Arbitrary Images via Character-Aware Diffusion Models
Zhao, Y., & Lian, Z. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72751-1_13 · Cited by 27
Classification of artificial intelligence techniques for early architectural design stages
Vissers-Similon, E., Dounas, T., & Walsche, J. D. (2024) · International Journal of Architectural Computing · doi:10.1177/14780771241260857 · Cited by 35
Computers as co-creative assistants. A comparative study on the use of text-to-image AI models for computer aided conceptual design
Alcaide-Marzal, J., & Diego-Más, J. A. (2024) · Computers in Industry · doi:10.1016/j.compind.2024.104168 · Cited by 29
Diffusion Handles Enabling 3D Edits for Diffusion Models by Lifting Activations to 3D
Pandey, K., Guerrero, P., Gadelha, M., Hold-Geoffroy, Y., Singh, K., & Mitra, N. J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00735 · Cited by 35
ART•V: Auto-Regressive Text-to-Video Generation with Diffusion Models
Weng, W., Feng, R., Wang, Y., Dai, Q., Wang, C., Yin, D., Zhao, Z., Qiu, K., Bao, J., Yuan, Y., et al. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00735 · Cited by 31
Personalized Generative Storytelling with AI-Visual Illustrations for the Promotion of Knowledge in Cultural Heritage Tourism
Ferracani, A., Bertini, M., Pala, P., Nannotti, G., Principi, F., & Becchi, G. (2024) · doi:10.1145/3689094.3689465 · Cited by 31
UGG: Unified Generative Grasping
Lu, J., Kang, H., Li, H., Liu, B., Yang, Y., Huang, Q., & Hua, G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72855-6_24 · Cited by 44
TEXGen: a Generative Diffusion Model for Mesh Textures
Yu, X., Yuan, Z., Guo, Y.-C., Liu, Y.-T., Liu, J., Li, Y., Cao, Y., Liang, D., & Qi, X. (2024) · ACM Transactions on Graphics · doi:10.1145/3687909 · Cited by 31
Customizing Text-to-Image Models with a Single Image Pair
Jones, M., Wang, S.-Y., Kumari, N., Bau, D., & Zhu, J.-Y. (2024) · SIGGRAPH Asia Conference Papers · doi:10.1145/3680528.3687642 · Cited by 38
The Role of Generative Artificial Intelligence in Internet of Electric Vehicles
Zhang, H., Niyato, D. T., Zhang, W., Zhao, C., Du, H., Jamalipour, A., Sun, S., & Pei, Y. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3511961 · Cited by 33

Authors: Zhang, L., Rao, A., & Agrawala, M.
Affiliations: Stanford University (US)
ORCID: Zhang, Lvmin
Rao, Anyi
Agrawala, Maneesh
Venue: 2023 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher: IEEE
Published: 2023-10-01
Pages: 3813-3824
DOI: 10.1109/iccv51070.2023.00355
OpenAlex: W4390873054
Cited by: 4,547
References: 91
Citations by year: 2026: 1290 · 2025: 1847 · 2024: 1255 · 2023: 155
FWCI: 486.28
Type: conference paper
Language: en
Open access: closed
Licence: doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
Funders: Brown Institute for Media Innovation
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords: ControlNet, text-to-image diffusion models, spatial conditioning, conditional image generation, Stable Diffusion, parameter-efficient fine-tuning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Adding Conditional Control to Text-to-Image Diffusion Models

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks