Cited by 4,547Generative Adversarial Networks and Image SynthesisAdvanced Neural Network ApplicationsMultimodal Machine Learning Applications

The DOI matches its Crossref record; 88 of the 91 references it cites were found in a registry or an index.

Adding Conditional Control to Text-to-Image Diffusion Models

Zhang, Lvmin, Rao, Anyi, Agrawala, Maneesh·2023·2023 IEEE/CVF International Conference on Computer Vision (ICCV)

DOI 10.1109/iccv51070.2023.00355

Open this paper in CoChat — free account

Abstract

We present ControlNet, a neural network architecture to add spatial conditioning controls to large, pretrained text-to-image diffusion models. ControlNet locks the production-ready large diffusion models, and reuses their deep and robust encoding layers pretrained with billions of images as a strong backbone to learn a diverse set of conditional controls. The neural architecture is connected with "zero convolutions" (zero-initialized convolution layers) that progressively grow the parameters from zero and ensure that no harmful noise could affect the finetuning. We test various conditioning controls, e.g., edges, depth, segmentation, human pose, etc., with Stable Diffusion, using single or multiple conditions, with or without prompts. We show that the training of ControlNets is robust with small (1m) datasets. Extensive results show that ControlNet may facilitate wider applications to control image diffusion models.

Integrity signals

Registry

Crossref · 10.1109/iccv51070.2023.00355

Cites

88 of 88 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Deep learning
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
A Computational Approach to Edge Detection
Canny, J. (1986) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1986.4767851 · Cited by 30,004
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881

Show all 91 references

Research works printed but not linked: 13

2022, Disco diffusion
2022, Protogen x3.4 (photorealism) official release
Ha, Hypernetworks
2022, Hypernetwork style training, a tiny guide, stable-diffusion-webui
10.1109/TPAMI.2020.2970919 · DOI · Paper page
2022, Novelai improvements on stable diffusion
Lehtinen, Noise2noise: Learning image restoration without clean data
2023, Midjourney
2022, Comic-diffusion v2, trained on 6 styles at once
2023, Dall-e-2
Schuhmann, LAION-5b: An open large-scale dataset for training next generation image-text models
2022, Stable diffusion v1.5 model card
2022, Stable diffusion v2 model card, stable-diffusion-2-depth

Sorted by citations

Items 901–1,000 of 1,012

Quadratic Neuron-Empowered Heterogeneous Autoencoder for Unsupervised Anomaly Detection
Liao, J.-X., Hou, B., Dong, H., Zhang, H., Zhang, X., Sun, J., Zhang, S., & Fan, F. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3394795 · Cited by 15
Combinediff: a GenAI creative support tool for image combination exploration
Wang, Z., Tan, Z., & Ma, Y. (2024) · Journal of Engineering Design · doi:10.1080/09544828.2024.2358722 · Cited by 13
GenesisTex: Adapting Image Denoising Diffusion to Texture Space
Gao, C., Jiang, B., Li, X., Zhang, Y., & Yu, Q. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00442 · Cited by 13
Instruct 4D-to-4D: Editing 4D Scenes as Pseudo-3D Scenes Using 2D Diffusion
Mou, L., Chen, J., & Wang, Y.-X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01907 · Cited by 13
Learning Inclusion Matching for Animation Paint Bucket Colorization
Dai, Y., Zhou, S., Li, Q., Li, C., & Loy, C. C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02413 · Cited by 13
Layout-Agnostic Scene Text Image Synthesis with Diffusion Models
Zhangli, Q., Jiang, J., Liu, D., Yu, L., Dai, X., Ramchandani, A., Pang, G., Metaxas, D., & Krishnan, P. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00716 · Cited by 13
Guess The Unseen: Dynamic 3D Scene Reconstruction from Partial 2D Glimpses
Lee, I., Kim, B., & Joo, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00107 · Cited by 19
TiNO-Edit: Timestep and Noise Optimization for Robust Diffusion-Based Image Editing
Chen, X., Vaxman, Y., Baruch, E. B., Asulin, D., Moreshet, A., Lien, K.-C., Sra, M., & Sen, P. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00606 · Cited by 18
DiffusionRegPose: Enhancing Multi-Person Pose Estimation Using a Diffusion-Based End-to-End Regression Approach
Tan, D., Chen, H., Tian, W., & Xiong, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00217 · Cited by 13
DragAPart: Learning a Part-Level Motion Prior for Articulated Objects
Li, R., Zheng, C., Rupprecht, C., & Vedaldi, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72627-9_10 · Cited by 24
DiffUHaul: A Training-Free Method for Object Dragging in Images
Avrahami, O., Gal, R., Chechik, G., Fried, O., Lischinski, D., Vahdat, A., & Nie, W. (2024) · SIGGRAPH Asia Conference Papers · doi:10.1145/3680528.3687590 · Cited by 13
StepIdeator: Utilizing Mixed Representations to Support Step-By-Step Design With Generative Artificial Intelligence
Yao, J., Chen, P., Li, Z., Cai, Y., Wu, Y., You, W., & Sun, L. (2024) · Journal of Mechanical Design · doi:10.1115/1.4067426 · Cited by 15
Semantic Similarity Score for Measuring Visual Similarity at Semantic Level
Fan, S., Bao, Z., Dong, C., Liang, H., Xu, X., & Zhang, P. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3518543 · Cited by 13
AutoStory: Generating Diverse Storytelling Images with Minimal Human Efforts
Wang, W., Zhao, C., Chen, H., Chen, Z., Zheng, K., & Shen, C. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02309-y · Cited by 27
FLAME Diffuser: Wildfire Image Synthesis using Mask Guided Diffusion
Wang, H., Boroujeni, S. P. H., Chen, X., Bastola, A., Li, H., Zhu, W., & Razi, A. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10826040 · Cited by 18
Cityscape-Adverse: Benchmarking Robustness of Semantic Segmentation With Realistic Scene Modifications via Diffusion-Based Image Editing
Suryanto, N., Adiputra, A. A., Kadiptya, A. Y., Le, T.-T.-H., Pratama, D., Kim, Y., & Kim, H. (2025) · IEEE Access · doi:10.1109/access.2025.3537981 · Cited by 14
Controllable Generative Knowledge-Driven Few-Shot Object Detection From Optical Remote Sensing Imagery
Zhang, T., Zhuang, Y., Wang, G., Chen, H., Wang, H., Li, L., & Li, J. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3541937 · Cited by 14
Guess What I Think: Streamlined EEG-to-Image Generation with Latent Diffusion Models
Lopez, E., Sigillo, L., Colonnese, F., Panella, M., & Comminiello, D. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890059 · Cited by 18
CATS: cultural-heritage classification using LLMs and distribute model
Hwang, H., Park, C.-W., Kim, H., & Lee, J.-H. (2025) · npj Heritage Science · doi:10.1038/s40494-025-01621-1 · Cited by 13
PromptMap: An Alternative Interaction Style for AI-Based Image Generation
Adamkiewicz, K., Woźniak, P. W., Dominiak, J., Romanowski, A., Karolus, J., & Frolov, S. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712150 · Cited by 17
SimuScope: Realistic Endoscopic Synthetic Dataset Generation Through Surgical Simulation and Diffusion Models
Martyniak, S., Kaleta, J., Dall’Alba, D., Naskręt, M., Płotka, S., & Korzeniowski, P. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00419 · Cited by 14
Towards AI-driven Sign Language Generation with Non-manual Markers
Zhang, H., Shalev-Arkushin, R., Baltatzis, V., Gillis, C., Laput, G., Kushalnagar, R., Quandt, L. C., Findlater, L., Bedri, A., & Lea, C. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713855 · Cited by 19
BootPIG: Bootstrapping Zero-Shot Personalized Image Generation Capabilities in Pretrained Diffusion Models
Purushwalkam, S., Gokul, A., Joty, S., & Naik, N. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-91907-7_15 · Cited by 24
DrivingSphere: Building a High-fidelity 4D World for Closed-loop Simulation
Yan, T., Wu, D., Han, W., Jiang, J., Zhou, X., Zhan, K., Xu, C., & Shen, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02564 · Cited by 27
Generative Inbetweening through Frame-wise Conditions-Driven Video Generation
Zhu, T., Ren, D., Wang, Q., Wu, X., & Zuo, W. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02604 · Cited by 13
AutoVFX: Physically Realistic Video Editing from Natural Language Instructions
Hsu, H.-Y., Lin, C., Zhai, A. J., Xia, H., & Wang, S. (2025) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv66043.2025.00076 · Cited by 33
Learning Strategies for Sensitive Content Detection
Álvarez, D. P., Orozco, A. L. S., García-Miguel, J. P., & Villalba, L. J. G. (2023) · Electronics · doi:10.3390/electronics12112496 · Cited by 16
Degeneration-Tuning: Using Scrambled Grid shield Unwanted Concepts from Stable Diffusion
Ni, Z., Wei, L., Li, J., Tang, S., Zhuang, Y., & Tian, Q. C. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611867 · Cited by 21
Controllable image generation based on causal representation learning
Huang, S., Wang, Y., Gong, Z., Liao, J., Wang, S., & Liu, L. (2024) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2300303 · Cited by 13
ProteusNeRF: Fast Lightweight NeRF Editing using 3D-Aware Image Context
Wang, B., Dutt, N. S., & Mitra, N. J. (2024) · Proceedings of the ACM on Computer Graphics and Interactive Techniques · doi:10.1145/3651290 · Cited by 13
AnimatableDreamer: Text-Guided Non-rigid 3D Model Generation and Reconstruction with Canonical Score Distillation
Wang, X., Wang, Y., Ye, J., Sun, F., Wang, Z., Wang, L., Liu, P., Sun, K., Wang, X., Xie, W., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72698-9_19 · Cited by 13
Steerable Conditional Diffusion for Out-of-Distribution Adaptation in Medical Image Reconstruction
Barbano, R., Denker, A., Chung, H., Roh, T. H., Arridge, S., Maaß, P., Jin, B., & Ye, J. C. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3524797 · Cited by 27
HumanLiff: Layer-wise 3D Human Diffusion Model
Hu, S., Hong, F., Hu, T., Pan, L., Mei, H., Xiao, W., Yang, L., & Liu, Z. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02477-5 · Cited by 23
VideoPainter: Any-length Video Inpainting and Editing with Plug-and-Play Context Control
Bian, Y., Zhang, Z., Ju, X., Cao, M., Xie, L., Shan, Y., & Xu, Q. (2025) · Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers · doi:10.1145/3721238.3730673 · Cited by 12
An intelligent generative method of fashion design combining attribute knowledge and Stable Diffusion Model
Chen, Y., & Ma, J. (2024) · Textile Research Journal · doi:10.1177/00405175241289578 · Cited by 13
Exploration of Generative Neural Networks for Police Facial Sketches
Sádaba-Campo, N., & Gómez-Moreno, H. (2025) · Big Data and Cognitive Computing · doi:10.3390/bdcc9020042 · Cited by 6
SwiftSketch: A Diffusion Model for Image-to-Vector Sketch Generation
Arar, E., Frenkel, Y., Cohen‐Or, D., Shamir, A., & Vinker, Y. (2025) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3721238.3730612 · Cited by 12
BrepDiff: Single-Stage B-rep Diffusion Model
Lee, M., Zhang, D., Jambon, C., & Kim, Y. M. (2025) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3721238.3730698 · Cited by 22
Autoregressive Sequential Pretraining for Visual Tracking
Liang, S., Bai, Y., Gong, Y., & Wei, X. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00680 · Cited by 12
EdgeDiff: Edge-aware Diffusion Network for Building Reconstruction from Point Clouds
Liu, Y., Wang, R., Huang, S., & Cai, G. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01585 · Cited by 12
Few-shot agricultural disease detection method using contextual attention generation
Ning, X., Gao, S., Liu, J., Cheng, L., & Zhang, Y. (2025) · Alexandria Engineering Journal · doi:10.1016/j.aej.2025.08.045 · Cited by 12
Box It to Bind It: Unified Layout Control and Attribute Binding in Text-to-Image Diffusion Models
Taghipour, A., Ghahremani, M., Bennamoun, M., Rekavandi, A. M., Laga, H., & Boussaïd, F. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3607759 · Cited by 18
InstanceRSR: Real-World Super-Resolution via Instance-Aware Representation Alignment
Guo, Z., Zhao, K., & Zhang, L. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11462690 · Cited by 17
Self-citation
Semantic Generative Augmentations for Few-Shot Counting
Doubinsky, P., Audebert, N., Crucianu, M., & Borgne, H. L. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00536 · Cited by 15
Region Assisted Sketch Colorization
Wang, N., Niu, M., Wang, Z., Hu, K., Liu, B., Wang, Z., & Li, H. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3326682 · Cited by 12
Learning Image Harmonization in the Linear Color Space
Xu, K., Hancke, G. P., & Lau, R. W. H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01155 · Cited by 12
Learning Playing Piano with Bionic-Constrained Diffusion Policy for Anthropomorphic Hand
Yang, Y., Wang, Z., Xing, D., & Wang, P. (2024) · Cyborg and Bionic Systems · doi:10.34133/cbsystems.0104 · Cited by 12
CAGEN: Controllable Anomaly Generator using Diffusion Model
Jiang, B., Xie, Y., Li, J., Li, N., Jiang, Y., & Xia, S. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447663 · Cited by 13
As-Plausible-As-Possible: Plausibility-Aware Mesh Deformation Using 2D Diffusion Priors
Yoo, S., Kim, K., Kim, V. G., & Sung, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00413 · Cited by 12
Generating Non-Stationary Textures Using Self-Rectification
Zhou, Y., Xiao, R., Lischinski, D., Cohen‐Or, D., & Huang, H. F. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00742 · Cited by 13
Text2QR: Harmonizing Aesthetic Customization and Scanning Robustness for Text-Guided QR Code Generation
Wu, G., Liu, X., Jia, J., Cui, X., & Zhai, G. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00808 · Cited by 12
Learning Disentangled Identifiers for Action-Customized Text-to-Image Generation
Huang, S., Gong, B., Feng, Y., Chen, X., Fu, Y., Liu, Y., & Wang, D. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00745 · Cited by 18
Towards Effective Usage of Human-Centric Priors in Diffusion Models for Text-based Human Image Generation
Wang, J., Sun, Z., Tan, Z., Chen, X., Chen, W., Li, H., Zhang, C., & Song, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00807 · Cited by 20
WordRobe: Text-Guided Generation of Textured 3D Garments
Srivastava, A., Manu, P., Raj, A., Jampani, V., & Sharma, A. K. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73232-4_26 · Cited by 16
Factorized Diffusion: Perceptual Illusions by Noise Decomposition
Geng, D., Park, I., & Owens, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72998-0_21 · Cited by 20
CLAP: Isolating Content from Style Through Contrastive Learning with Augmented Prompts
Cai, Y., Liu, Y., Zhang, Z., & Shi, Q. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72664-4_8 · Cited by 12
SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfer
Wu, Z., Yu, C., Jiang, Y., Cao, C., Wang, F., & Bai, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72624-8_21 · Cited by 12
Magic Clothing: Controllable Garment-Driven Image Synthesis
Chen, W., Gu, T., Xu, Y., & Chen, C. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680691 · Cited by 12
FedDEO: Description-Enhanced One-Shot Federated Learning with Diffusion Models
Yang, M., Su, S., Li, B., & Xue, X. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681490 · Cited by 20
Diffusion Domain Teacher: Diffusion Guided Domain Adaptive Object Detector
He, B., Ji, Y., Tan, Z., & Wu, L. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680962 · Cited by 15
ConceptExpress: Harnessing Diffusion Models for Single-Image Unsupervised Concept Extraction
Hao, S., Han, K., Lv, Z., Zhao, S., & Wong, K. K. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73202-7_13 · Cited by 12
Instant 3D Human Avatar Generation Using Image Diffusion Models
Kolotouros, N., Alldieck, T., Corona, E., Băzăvan, E. G., & Sminchisescu, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73021-4_11 · Cited by 16
Faithful crack image synthesis from evolutionary pixel-level annotations via latent semantic diffusion model
Lei, Q., Zhong, J., Dong, M., & Ota, K. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126986 · Cited by 13
MMO-IG: Multiclass and Multiscale Object Image Generation for Remote Sensing
Yang, C., Zhao, B., Zhou, Q., & Wang, Q. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3550404 · Cited by 15
MineVRA: Exploring the Role of Generative AI-Driven Content Development in XR Environments through a Context-Aware Approach
Santarnecchi, E., Balloni, E., Paolanti, M., Frontoni, E., Stacchio, L., Zingaretti, P., & Pierdicca, R. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3549160 · Cited by 12
MagicScroll: Enhancing Immersive Storytelling with Controllable Scroll Image Generation
Wang, B., Meng, H., Cao, R., Cai, Z., Li, L., Ma, Y., Chen, Q., & Wang, Z. (2025) · IEEE Conference on Virtual Reality and 3D User Interfaces (VR) · doi:10.1109/vr59515.2025.00067 · Cited by 13
Towards Better Cephalometric Landmark Detection With Diffusion Data Generation
Guo, D., Han, W., Lyu, P., Zhou, Y., & Shen, J. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2025.3557430 · Cited by 12
Refining Text-to-Image Generation: Towards Accurate Training-Free Glyph-Enhanced Image Generation
Lakhanpal, S., Chopra, S., Jain, V., Chadha, A. R., & Luo, M. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00429 · Cited by 19
IllusionCAPTCHA: A CAPTCHA based on Visual Illusion
Ding, Z., Deng, G., Liu, Y., Ding, J., Chen, J., Sui, Y., & Li, Y. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714726 · Cited by 13
Dreamcrafter: Immersive Editing of 3D Radiance Fields Through Flexible, Generative Inputs and Outputs
Vachha, C., Kang, Y., Dive, Z., Chidambaram, A., GUPTA, A. C., Jun, E., & Hartmann, B. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714312 · Cited by 12
Sketch-guided scene image generation with diffusion model
Zhang, T., Xie, X., Du, X., & Xie, H. (2025) · Computers & Graphics · doi:10.1016/j.cag.2025.104226 · Cited by 19
Style transfer with diffusion models for synthetic-to-real domain adaptation
Chigot, E., Wilson, D. G., Ghrib, M., & Oberlin, T. (2025) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2025.104445 · Cited by 17
A novel flexible identity-net with diffusion models for painting-style generation
Zhao, Y., Liang, Z., Qiu, Y., Wang, X., & Yang, W. (2025) · Scientific Reports · doi:10.1038/s41598-025-12434-4 · Cited by 13
Large-Scale Text-to-Image Model with Inpainting is a Zero-Shot Subject-Driven Image Generator
Shin, C., Choi, J., Kim, H., & Yoon, S. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00748 · Cited by 28
Precise, Fast, and Low-cost Concept Erasure in Value Space: Orthogonal Complement Matters
Wang, Y., Li, O., Mu, T., Hao, Y., Liu, K., Wang, X., & He, X. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02678 · Cited by 12
EmoEdit: Evoking Emotions through Image Manipulation
Yang, J., Feng, J., Luo, W., Lischinski, D., Cohen‐Or, D., & Huang, H. F. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02299 · Cited by 12
HumanRig: Learning Automatic Rigging for Humanoid Character in a Large Scale Dataset
Chu, Z., Xiong, F., Liu, M., Zhang, J., Shao, M., Sun, Z., Wang, D., & Xu, M. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00037 · Cited by 12
SuperDiff: A diffusion super-resolution method for digital pathology with comprehensive quality assessment
Xu, X., Kapse, S., & Prasanna, P. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103808 · Cited by 16
Mitigating misalignment in MRI-to-CT synthesis for improved synthetic CT generation: an iterative refinement and knowledge distillation approach
Zhou, L., Ni, X., Kong, Y., Zeng, H., Xu, M., Zhou, J., Wang, Q., & Liu, C. (2023) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ad0ddc · Cited by 12
UniHuman: A Unified Model For Editing Human Images in the Wild
Li, N., Liu, Q., Singh, K. K., Wang, Y., Zhang, J., Plummer, B. A., & Lin, Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00199 · Cited by 12
SCoFT: Self-Contrastive Fine-Tuning for Equitable Image Generation
Liu, Z., Schaldenbrand, P., Okogwu, B.-C., Peng, W., Peng, W., Yun, Y., Hundt, A., Kim, J., & Oh, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01029 · Cited by 12
NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Models
Dalva, Y., & Yanardag, P. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02285 · Cited by 29
Getting it Right: Improving Spatial Consistency in Text-to-Image Models
Chatterjee, A., Stan, G. B. M., Aflalo, E., Paul, S., Ghosh, D., Gokhale, T., Schmidt, L., Hajishirzi, H., Lal, V., Baral, C. R., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72670-5_12 · Cited by 21
Infinite-ID: Identity-Preserved Personalization via ID-Semantics Decoupling Paradigm
Wu, Y., Li, Z., Zheng, H., Wang, C., & Li, B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73242-3_16 · Cited by 12
StructLDM: Structured Latent Diffusion for 3D Human Generation
Hu, T., Hong, F., & Liu, Z. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72983-6_21 · Cited by 12
Pulling Target to Source: A New Perspective on Domain Adaptive Semantic Segmentation
Wang, H., Shen, Y., Fei, J., Li, W., Wu, L., Wang, Y., & Zhang, Z. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02285-3 · Cited by 12
PanoGen++: Domain-adapted text-guided panoramic environment generation for vision-and-language navigation
Wang, S., Zhou, D., Xie, L., Xu, C., Yan, Y., & Yin, E. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107320 · Cited by 11
Pushing the Boundaries of Salient Object Detection: A Denoising-Driven Approach
Song, M., Li, L., Yu, X., & Chen, C. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3576993 · Cited by 11
FlexiAct: Towards Flexible Action Control in Heterogeneous Scenarios
Zhang, S., Zhuang, J., Zhang, Z., Shan, Y., & Tang, Y. (2025) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3721238.3730683 · Cited by 11
DDEG: Addressing data scarcity in additive manufacturing defect detection through joint generation of CT images and defect annotations
Han, R., Wang, Y., Guo, W., Wang, C., Zhang, Y., Zi, Y., & Zhao, J. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2025.103619 · Cited by 11
A Survey of Multimodal Hallucination Evaluation and Detection
Chen, Z., Min, Y., Zhang, J., Yan, B., Wang, J., Wang, X., & Shan, S. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02756-9 · Cited by 31
Dimensionx: Create Any 3D and 4D Scenes From a Single Image With Decoupled Video Diffusion
Sun, W., Chen, S., Liu, F., Chen, Z. L., Duan, Y., Zhu, J., Zhang, J., & Wang, Y. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01271 · Cited by 22
MakeupDiffuse: a double image-controlled diffusion model for exquisite makeup transfer
Lu, X., Liu, F., Rong, Y., Chen, Y., & Xiong, S. (2024) · The Visual Computer · doi:10.1007/s00371-024-03317-2 · Cited by 11
UWAFA-GAN: Ultra-Wide-Angle Fluorescein Angiography Transformation via Multi-Scale Generation and Registration Enhancement
Ge, R., Fang, Z., Wei, P., Chen, Z., Jiang, H., Elazab, A., Li, W., Wan, X., Zhang, S., & Wang, C. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3394597 · Cited by 11
Sketch-Guided Text-to-Image Generation with Spatial Control
Zhang, T., & Xie, H. (2024) · International Conference on Computer Graphics and Image Processing (CGIP) · doi:10.1109/cgip62525.2024.00035 · Cited by 15
Separate-and-Enhance: Compositional Finetuning for Text-to-Image Diffusion Models
Bao, Z., Li, Y., Singh, K. K., Wang, Y.-X., & Hebert, M. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657527 · Cited by 16
Object-level Scene Deocclusion
Liu, Z., Liu, Q., Chang, C., Zhang, J., Pakhomov, D., Zheng, H., Lin, Z., Cohen‐Or, D., & Fu, C. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657409 · Cited by 11
LD-CSNet: A latent diffusion-based architecture for perceptual Compressed Sensing
Zheng, B., Sun, G., Dong, L., & Wang, S. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106541 · Cited by 11
3D Character Animation and Asset Generation Using Deep Learning
Lungu-Stan, V.-C., & Mocanu, I. G. (2024) · Applied Sciences · doi:10.3390/app14167234 · Cited by 11
Text-guided image-to-sketch diffusion models
Ke, A., Huang, Y., Yang, J., & Cai, B. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112441 · Cited by 12

Authors: Zhang, L., Rao, A., & Agrawala, M.
Affiliations: Stanford University (US)
ORCID: Zhang, Lvmin
Rao, Anyi
Agrawala, Maneesh
Venue: 2023 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher: IEEE
Published: 2023-10-01
Pages: 3813-3824
DOI: 10.1109/iccv51070.2023.00355
OpenAlex: W4390873054
Cited by: 4,547
References: 91
Citations by year: 2026: 1290 · 2025: 1847 · 2024: 1255 · 2023: 155
FWCI: 486.28
Type: conference paper
Language: en
Open access: closed
Licence: doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
Funders: Brown Institute for Media Innovation
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords: ControlNet, text-to-image diffusion models, spatial conditioning, conditional image generation, Stable Diffusion, parameter-efficient fine-tuning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Adding Conditional Control to Text-to-Image Diffusion Models

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks