Cited by 4,547Generative Adversarial Networks and Image SynthesisAdvanced Neural Network ApplicationsMultimodal Machine Learning Applications

The DOI matches its Crossref record; 88 of the 91 references it cites were found in a registry or an index.

Adding Conditional Control to Text-to-Image Diffusion Models

Zhang, Lvmin, Rao, Anyi, Agrawala, Maneesh·2023·2023 IEEE/CVF International Conference on Computer Vision (ICCV)

DOI 10.1109/iccv51070.2023.00355

Open this paper in CoChat — free account

Abstract

We present ControlNet, a neural network architecture to add spatial conditioning controls to large, pretrained text-to-image diffusion models. ControlNet locks the production-ready large diffusion models, and reuses their deep and robust encoding layers pretrained with billions of images as a strong backbone to learn a diverse set of conditional controls. The neural architecture is connected with "zero convolutions" (zero-initialized convolution layers) that progressively grow the parameters from zero and ensure that no harmful noise could affect the finetuning. We test various conditioning controls, e.g., edges, depth, segmentation, human pose, etc., with Stable Diffusion, using single or multiple conditions, with or without prompts. We show that the training of ControlNets is robust with small (1m) datasets. Extensive results show that ControlNet may facilitate wider applications to control image diffusion models.

Integrity signals

Registry

Crossref · 10.1109/iccv51070.2023.00355

Cites

88 of 88 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Deep learning
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
A Computational Approach to Edge Detection
Canny, J. (1986) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1986.4767851 · Cited by 30,004
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881

Show all 91 references

Research works printed but not linked: 13

2022, Disco diffusion
2022, Protogen x3.4 (photorealism) official release
Ha, Hypernetworks
2022, Hypernetwork style training, a tiny guide, stable-diffusion-webui
10.1109/TPAMI.2020.2970919 · DOI · Paper page
2022, Novelai improvements on stable diffusion
Lehtinen, Noise2noise: Learning image restoration without clean data
2023, Midjourney
2022, Comic-diffusion v2, trained on 6 styles at once
2023, Dall-e-2
Schuhmann, LAION-5b: An open large-scale dataset for training next generation image-text models
2022, Stable diffusion v1.5 model card
2022, Stable diffusion v2 model card, stable-diffusion-2-depth

Sorted by citations

Items 601–700 of 1,012

Sat2Scene: 3D Urban Scene Generation from Satellite Images with Diffusion
Li, Z., Li, Z., Cui, Z., Pollefeys, M., & Oswald, M. R. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00682 · Cited by 33
PosterLlama: Bridging Design Ability of Language Model to Content-Aware Layout Generation
Seol, J., Kim, S., & Yoo, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73007-8_26 · Cited by 39
LATTE3D: Large-scale Amortized Text-To-Enhanced3D Synthesis
Xie, K., Lorraine, J., Cao, T., Gao, J., Lucas, J. M., Torralba, A. B., Fidler, S., & Zeng, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72980-5_18 · Cited by 39
UniEdit: A Unified Tuning-Free Framework for Video Motion and Appearance Editing
Bai, J., He, T., Wang, Y., Guo, J., Hu, H. R., Liu, Z., & Bian, J. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755462 · Cited by 22
Improving the Stability and Efficiency of Diffusion Models for Content Consistent Super-Resolution
Sun, L., Wu, R., Liang, J., Zhang, Z., Yong, H., & Zhang, L. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3640863 · Cited by 46
Playgrounds and Prejudices: Exploring Biases in Generative AI For Children.
Baines, A., Gruia, L., Collyer-Hoar, G., & Rubegni, E. (2024) · Proceedings of the 23rd Annual ACM Interaction Design and Children Conference · doi:10.1145/3628516.3659404 · Cited by 21
DesignWeaver: Dimensional Scaffolding for Text-to-Image Product Design
Tao, S., Liang, I., Peng, C., Wang, Z., Palani, S., & Dow, S. P. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714211 · Cited by 21
MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Model
Niu, M., Cun, X., Wang, X., Zhang, Y., Shan, Y., & Zheng, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72655-2_7 · Cited by 21
Design Transformation Pathways for AI-Generated Images in Chinese Traditional Architecture
Lu, Y., Wu, J., Wang, M., Fu, J., Xie, W., Wang, P., & Zhao, P. (2025) · Electronics · doi:10.3390/electronics14020282 · Cited by 22
Video Anomaly Detection with Motion and Appearance Guided Patch Diffusion Model
Zhou, H., Cai, J., Ye, Y., Feng, Y., Gao, C., Yu, J., Song, Z., & Yang, W. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i10.33169 · Cited by 26
Brickify: Enabling Expressive Design Intent Specification through Direct Manipulation on Design Tokens
Shi, X., Wang, Y., Rossi, R. A., & Zhao, J. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714087 · Cited by 21
Leveraging generative models with periodicity-aware, invertible and invariant representations for crystalline materials design
Wang, Z., & You, F. (2025) · Nature Computational Science · doi:10.1038/s43588-025-00797-7 · Cited by 23
Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Models
Wu, R., Su, W., & Liao, J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02206 · Cited by 21
DesignDiffusion: High-Quality Text-to-Design Image Generation with Diffusion Models
Wang, Z., Bao, J., Gu, S., Chen, D., Zhou, W., & Li, H. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01947 · Cited by 30
Toward Generalized Image Quality Assessment: Relaxing the Perfect Reference Quality Assumption
Du, C., Wu, T., Ma, K., & Lei Zhang, A. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01189 · Cited by 21
Single-Step Latent Diffusion for Underwater Image Restoration
Wu, J., Wang, T., Siddique, M. A. B., Islam, M. J., Fermuller, C. M., Aloimonos, Y., & Metzler, C. A. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3599775 · Cited by 23
Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls for Video Generation
Cao, C., Zhou, J., Li, S., Liang, J., Yu, C., Wang, F., Xue, X., & Fu, Y. (2025) · SIGGRAPH Asia Conference Papers · doi:10.1145/3757377.3763842 · Cited by 57
Edge Guided GANs With Multi-Scale Contrastive Learning for Semantic Image Synthesis
Tang, H., Sun, G.-L., Sebe, N., & van Gool, L. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3298721 · Cited by 26
AdaptNet: Policy Adaptation for Physics-Based Character Control
Xu, P., Xie, K., Andrews, S., Kry, P. G., Neff, M., McGuire, M., Karamouzas, I., & Zordan, V. B. (2023) · ACM Transactions on Graphics · doi:10.1145/3618375 · Cited by 21
HyperDreamer: Hyper-Realistic 3D Content Generation and Editing from a Single Image
Wu, T., Li, Z., Yang, S., Zhang, P., Pan, X., Wang, J., Lin, D., & Liu, Z. (2023) · SIGGRAPH Asia Conference Papers · doi:10.1145/3610548.3618168 · Cited by 21
Recasting Regional Lighting for Shadow Removal
Liu, Y., Ke, Z., Xu, K., Liu, F., Wang, Z., & Lau, R. W. H. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i4.28172 · Cited by 30
TokenCompose: Text-to-Image Diffusion with Token-Level Supervision
Wang, Z., Sha, Z., Ding, Z., Wang, Y., & Tu, Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00817 · Cited by 39
LivePhoto: Real Image Animation with Text-Guided Motion Control
Chen, X., Liu, Z., Chen, M., Feng, Y., Liu, Y., Shen, Y., & Zhao, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72649-1_27 · Cited by 21
LighTDiff: Surgical Endoscopic Image Low-Light Enhancement with T-Diffusion
Chen, T., Lyu, Q., Bai, L., Guo, E., Gao, H., Yang, X., Ren, H., & Zhou, L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72089-5_35 · Cited by 21
Brain-Conditional Multimodal Synthesis: A Survey and Taxonomy
Mai, W., Zhang, J., Fang, P., & Zhang, Z. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3516698 · Cited by 26
Enhancing participatory planning with ChatGPT-assisted planning support systems: a hypothetical case study in Seoul
Quan, S. J., & Lee, S. (2025) · International Journal of Urban Sciences · doi:10.1080/12265934.2025.2462823 · Cited by 23
Paragraph-to-Image Generation with Information-Enriched Diffusion Model
Wu, W., Li, Z., He, Y., Shou, M. Z., Shen, C., Cheng, L., Li, Y., Gao, T., & Zhang, D. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02435-1 · Cited by 38
ConsistentID: Portrait Generation With Multimodal Fine-Grained Identity Preserving
Huang, J., Dong, X. M., Song, W., Zheng, C., Tang, Z., Zhou, J., Cheng, Y., Chen, L., Li, H., Yan, Y., et al. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3652557 · Cited by 52
AI art curation: re-imagining the city of Helsinki in occasion of its Biennial
Schaerf, L., Zapata, P. B., Bernasconi, V., Neri, I., & Castillo, D. N. D. (2026) · Multimedia Tools and Applications · doi:10.1007/s11042-026-21219-4 · Cited by 22
Coin3D: Controllable and Interactive 3D Assets Generation with Proxy-Guided Conditioning
Dong, W., Yang, B., Ma, L., Liu, X., Cui, L., Bao, H., Ma, Y., & Cui, Z. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657425 · Cited by 20
CosmicMan: A Text-to-Image Foundation Model for Humans
Li, S., Fu, J., Liu, K., Wang, W., Lin, K.-Y., & Wu, W. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00664 · Cited by 26
Artificial intelligence for geometry-based feature extraction, analysis and synthesis in artistic images: a survey
Vijendran, M., Deng, J., Chen, S., Ho, E. S. L., & Shum, H. P. H. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-11051-3 · Cited by 20
A review on 3D Gaussian splatting for sparse view reconstruction
Liu, H., Liu, B., Hu, Q., Du, P., Li, J., Bao, Y., & Wang, F. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11171-4 · Cited by 20
Reconstruction of Cultural Heritage in Virtual Space Following Disasters
Chen, G., Tong, Y., Wu, Y., Wu, Y., Liu, Z., & Huang, J. (2025) · Buildings · doi:10.3390/buildings15122040 · Cited by 22
Towards Enhanced Image Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistency
Wang, Y., Cao, C., Yu, J., Fan, K., Xue, X., & Fu, Y. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02164 · Cited by 20
DreamO: A Unified Framework for Image Customization
Mou, C., Wu, Y., Wu, W., Guo, Z., Zhang, P., Cheng, Y., Yiming, L., Ding, F., Zhang, S., Li, X., et al. (2025) · SIGGRAPH Asia Conference Papers · doi:10.1145/3757377.3763956 · Cited by 73
Painterly Image Harmonization using Diffusion Model
Lu, L., Li, J., Cao, J., Niu, L., & Zhang, L. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612451 · Cited by 23
Self-citation
Tangible Diffusion: Exploring Artwork Generation via Tangible Elements and AI Generative Models in Arts and Design Education
Han, K., Tang, K., & Wang, M. (2024) · International Conference on Tangible, Embedded, and Embodied Interaction (TEI) · doi:10.1145/3623509.3633394 · Cited by 20
Controllable Mind Visual Diffusion Model
Zeng, B., Li, S., Liu, X., Gao, S., Jiang, X., Tang, X., Hu, Y. B., Liu, J., & Zhang, B. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i7.28519 · Cited by 35
A Diffusion Approach to Radiance Field Relighting using Multi‐Illumination Synthesis
Poirier‐Ginter, Y., Gauthier, A., Philip, J., Lalonde, J., & George Drettakis, B. (2024) · Computer Graphics Forum · doi:10.1111/cgf.15147 · Cited by 20
GALA: Generating Animatable Layered Assets from a Single Scan
Kim, T., Kim, B., Saito, S., & Joo, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00152 · Cited by 22
CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image Generation
Mei, K., Delbracio, M., Talebi, H., Tu, Z., Patel, V. M., & Milanfar, P. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00864 · Cited by 24
DPMesh: Exploiting Diffusion Prior for Occluded Human Mesh Recovery
Zhu, Y., Li, A., Tang, Y., Zhao, W., Zhou, J., & Lu, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00111 · Cited by 24
CapHuman: Capture Your Moments in Parallel Universes
Liang, C., Ma, F., Zhu, L., Deng, Y., & Yang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00612 · Cited by 22
LeftRefill: Filling Right Canvas based on Left Reference through Generalized Text-to-Image Diffusion Model
Cao, C., Cai, Y., Dong, Q., Wang, Y., & Fu, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00736 · Cited by 20
Exploiting Diffusion Prior for Generalizable Dense Prediction
Lee, H.-Y., Tseng, H.-Y., Lee, H.-Y., & Yang, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00751 · Cited by 30
Domain Targeted Synthetic Plant Style Transfer using Stable Diffusion, LoRA and ControlNet
Hartley, Z. K. J., Lind, R. J., Pound, M. P., & French, A. P. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00546 · Cited by 23
DirectL: Efficient Radiance Fields Rendering for 3D Light Field Displays
Yang, Z., Liu, B., Song, Y., Yi, L., Xiong, Y., Zhang, Z., & Yu, X. (2024) · ACM Transactions on Graphics · doi:10.1145/3687897 · Cited by 20
Lossy Image Compression with Foundation Diffusion Models
Relic, L., Azevedo, R. G. D. A., Groß, M. H., & Schroers, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73030-6_17 · Cited by 20
TrackDiffusion: Tracklet-Conditioned Video Generation via Diffusion Models
Li, P., Chen, K., Liu, Z., Gao, R., Hong, L., Yeung, D., Lu, H., & Xu, J. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00349 · Cited by 40
Garment3DGen: 3D Garment Stylization and Texture Generation
Sarafianos, N., Stuyck, T., Xiang, X., Li, Y., Popović, J., & Ranjan, R. (2025) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv66043.2025.00131 · Cited by 20
Effective variance attention-enhanced diffusion model for crop field aerial image super resolution
Lu, X., Zhang, J., Yang, R., Yang, Q., Chen, M., Xu, H., Wan, P., Guo, J., & Liu, F. (2024) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2024.08.017 · Cited by 22
DiffH2O: Diffusion-Based Synthesis of Hand-Object Interactions from Textual Descriptions
Christen, S., Hampali, S., Şener, F., Remelli, E., Hodaň, T., Sauser, E. L., Ma, S., & Tekin, B. (2024) · SIGGRAPH Asia Conference Papers · doi:10.1145/3680528.3687563 · Cited by 44
SSP-IR: Semantic and Structure Priors for Diffusion-Based Realistic Image Restoration
Zhang, Y., Zhang, H., Cheng, Z., Xie, R., Song, L., & Zhang, W. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3538772 · Cited by 20
A Survey of Representation Learning, Optimization Strategies, and Applications for Omnidirectional Vision
Ai, H., Cao, Z., & Wang, L. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02391-w · Cited by 19
DesignFromX:Empowering Consumer-Driven Design Space Exploration through Feature Composition of Referenced Products
Duan, R., Zhu, C., Chen, Y., Hu, Y., Shi, J., & Ramani, K. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735824 · Cited by 21
Noise-Consistent Siamese-Diffusion for Medical Image Synthesis and Segmentation
Qiu, K., Gao, Z., Zhou, S. Z., Sun, M., & Guo, Y. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01461 · Cited by 19
Semantic Communication Based on Large Language Model for Underwater Image Transmission
Chen, W., Xu, W., Chen, H., Zhang, X., Qin, Z., Zhang, Y., & Han, Z. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3607717 · Cited by 26
Prompt-Guided Dual Latent Steering for Inversion Problems
Wu, Y., Liu, T., Zhao, C., & Wu, X. (2025) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta68720.2025.11302481 · Cited by 19
Generative AI for Cel-Animation: A Survey
Tang, Y., Guo, J., Liu, P., Wang, Z., Hua, H., Zhong, J.-X., Xiao, Y., Huang, C., Song, L., Liang, S., et al. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00400 · Cited by 19
MagicDrive-V2: High-Resolution Long Video Generation for Autonomous Driving with Adaptive Control
Gao, R., Chen, K., Xiao, B., Hong, L., Li, Z., & Xu, Q. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.02612 · Cited by 79
Generative, High-Fidelity Network Traces
Jiang, X., Liu, S., Gember-Jacobson, A., Schmitt, P., Bronzino, F., & Feamster, N. (2023) · ACM Workshop on Hot Topics in Networks (HotNets) · doi:10.1145/3626111.3628196 · Cited by 19
Trust-Free Blockchain Framework for AI-Generated Content Trading and Management in Metaverse
Truong, V. T., Le, H. D., & Le, L. B. (2024) · IEEE Access · doi:10.1109/access.2024.3376509 · Cited by 19
Personalized Face Inpainting with Diffusion Models by Parallel Visual Attention
Xu, J., Motamed, S., Vaddamanu, P., Wu, C., Häne, C., Bazin, J., & la Torre, F. D. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00535 · Cited by 21
An artificial intelligence approach for interpreting creative combinational designs
Chen, L., Xiao, S., Chen, Y., Sun, L., Childs, P., & Han, J. (2024) · Journal of Engineering Design · doi:10.1080/09544828.2024.2377068 · Cited by 22
IntrinsicDiffusion: Joint Intrinsic Layers from Latent Diffusion Models
Luo, J., Ceylan, D., Yoon, J. S., Zhao, N., Philip, J., Frühstück, A., Li, W., Richardt, C., & Wang, T. Y. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657472 · Cited by 19
TexPainter: Generative Mesh Texturing with Multi-view Consistency
Zhang, H., Pan, Z., Zhang, C., Zhu, L., & Gao, X. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657494 · Cited by 19
Exploring Generative AI for Sim2Real in Driving Data Synthesis
Zhao, H., Wang, Y., Bashford‐Rogers, T., Donzella, V., & Debattista, K. (2024) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv55156.2024.10588493 · Cited by 21
Text-to-3D Generation with Bidirectional Diffusion Using Both 2D and 3D Priors
Ding, L., Dong, S., Huang, Z., Wang, Z., Zhang, Y., Gong, K., Xu, D., & Xue, T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00489 · Cited by 25
ASAM: Boosting Segment Anything Model with Adversarial Tuning
Li, B. H., Xiao, H., & Tang, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00355 · Cited by 24
ZeST: Zero-Shot Material Transfer from a Single Image
Cheng, T.-Y., Sharma, P., Markham, A., Trigoni, N., & Jampani, V. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73232-4_21 · Cited by 28
HeartBeat: Towards Controllable Echocardiography Video Synthesis with Multimodal Conditions-Guided Diffusion Models
Zhou, X., Huang, Y., Xue, W., Dou, H., Cheng, J., Zhou, H., & Ni, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72104-5_35 · Cited by 24
Deep Learning-Based Weather-Related Power Outage Prediction with Socio-Economic and Power Infrastructure Data
Wang, X., Fatehi, N., Wang, C., & Nazari, M. H. (2024) · IEEE Power Engineering Society General Meeting · doi:10.1109/pesgm51994.2024.10688596 · Cited by 19
SpaceBlender: Creating Context-Rich Collaborative Spaces Through Generative 3D Scene Blending
Numan, N., Rajaram, S., Kumaravel, B. T., Marquardt, N., & Wilson, A. D. (2024) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3654777.3676361 · Cited by 19
ReactFace: Online Multiple Appropriate Facial Reaction Generation in Dyadic Interactions
Luo, C. X., Song, S., Xie, W., Spitale, M., Ge, Z., Shen, L., & Güneş, H. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3490613 · Cited by 34
LCM-Lookahead for Encoder-Based Text-to-Image Personalization
Gal, R., Lichter, O., Richardson, E., Patashnik, O., Bermano, A. H., Chechik, G., & Cohen‐Or, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72630-9_19 · Cited by 19
Wear-Any-Way: Manipulable Virtual Try-on via Sparse Correspondence Alignment
Chen, M., Chen, X., Zhai, Z., Ju, C., Hong, X., Lan, J., & Xiao, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72630-9_8 · Cited by 19
An Image is Worth Multiple Words: Multi-Attribute Inversion for Constrained Text-To-Image Synthesis
Agarwal, A., Karanam, S., Shukla, T., & Srinivasan, B. V. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00590 · Cited by 29
Semantic-Aware Power Allocation for Generative Semantic Communications with Foundation Models
Xu, C., Mashhadi, M. B., Ma, Y., & Tafazolli, R. (2024) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/gcwkshp64532.2024.11100549 · Cited by 27
A Survey on Personalized Content Synthesis with Diffusion Models
Zhang, X.-L., Wei, X.-Y., Hu, W., Wu, J., Wu, J., Zhang, W., Zhang, Z., Lei, Z., & Li, Q. (2025) · Machine Intelligence Research · doi:10.1007/s11633-025-1563-3 · Cited by 19
Survey on Visual Signal Coding and Processing With Generative Models: Technologies, Standards, and Optimization
Chen, Z., Sun, H., Zhang, L., & Zhang, F. (2024) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2024.3403524 · Cited by 20
Self-citation
Universal Fingerprint Generation: Controllable Diffusion Model With Multimodal Conditions
Grosz, S. A., & Jain, A. K. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3486179 · Cited by 19
Tackling Structural Hallucination in Image Translation with Local Diffusion
Kim, S., Jin, C., Diethe, T., Figini, M., Tregidgo, H. F. J., Mullokandov, A., Teare, P., & Alexander, D. C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73004-7_6 · Cited by 18
Efficient End-to-End Diffusion Model for One-Step SAR-to-Optical Translation
Qin, J., Zou, B., Li, H., & Zhang, L. (2024) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2024.3506566 · Cited by 20
Self-citation
SketchFlex: Facilitating Spatial-Semantic Coherence in Text-to-Image Generation with Region-Based Sketches
Lin, H., Ye, Y., Xia, J., & Zeng, W. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713801 · Cited by 28
Sketch2Anim: Towards Transferring Sketch Storyboards into 3D Animation
Zhong, L., Guo, C., Xie, Y., Wang, J., & Li, C. (2025) · ACM Transactions on Graphics · doi:10.1145/3731167 · Cited by 18
SleeperMark: Towards Robust Watermark against Fine-Tuning Text-to-image Diffusion Models
Wang, Z., Guo, J., Zhu, J., Li, Y., Huang, H., Chen, M., & Tu, Z. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00769 · Cited by 18
From Fragment to One Piece: A Review on AI-Driven Graphic Design
Zou, X., Zhang, W., & Zhao, N. (2025) · Journal of Imaging · doi:10.3390/jimaging11090289 · Cited by 25
Diffusion Models for Agricultural Imaging: A Systematic Review of Methods, Applications and Future Prospects
Zangana, H. M., Li, S., & Wani, S. (2025) · Impact in Agriculture · doi:10.65500/agriculture-2025-003 · Cited by 18
Modulating Pretrained Diffusion Models for Multimodal Image Synthesis
Ham, C., Hays, J. H., Lu, J., Singh, K. K., Zhang, Z., & Hinz, T. (2023) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3588432.3591549 · Cited by 25
A Realistic Synthetic Mushroom Scenes Dataset
Anagnostopoulou, D., Retsinas, G., Efthymiou, N., Filntisis, P. P., & Maragos, P. A. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00668 · Cited by 18
FashionDiff: A Controllable Diffusion Model Using Pairwise Fashion Elements for Intelligent Design
Yan, H., Zhang, H., Mu, X., Fan, J., & Zhang, Z. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612127 · Cited by 18
Masked-attention diffusion guidance for spatially controlling text-to-image generation
Endo, Y. (2023) · The Visual Computer · doi:10.1007/s00371-023-03151-y · Cited by 18
Text-Guided Synthesis of Eulerian Cinemagraphs
Mahapatra, A., Siarohin, A., Lee, H.-Y., Tulyakov, S., & Zhu, J.-Y. (2023) · ACM Transactions on Graphics · doi:10.1145/3618326 · Cited by 20
Label Freedom: Stable Diffusion for Remote Sensing Image Semantic Segmentation Data Generation
Zhao, C., Ogawa, Y., Chen, S., Yang, Z., & Sekimoto, Y. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386381 · Cited by 18
SSMG: Spatial-Semantic Map Guided Diffusion Model for Free-Form Layout-to-Image Generation
Jia, C., Luo, M., Dang, Z., Dai, G., Chang, X., Wang, M., & Wang, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i3.28024 · Cited by 23
HD-Fusion: Detailed Text-to-3D Generation Leveraging Multiple Noise Estimation
Wu, J., Gao, X., Liu, X., Shen, Z., Zhao, C. X., Feng, H., Liu, J., & Ding, E. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00317 · Cited by 27
SAW-GAN: Multi-granularity Text Fusion Generative Adversarial Networks for text-to-image generation
Jin, D., Yu, Q., Yu, L., & Qi, M. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.111795 · Cited by 18
Denoising Diffusion Straightforward Models for Energy Conversion Monitoring Data Imputation
Xu, H., Liu, Z., Wang, H., Li, C., Niu, Y., Wang, W., & Liu, X. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3413349 · Cited by 21
Edit Temporal-Consistent Videos with Image Diffusion Model
Wang, Y., Li, Y., Zhang, X., Liu, X., Dai, A., Chan, A. B., & Cui, Z. (2024) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3691344 · Cited by 20

Authors: Zhang, L., Rao, A., & Agrawala, M.
Affiliations: Stanford University (US)
ORCID: Zhang, Lvmin
Rao, Anyi
Agrawala, Maneesh
Venue: 2023 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher: IEEE
Published: 2023-10-01
Pages: 3813-3824
DOI: 10.1109/iccv51070.2023.00355
OpenAlex: W4390873054
Cited by: 4,547
References: 91
Citations by year: 2026: 1290 · 2025: 1847 · 2024: 1255 · 2023: 155
FWCI: 486.28
Type: conference paper
Language: en
Open access: closed
Licence: doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
Funders: Brown Institute for Media Innovation
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords: ControlNet, text-to-image diffusion models, spatial conditioning, conditional image generation, Stable Diffusion, parameter-efficient fine-tuning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Adding Conditional Control to Text-to-Image Diffusion Models

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks