Cited by 4,547Generative Adversarial Networks and Image SynthesisAdvanced Neural Network ApplicationsMultimodal Machine Learning Applications

The DOI matches its Crossref record; 88 of the 91 references it cites were found in a registry or an index.

Adding Conditional Control to Text-to-Image Diffusion Models

Zhang, Lvmin, Rao, Anyi, Agrawala, Maneesh·2023·2023 IEEE/CVF International Conference on Computer Vision (ICCV)

DOI 10.1109/iccv51070.2023.00355

Open this paper in CoChat — free account

Abstract

We present ControlNet, a neural network architecture to add spatial conditioning controls to large, pretrained text-to-image diffusion models. ControlNet locks the production-ready large diffusion models, and reuses their deep and robust encoding layers pretrained with billions of images as a strong backbone to learn a diverse set of conditional controls. The neural architecture is connected with "zero convolutions" (zero-initialized convolution layers) that progressively grow the parameters from zero and ensure that no harmful noise could affect the finetuning. We test various conditioning controls, e.g., edges, depth, segmentation, human pose, etc., with Stable Diffusion, using single or multiple conditions, with or without prompts. We show that the training of ControlNets is robust with small (1m) datasets. Extensive results show that ControlNet may facilitate wider applications to control image diffusion models.

Integrity signals

Registry

Crossref · 10.1109/iccv51070.2023.00355

Cites

88 of 88 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Deep learning
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
A Computational Approach to Edge Detection
Canny, J. (1986) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1986.4767851 · Cited by 30,004
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881

Show all 91 references

Research works printed but not linked: 13

2022, Disco diffusion
2022, Protogen x3.4 (photorealism) official release
Ha, Hypernetworks
2022, Hypernetwork style training, a tiny guide, stable-diffusion-webui
10.1109/TPAMI.2020.2970919 · DOI · Paper page
2022, Novelai improvements on stable diffusion
Lehtinen, Noise2noise: Learning image restoration without clean data
2023, Midjourney
2022, Comic-diffusion v2, trained on 6 styles at once
2023, Dall-e-2
Schuhmann, LAION-5b: An open large-scale dataset for training next generation image-text models
2022, Stable diffusion v1.5 model card
2022, Stable diffusion v2 model card, stable-diffusion-2-depth

Sorted by citations

Items 701–800 of 1,012

Zero-Shot Structure-Preserving Diffusion Model for High Dynamic Range Tone Mapping
Zhu, R., Xu, S., Liu, P., Li, S., Lu, Y., Niu, D., Liu, Z., Meng, Z., Li, Z., Chen, X., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02469 · Cited by 18
Shadow Generation for Composite Image Using Diffusion Model
Liu, Q., You, J., Wang, J., Tao, X., Zhang, B., & Niu, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00776 · Cited by 22
Holo-Relighting: Controllable Volumetric Portrait Relighting from a Single Image
Mei, Y., Zeng, Y., Zhang, H., Shu, Z., Zhang, X., Bi, S., Zhang, J., Jung, H., & Patel, V. M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00408 · Cited by 18
PICTURE: PhotorealistIC Virtual Try-on from UnconstRained dEsigns
Ning, S., Wang, D., Qin, Y., Jin, Z., Wang, B., & Han, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00666 · Cited by 18
Curriculum Point Prompting for Weakly-Supervised Referring Image Segmentation
Dai, Q., & Yang, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01301 · Cited by 24
Bayesian Diffusion Models for 3D Shape Reconstruction
Xu, H., Yu, L., Chen, Z., Zhang, X., Zhao, Y., Wang, Y., & Tu, Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01011 · Cited by 22
CustomNet: Object Customization with Variable-Viewpoints in Text-to-Image Diffusion Models
Yuan, Z., Cao, M., Wang, X., Qi, Z., Yuan, C., & Shan, Y. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681396 · Cited by 18
Reliability in Semantic Segmentation: Can We Use Synthetic Data?
Loiseau, T., Vu, T.-H., Chen, M., Pérez, P., & Cord, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73337-6_25 · Cited by 22
RodinHD: High-Fidelity 3D Avatar Generation with Diffusion Models
Zhang, B., Cheng, Y., Wang, C., Zhang, T., Yang, J., Tang, Y., Zhao, F., Chen, D., & Guo, B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72630-9_27 · Cited by 18
Global multi-scale extraction and local mixed multi-head attention for facial expression recognition in the wild
Fei, Z., Zhang, B., Zhou, W., Li, X., Zhang, Y., & Fei, M. (2025) · Neurocomputing · doi:10.1016/j.neucom.2024.129323 · Cited by 18
StableMakeup: When Real-World Makeup Transfer Meets Diffusion Model
Zhang, Y., Yuan, Y., Song, Y., & Liu, J. (2025) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3721238.3730702 · Cited by 18
Magic Fixup: Streamlining Photo Editing by Watching Dynamic Videos
Alzayer, H., Xia, Z., Zhang, X., Shechtman, E., Huang, J., & Gharbi, M. (2025) · ACM Transactions on Graphics · doi:10.1145/3750722 · Cited by 30
DreamStory: Open-Domain Story Visualization by LLM-Guided Multi-Subject Consistent Diffusion
He, H., Yang, H., Tuo, Z., Zhou, Y., Wang, Q., Zhang, Y., Liu, Z., Huang, W., Chao, H., & Yin, J. P. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3600149 · Cited by 35
Building a Chinese ancient architecture multimodal dataset combining image, annotation and style-model
Li, B., Feng, J., Yan, Y., Kou, G., Li, H., Du, Y., Wang, X., Li, T., Peng, Y., Guo, K., et al. (2024) · Scientific Data · doi:10.1038/s41597-024-03946-1 · Cited by 18
A Survey of Multimodal Controllable Diffusion Models
Jiang, R., Zheng, G., Li, T., Yang, T., Wang, J., & Li, X. (2024) · Journal of Computer Science and Technology · doi:10.1007/s11390-024-3814-0 · Cited by 22
Generating HSR Bogie Vibration Signals via Pulse Voltage-Guided Conditional Diffusion Model
Liu, X., Chen, J., Xie, J., & Chang, Y. (2024) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2024.3482106 · Cited by 18
MoA: Mixture-of-Attention for Subject-Context Disentanglement in Personalized Image Generation
Wang, K.-C. J., Ostashev, D., Fang, Y., Tulyakov, S., & Aberman, K. (2024) · SIGGRAPH Asia Conference Papers · doi:10.1145/3680528.3687662 · Cited by 17
D4: Text-guided diffusion model-based domain adaptive data augmentation for vineyard shoot detection
Hirahara, K., Nakane, C., Ebisawa, H., Kuroda, T., Iwaki, Y., Utsumi, T., Nomura, Y., Koike, M., & Mineno, H. (2024) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2024.109849 · Cited by 17
Self‐Driving Microscopes: AI Meets Super‐Resolution Microscopy
Ward, E. N., Scheeder, A., Barysevich, M., & Kaminski, C. (2025) · Small Methods · doi:10.1002/smtd.202401757 · Cited by 14
Brain Latent Progression: Individual-based spatiotemporal disease progression on 3D Brain MRIs via latent diffusion
Puglisi, L., Alexander, D. C., & Ravì, D. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103734 · Cited by 26
BooW-VTON: Boosting In-the-Wild Virtual Try-On via Mask-Free Pseudo Data Training
Zhang, X., Song, D., Zhan, P., Chang, T., Zeng, J., Chen, Q.-G., Luo, W., & Liu, A.-A. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02458 · Cited by 17
Generative Artificial Intelligence in the Metaverse Era: A Review on Models and Applications
Zhou, H., Chen, X., Li, J., Zhang, Z., Fu, Y., Pérez-Liva, M., Greenbaum, D. S., & Hui, P. (2025) · Research · doi:10.34133/research.0804 · Cited by 21
Foundation Models and Biometrics: A Survey and Outlook
Shahreza, H. O., & Marcel, S. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3602233 · Cited by 20
MaskControl: Spatio-Temporal Control for Masked Motion Synthesis
Pinyoanuntapong, E., Saleem, M. U., Karunratanakul, K., Wang, P., Xue, H., Chen, C., Guo, C., Cao, J., Ren, J., & Tulyakov, S. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.00928 · Cited by 17
ViT-TTS: Visual Text-to-Speech with Scalable Diffusion Transformer
Liu, H., Huang, R., Lin, X., Xu, W., Zheng, M., Chen, H., He, J., & Zhao, Z. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.990 · Cited by 35
DiffMat: Latent diffusion models for image-guided material generation
Yuan, L., Yan, D., Saito, S., & Fujishiro, I. (2024) · Visual Informatics · doi:10.1016/j.visinf.2023.12.001 · Cited by 17
DiffGuard: Semantic Mismatch-Guided Out-of-Distribution Detection using Pre-trained Diffusion Models
Gao, R., Zhao, C., Hong, L., & Xu, Q. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00152 · Cited by 18
Controllable Text-to-Image Synthesis for Multi-Modality MR Images
Kim, K., Na, Y., Ye, S., Lee, J., Ahn, S. S., Park, J. E., & Kim, H. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00775 · Cited by 22
Identity-Preserving Face Swapping via Dual Surrogate Generative Models
Huang, Z., Tang, F., Zhang, Y., Cao, J., Li, C., Tang, S., Li, J., & Lee, T. (2024) · ACM Transactions on Graphics · doi:10.1145/3676165 · Cited by 17
Empower Generalizability for Pansharpening Through Text-Modulated Diffusion Model
Xing, Y., Qu, L., Zhang, S., Feng, J., Zhang, X., & Zhang, Y. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3434431 · Cited by 20
Learning to reconstruct accelerated MRI through K-space cold diffusion without noise
Shen, G., Li, M., Farris, C. W., Anderson, S. W., & Zhang, X. (2024) · Scientific Reports · doi:10.1038/s41598-024-72820-2 · Cited by 18
Readout Guidance: Learning Control from Diffusion Features
Luo, G., Darrell, T., Wang, O., Goldman, D. B., & Holynski, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00785 · Cited by 35
FreeMan: Towards Benchmarking 3D Human Pose Estimation Under Real-World Conditions
Wang, J., Yang, F., Li, B., Gou, W., Yan, D., Zeng, A., Gao, Y., Wang, J., Jing, Y., & Zhang, R. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02075 · Cited by 17
GenN2N: Generative NeRF2NeRF Translation
Liu, X., Han, X., Luo, K., Tan, P., & Yi, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00488 · Cited by 19
Accurate and Efficient Object Pose Estimation via the Aggregation of Diffusion Features
Wang, T., Hu, G., & Wang, H. G. (2024) · International Journal of Computer Vision · doi:10.1109/cvpr52733.2024.00975 · Cited by 17
Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Prior
Cheng, C., Yang, X., Yang, F., Feng, C., Fu, Z., Foo, C.-S., Lin, G., & Liu, F. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00974 · Cited by 25
Put Myself in Your Shoes: Lifting the Egocentric Perspective from Exocentric Videos
Luo, M., Xue, Z., Dimakis, A. G., & Grauman, K. L. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72920-1_23 · Cited by 27
SDA: Low-Bit Stable Diffusion Acceleration on Edge FPGAs
Yang, G., Xie, Y., Xue, Z. J., Chang, S.-E., Li, Y., Dong, P., Lei, J., Xie, W., Wang, Y., Lin, X., et al. (2024) · International Conference on Field Programmable Logic and Applications (FPL) · doi:10.1109/fpl64840.2024.00044 · Cited by 17
DifFRelight: Diffusion-Based Facial Performance Relighting
He, M., Clausen, P., Taşel, A. L., Ma, L., Pilarski, O., Xian, W., Rikker, L., Yu, X., Burgert, R., Yu, N., et al. (2024) · SIGGRAPH Asia Conference Papers · doi:10.1145/3680528.3687644 · Cited by 33
Automated construction site layout design system for prefabricated buildings using transformer based conditional GAN
Yang, Y., Chen, C., & Li, T. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102885 · Cited by 17
DMDAT: Diffusion Model-Based Data Augmentation Technique for Vision-Based Accident Detection in Vehicular Networks
Sai, S., Mittal, U., & Chamola, V. (2024) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2024.3491170 · Cited by 17
SPIRE: Semantic Prompt-Driven Image Restoration
Qi, C., Tu, Z., Ye, K., Delbracio, M., Milanfar, P., Chen, Q., & Talebi, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73661-2_25 · Cited by 17
ArchiDiffusion: A novel diffusion model connecting architectural layout generation from sketches to Shear Wall Design
Leng, H., Gao, Y., & Zhou, Y. (2024) · Journal of Building Engineering · doi:10.1016/j.jobe.2024.111373 · Cited by 19
On the Trustworthiness Landscape of State-of-the-art Generative Models: A Survey and Outlook
Fan, M., Wang, C., Chen, C., Liu, Y., & Huang, J. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02375-w · Cited by 40
Advances in 3D Neural Stylization: A Survey
Chen, Y., Shao, G., Shum, K. C., Hua, B., & Yeung, S.-K. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02403-9 · Cited by 21
GeoDiffuser: Geometry-Based Image Editing with Diffusion Models
Sajnani, R., Vanbaar, J., Min, J., Katyal, K. D., & Sridhar, S. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00056 · Cited by 17
EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation
Meng, R., Zhang, X., Li, Y., & Ma, C. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00516 · Cited by 17
StableIdentity: Inserting Anybody Into Anywhere at First Sight
Wang, Q., Jia, X., Li, X., Li, T., Ma, L., Zhuge, Y., & Lu, H. (2025) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2025.3613113 · Cited by 35
Boosting GUI Prototyping with Diffusion Models
Wei, J., Courbis, A.-L., Lambolais, T., Xu, B., Bernard, P. L., & Dray, G. (2023) · IEEE International Requirements Engineering Conference (RE) · doi:10.1109/re57278.2023.00035 · Cited by 27
Cednet: A Cascade Encoder-Decoder Network for Dense PredictionPreprint
Zhang, G., Li, Z., Tang, C., Li, J., & Hu, X. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4857945 · Cited by 17
MagicDrive-V2: High-Resolution Long Video Generation for Autonomous Driving with Adaptive Control
Gao, R., Chen, K., Xiao, B., Hong, L., Li, Z., & Xu, Q. (2024) · arXiv · doi:10.48550/arxiv.2411.13807 · Cited by 16
Compositional Structures as Substrates for Human-AI Co-creation Environment: A Design Approach and A Case Study
Cao, Y., Huang, Y., Truong, A., Shin, H. V., & Xia, H. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713401 · Cited by 16
A Few-Shot Steel Surface Defect Generation Method Based on Diffusion Models
Li, H., Liu, Y., Liu, C., Pang, H., & Xu, K. (2025) · Sensors · doi:10.3390/s25103038 · Cited by 22
Artworks Reimagined: Exploring Human-AI Co-Creation through Body Prompting
Oppenlaender, J., Johnston, H., Silvennoinen, J. M., & Barranha, H. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3734189 · Cited by 20
EditSplat: Multi-View Fusion and Attention-Guided Optimization for View-Consistent 3D Scene Editing with 3D Gaussian Splatting
Lee, D. I., Park, H., Seo, J., Park, E., Park, H., Baek, H.-G., Shin, S.-H., Kim, S., & Kim, S. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01040 · Cited by 25
A Systematic Review of U‐Net Optimizations: Advancing Tumour Segmentation in Medical Imaging
Abueed, O., Wang, Y., & Khasawneh, M. T. (2025) · IET Image Processing · doi:10.1049/ipr2.70203 · Cited by 16
Flow Matching for Medical Image Synthesis: Bridging the Gap Between Speed and Quality
Yazdani, M., Medghalchi, Y., Ashrafian, P., Hacihaliloglu, I., & Shahriari, D. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05325-1_21 · Cited by 16
Thing2Reality: Enabling Spontaneous Creation of 3D Objects from 2D Content using Generative AI in XR Meetings
Hu, E., Li, M., Hong, J., Qian, X., Olwal, A., Kim, D., Heo, S., & Du, R. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747621 · Cited by 16
Harnessing Diffusion-Yielded Score Priors for Image Restoration
Lin, X., Yu, F., Hu, J., You, Z., Shi, W., Ren, J., Gu, J., & Dong, C. (2025) · ACM Transactions on Graphics · doi:10.1145/3763346 · Cited by 30
DiffuMural: a diffusion model for dunhuang murals restoration via multi-scale convergence and cooperative guidance
Han, P., Pan, Y., Pan, E., Jin, Q., Jiang, J., Zhang, Z., Liu, Z., Kang, J., & Gong, L. (2026) · npj Heritage Science · doi:10.1038/s40494-025-02274-w · Cited by 17
Multi-scale conditional diffusion model for deposited droplet volume measurement in inkjet printing manufacturing
Zhang, Z., Yang, H. S., Chen, J., & Yin, Z. (2023) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2023.10.004 · Cited by 17
Deep Geometrized Cartoon Line Inbetweening
Li, S., Gu, T., Xiao, W., Ding, H., Liu, Z., & Loy, C. C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00670 · Cited by 16
GlueGen: Plug and Play Multi-modal Encoders for X-to-image Generation
Qin, C., Yu, N., Chen, X., Zhang, S., Chen, Z., Ermon, S., Fu, Y., Xiong, C., & Xu, R. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02110 · Cited by 24
DreamIdentity: Enhanced Editability for Efficient Face-Identity Preserved Image Generation
Chen, Z., Fang, S., Liu, W., He, Q., Huang, M., & Mao, Z. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i2.27891 · Cited by 16
SphereDiffusion: Spherical Geometry-Aware Distortion Resilient Diffusion Model
Wu, T., Li, X., Qi, Z., Hu, D., Wang, X., Shan, Y., & Xi, L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i6.28429 · Cited by 19
Diffusion-Based Generative Network for de Novo Synthetic Promoter Design
Lin, J., Wang, X., Liu, T., Teng, Y., & Cui, W. (2024) · ACS Synthetic Biology · doi:10.1021/acssynbio.4c00041 · Cited by 16
Conditioning diffusion models via attributes and semantic masks for face generation
Lisanti, G., & Giambi, N. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.104026 · Cited by 19
Learning to Generate Urban Design Images From the Conditional Latent Diffusion Model
Cui, X., Feng, X. M., & Sun, S. (2024) · IEEE Access · doi:10.1109/access.2024.3419159 · Cited by 18
Cascaded learning with transformer for simultaneous eye landmark, eye state and gaze estimation
Gou, C., Yu, Y., Guo, Z., Chen, X., & Cai, M. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110760 · Cited by 18
Streetscapes: Large-scale Consistent Street View Generation Using Autoregressive Video Diffusion
Deng, B., Tucker, R. P., Li, Z., Guibas, L. J., Snavely, N., & Wetzstein, G. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657513 · Cited by 26
MaPa: Text-driven Photorealistic Material Painting for 3D Shapes
Zhang, S., Peng, S., Xu, T., Yang, Y., Chen, T., Xue, N., Shen, Y., Bao, H., Hu, R., & Zhou, X. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657504 · Cited by 28
DiffMap: Enhancing Map Segmentation With Map Prior Using Diffusion Model
Jia, P., Wen, T., Luo, Z., Yang, M., Jiang, K., Liu, Z., Tang, X., Lei, Z., Cui, L., Zhang, B., et al. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3455853 · Cited by 19
DCG-GAN: design concept generation with generative adversarial networks
Ghasemi, P., Yuan, C., Marion, T. J., & Moghaddam, M. (2024) · Design Science · doi:10.1017/dsj.2024.13 · Cited by 16
DiffPortrait3D: Controllable Diffusion for Zero-Shot Portrait View Synthesis
Gu, Y., Xu, H., Xie, Y. X., Song, G., Shi, Y., Chang, D., Yang, J., & Luo, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00995 · Cited by 21
3DiffTection: 3D Object Detection with Geometry-Aware Diffusion Features
Xu, C., Ling, H., Fidler, S., & Litany, O. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01010 · Cited by 23
APISR: Anime Production Inspired Real-World Anime Super-Resolution
Wang, B., Yang, F., Yu, X., Zhang, C., & Zhao, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02416 · Cited by 18
SCEdit: Efficient and Controllable Image Diffusion Generation via Skip Connection Editing
Jiang, Z., Mao, C., Pan, Y., Han, Z., & Zhang, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00859 · Cited by 23
Robust-Wide: Robust Watermarking Against Instruction-Driven Image Editing
Hu, R., Zhang, J., Xu, T., Li, J., & Zhang, T. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72670-5_2 · Cited by 22
Generative Models for the Psychology of Art and Aesthetics
Hertzmann, A. (2024) · Empirical Studies of the Arts · doi:10.1177/02762374241288696 · Cited by 19
Safe-SD: Safe and Traceable Stable Diffusion with Text Prompt Trigger for Invisible Generative Watermarking
Ma, Z., Jia, G., Qi, B., & Zhou, B. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681418 · Cited by 22
MagicFight: Personalized Martial Arts Combat Video Generation
Huang, J., Yan, M., Chen, S., Huang, Y., & Chen, S. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680849 · Cited by 17
SmartControl: Enhancing ControlNet for Handling Rough Visual Conditions
Liu, X., Wei, Y., Liu, M., Lin, X., Ren, P., Xie, X., & Zuo, W. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73195-2_1 · Cited by 16
Style-NeRF2NeRF: 3D Style Transfer from Style-Aligned Multi-View Images
Fujiwara, H., Mukuta, Y., & Harada, T. (2024) · SIGGRAPH Asia Conference Papers · doi:10.1145/3680528.3687643 · Cited by 17
Read, Watch and Scream! Sound Generation from Text and Video
Jeong, Y., Kim, Y., Chun, S., & Lee, J.-Y. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i17.33934 · Cited by 36
Security and Privacy Challenges of AIGC in Metaverse: A Comprehensive Survey
Zhang, S., Li, H., Sun, K., Chen, H., Wang, Y., & Li, S. (2025) · ACM Computing Surveys · doi:10.1145/3729419 · Cited by 17
A conditional probabilistic deep learning approach for anomaly detection in structures under varying environmental conditions
Zhu, Y.-K., Wan, H., Ye, Y., & Todd, M. D. (2025) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2025.113005 · Cited by 16
LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relighting
Xing, X., Groh, K., Karaoğlu, S., Gevers, T., & Bhattad, A. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00050 · Cited by 16
Generating a novel synthetic dataset for rehabilitation exercises using pose-guided conditioned diffusion models: A quantitative and qualitative evaluation
Mennella, C., Maniscalco, U., Pietro, G. D., & Esposito, M. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107665 · Cited by 16
SignLLM: Sign Language Production Large Language Models
Fang, S., Chen, C., Wang, L., Zheng, C., Sui, C., & Tian, Y. (2025) · 2025 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw69036.2025.00690 · Cited by 15
Exploring Designers’ Perceptions and Practices of Collaborating with Generative AI as a Co-creative Agent in a Multi-stakeholder Design Process: Take the Domain of Avatar Design as an Example
He, Q., Zheng, W., Bao, H., Chen, R., & Tong, X. (2023) · Proceedings of the Eleventh International Symposium of Chinese CHI · doi:10.1145/3629606.3629675 · Cited by 15
From attributes to natural language: A survey and foresight on text-based person re-identification
Jiang, F., Yang, S., Jones, M. W., & Zhang, L. (2025) · Information Fusion · doi:10.1016/j.inffus.2024.102879 · Cited by 18
Self-citation
Balancing user preferences by social networks: A condition-guided social recommendation model for mitigating popularity bias
He, X., Fan, W., Wang, R., Wang, Y., Wang, X. Y., Pan, S., & Wang, X. Y. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107317 · Cited by 18
PointSea: Point Cloud Completion via Self-structure Augmentation
Zhu, Z., Chen, H., He, X., & Wei, M. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02400-y · Cited by 20
FusAIn: Composing Generative AI Visual Prompts Using Pen-based Interaction
Peng, X., Koch, J., & Mackay, W. E. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714027 · Cited by 20
About Time: Advances, Challenges, and Outlooks of Action Understanding
Stergiou, A., & Poppe, R. W. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02478-4 · Cited by 17
Text semantics to controllable design: A residential layout generation method based on stable diffusion model
Qiu, Z., Liu, J., Xia, Y., Qi, H., & Liu, P. (2025) · Developments in the Built Environment · doi:10.1016/j.dibe.2025.100691 · Cited by 15
CineMaster: A 3D-Aware and Controllable Framework for Cinematic Text-to-Video Generation
Wang, Q., Luo, Y., Shi, X., Jia, X., Lu, H., Xue, T., Wang, X., Wan, P., Zhang, D., & Gai, K. (2025) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3721238.3730755 · Cited by 34
Identity-Preserving Text-To-Video Generation by Frequency Decomposition
Yuan, S., Huang, J., He, X., Ge, Y., Shi, Y., Chen, L., Luo, J., & Li, Y. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01211 · Cited by 25
Dual Prompting Image Restoration with Diffusion Transformers
Kong, D., Li, F., Wang, Z., Xu, J., Pei, R., Li, W., & Ren, W. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01195 · Cited by 23
Latent Space Super-Resolution for Higher-Resolution Image Generation with Diffusion Models
Jeong, J., Han, S., Kim, J., & Kim, S. J. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00225 · Cited by 15

Authors: Zhang, L., Rao, A., & Agrawala, M.
Affiliations: Stanford University (US)
ORCID: Zhang, Lvmin
Rao, Anyi
Agrawala, Maneesh
Venue: 2023 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher: IEEE
Published: 2023-10-01
Pages: 3813-3824
DOI: 10.1109/iccv51070.2023.00355
OpenAlex: W4390873054
Cited by: 4,547
References: 91
Citations by year: 2026: 1290 · 2025: 1847 · 2024: 1255 · 2023: 155
FWCI: 486.28
Type: conference paper
Language: en
Open access: closed
Licence: doi.org/10.15223/policy-029 (stm-asf)
doi.org/10.15223/policy-037 (stm-asf)
Locations: IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
Funders: Brown Institute for Media Innovation
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords: ControlNet, text-to-image diffusion models, spatial conditioning, conditional image generation, Stable Diffusion, parameter-efficient fine-tuning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Adding Conditional Control to Text-to-Image Diffusion Models

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks