The DOI matches its Crossref record; 88 of the 91 references it cites were found in a registry or an index.
Adding Conditional Control to Text-to-Image Diffusion Models
Abstract
We present ControlNet, a neural network architecture to add spatial conditioning controls to large, pretrained text-to-image diffusion models. ControlNet locks the production-ready large diffusion models, and reuses their deep and robust encoding layers pretrained with billions of images as a strong backbone to learn a diverse set of conditional controls. The neural architecture is connected with "zero convolutions" (zero-initialized convolution layers) that progressively grow the parameters from zero and ensure that no harmful noise could affect the finetuning. We test various conditioning controls, e.g., edges, depth, segmentation, human pose, etc., with Stable Diffusion, using single or multiple conditions, with or without prompts. We show that the training of ControlNets is robust with small (1m) datasets. Extensive results show that ControlNet may facilitate wider applications to control image diffusion models.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Image annotations, including prompts and spatial cues [92, 121], are essential for training and evaluating image generation models as they provide localized and semantically rich supervision that help models learn meaningful visual representations.”
From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [121]
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
A Computational Approach to Edge Detection
Canny, J. (1986) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1986.4767851 · Cited by 30,004
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881
Research works printed but not linked: 13
- 2022, Disco diffusion
- 2022, Protogen x3.4 (photorealism) official release
- Ha, Hypernetworks
- 2022, Hypernetwork style training, a tiny guide, stable-diffusion-webui
- 10.1109/TPAMI.2020.2970919 · DOI · Paper page
- 2022, Novelai improvements on stable diffusion
- Lehtinen, Noise2noise: Learning image restoration without clean data
- 2023, Midjourney
- 2022, Comic-diffusion v2, trained on 6 styles at once
- 2023, Dall-e-2
- Schuhmann, LAION-5b: An open large-scale dataset for training next generation image-text models
- 2022, Stable diffusion v1.5 model card
- 2022, Stable diffusion v2 model card, stable-diffusion-2-depth
Cited by
Sorted by citations
Items 101–200 of 1,012
Efficient Diffusion Model for Image Restoration by Residual Shifting
Yue, Z., Wang, J., & Loy, C. C. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3461721 · Cited by 126
InstanceDiffusion: Instance-Level Control for Image Generation
Wang, X., Darrell, T., Rambhatla, S. S., Girdhar, R., & Misra, I. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00596 · Cited by 139
UAVs meet LLMs: Overviews and perspectives towards agentic low-altitude mobility
Tian, Y., Lin, F., Li, Y., Zhang, T., Zhang, Q., Fu, X., Huang, J., Dai, X., Wang, Y., Tian, C., et al. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103158 · Cited by 139
Diffusion model with disentangled modulations for sharpening multispectral and hyperspectral images
Cao, Z., Cao, S., Deng, L.-J., Wu, X., Hou, J., & Vivone, G. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102158 · Cited by 105
Multiscale Structure Guided Diffusion for Image Deblurring
Ren, M., Delbracio, M., Talebi, H., Gerig, G., & Milanfar, P. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00984 · Cited by 118
AI-Generated Content (AIGC) for Various Data Modalities: A Survey
Foo, L. G., Rahmani, H., & Liu, J. (2025) · ACM Computing Surveys · doi:10.1145/3728633 · Cited by 119
TexFusion: Synthesizing 3D Textures with Text-Guided Image Diffusion Models
Cao, T., Kreis, K., Fidler, S., Sharp, N., & Yin, K. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00385 · Cited by 104
Yang, Z., Peng, D., Kong, Y., Zhang, Y., Yao, C., & Jin, L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i7.28482 · Cited by 109
Grounded Text-to-Image Synthesis with Attention Refocusing
Phung, Q., Ge, S., & Huang, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00758 · Cited by 135
Data Augmentation for Object Detection via Controllable Diffusion Models
Fang, H., Han, B., Zhang, S., Zhou, S., Hu, C., & Ye, W.-M. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00129 · Cited by 103
TeCH: Text-Guided Reconstruction of Lifelike Clothed Humans
Huang, Y., Yi, H., Xiu, Y., Liao, T., Tang, J., Cai, D., & Thies, J. (2024) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv62453.2024.00152 · Cited by 116
Diff-Font: Diffusion Model for Robust One-Shot Font Generation
He, H., Chen, X., Wang, C., Liu, J., Du, B., Tao, D., & Qiao, Y. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02137-0 · Cited by 105
Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance
Zhu, S., Chen, J. L., Dai, Z., Dong, Z., Xu, Y., Yao, Y., Yao, Y., Zhu, H., & Zhu, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73001-6_9 · Cited by 171
DiffMOT: A Real-time Diffusion-based Multiple Object Tracker with Non-linear Prediction
Lv, W., Huang, Y., Zhang, N., Lin, R.-S., Han, M., & Zeng, D. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01828 · Cited by 104
Federated Learning-Empowered AI-Generated Content in Wireless Networks
Huang, X., Li, P., Du, H., Kang, J., Niyato, D. T., Kim, D. I., & Wu, Y. (2024) · IEEE Network · doi:10.1109/mnet.2024.3353377 · Cited by 102
DEADiff: An Efficient Stylization Diffusion Model with Disentangled Representations
Qi, T., Fang, S., Wu, Y., Xie, H., Liu, J., Chen, L., He, Q., & Zhang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00830 · Cited by 111
Analyzing and Improving the Training Dynamics of Diffusion Models
Karras, T., Aittala, M., Lehtinen, J., Hellsten, J., Aila, T., & Laine, S. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02282 · Cited by 186
Li, J., Li, B., Tu, Z., Liu, X., Guo, Q., Juefei-Xu, F., Xu, R., & Yu, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01440 · Cited by 97
Anti-DreamBooth: Protecting users from personalized text-to-image synthesis
Le, T. V., Phung, H., Nguyen, T. H., Dao, Q. T., Tran, N., & Tran, A. N. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00202 · Cited by 112
Deep Learning-Based Image and Video Inpainting: A Survey
Quan, W., Chen, J., Liu, Y., Dong-ming Yan, A., & Wonka, P. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-023-01977-6 · Cited by 106
Segment and Recognize Anything at Any Granularity
Li, F., Zhang, H., Sun, P., Zou, X., Liu, S., Li, C., Yang, J., Zhang, L., & Gao, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73195-2_27 · Cited by 89
Self-citationUpscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolution
Zhou, S., Yang, P., Wang, J., Luo, Y., & Loy, C. C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00245 · Cited by 108
Liu, G., Du, H., Niyato, D. T., Kang, J., Xiong, Z., Kim, D. I., & Shen, X. (2024) · IEEE Network · doi:10.1109/mnet.2024.3352917 · Cited by 87
Wang, J., Du, H., Niyato, D. T., Kang, J., Xiong, Z., Rajan, D., Mao, S., & Shen, X. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3377226 · Cited by 87
DesignPrompt: Using Multimodal Interaction for Design Exploration with Generative AI
Peng, X., Koch, J., & Mackay, W. E. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661588 · Cited by 110
A Task Is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpainting
Zhuang, J., Zeng, Y., Liu, W., Yuan, C. Y., & Chen, K. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73636-0_12 · Cited by 143
Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models: A Survey and Benchmark
Xin, Y., Yang, J., Luo, S., Du, Y., Qin, Q., Chen, H., Cen, K., He, Y., Fu, B., Cao, Y., et al. (2026) · International Journal of Computer Vision · doi:10.1007/s11263-026-02864-6 · Cited by 163
HumanSD: A Native Skeleton-Guided Diffusion Model for Human Image Generation
Ju, X., Zeng, A., Zhao, C., Wang, J., Zhang, L., & Xu, Q. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01465 · Cited by 112
Self-citationTraining-Free Consistent Text-to-Image Generation
Tewel, Y., Kaduri, O., Gal, R., Kasten, Y., Wolf, L., Chechik, G., & Atzmon, Y. (2024) · ACM Transactions on Graphics · doi:10.1145/3658157 · Cited by 68
Generative Image AI Using Design Sketches as input: Opportunities and Challenges
Zhang, C., Wang, W., Pangaro, P., Martelaro, N., & Byrne, D. (2023) · Creativity and Cognition · doi:10.1145/3591196.3596820 · Cited by 95
LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents
Liu, S., Cheng, H., Liu, H., Zhang, H., Li, F., Ren, T., Zou, X., Yang, J., Su, H., Zhu, J., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72970-6_8 · Cited by 182
Self-citationID.8: Co-Creating Visual Stories with Generative AI
Antony, V. N., & Huang, C.-M. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3672277 · Cited by 92
CRS-Diff: Controllable Remote Sensing Image Generation With Diffusion Model
Tang, D., Cao, X., Hou, X., Jiang, Z., Liu, J., & Meng, D. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3453414 · Cited by 103
Wang, J., Duan, H., Liu, J., Chen, S., Min, X., & Zhai, G. (2024) · Lecture Notes in Computer Science · doi:10.1007/978-981-99-9119-8_5 · Cited by 80
Jo, H., Lee, J.-K., Lee, Y.-C., & Choo, S. (2024) · Journal of Computational Design and Engineering · doi:10.1093/jcde/qwae017 · Cited by 83
MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesis
Zhou, D., Li, Y., Ma, F., Zhang, X., & Yang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00651 · Cited by 113
Anatomically-Controllable Medical Image Generation with Segmentation-Guided Diffusion Models
Konz, N., Chen, Y., Dong, H., & Mazurowski, M. A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72104-5_9 · Cited by 81
MotionDirector: Motion Customization of Text-to-Video Diffusion Models
Zhao, R., Gu, Y., Wu, J. Z., Zhang, D. J., Liu, J., Wu, W., Keppo, J., & Shou, M. Z. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72992-8_16 · Cited by 135
Cross-Image Attention for Zero-Shot Appearance Transfer
Alaluf, Y., Garibi, D., Patashnik, O., Averbuch‐Elor, H., & Cohen‐Or, D. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657423 · Cited by 79
GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image
Fu, X. Y., Yin, W., Hu, M., Wang, K., Ma, Y., Tan, P., Shen, S., Lin, D., & Long, X. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72670-5_14 · Cited by 156
Tracking Meets LoRA: Faster Training, Larger Model, Stronger Performance
Lin, L., Fan, H., Zhang, Z., Wang, Y., Xu, Y., & Ling, H. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73232-4_17 · Cited by 86
Make-Your-Video: Customized Video Generation Using Textual and Structural Guidance
Xing, J., Xia, M., Liu, Y., Zhang, Y., Zhang, Y., He, Y., Liu, H., Chen, H., Cun, X., Wang, X., et al. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3365804 · Cited by 112
Chen, X., & Wu, D. (2024) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2024.3378279 · Cited by 85
Has multimodal learning delivered universal intelligence in healthcare? A comprehensive survey
Lin, Q., Zhu, Y., Mei, X., Huang, L., Ma, J., He, K., Peng, Z., Cambria, E., & Feng, M. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102795 · Cited by 86
Alimisis, P., Mademlis, I., Grammatikis, P. I. R., Sarigiannidis, P. G., & Papadopoulos, G. T. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11116-x · Cited by 96
OmniGen: Unified Image Generation
Xiao, S., Wang, Y., Zhou, J., Yuan, H., Xing, X., Yan, R., Li, C., Wang, S., Huang, T., & Liu, Z. (2025) · 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52734.2025.01241 · Cited by 75
Generative AI and process systems engineering: The next frontier
Decardi‐Nelson, B., Alshehri, A. S., Ajagekar, A., & You, F. (2024) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2024.108723 · Cited by 90
Music ControlNet: Multiple Time-Varying Controls for Music Generation
Wu, S.-L., Donahue, C., Watanabe, S., & Bryan, N. J. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3399026 · Cited by 103
Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modeling
Shi, X., Huang, Z., Wang, F.-Y., Bian, W., Li, D., Zhang, Y., Zhang, M., Cheung, K. C., See, S., Qin, H., et al. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657497 · Cited by 116
Large Language Models for manufacturing
Li, Y., Zhao, H., Jiang, H., Pan, Y., Liu, Z., Wu, Z., Shu, P., Tian, J., Yang, T., Xu, S., et al. (2026) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2026.02.014 · Cited by 108
TADA! Text to Animatable Digital Avatars
Liao, T., Yi, H., Xiu, Y., Tang, J., Huang, Y., Thies, J., & Black, M. J. (2024) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv62453.2024.00150 · Cited by 104
Paint3D: Paint Anything 3D With Lighting-Less Texture Diffusion Models
Zeng, X., Chen, X., Qi, Z., Liu, W., Zhao, Z., Wang, Z., Fu, B., Liu, Y., & Yu, G. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00407 · Cited by 74
GaussianObject: High-Quality 3D Object Reconstruction from Four Views with Gaussian Splatting
Chen, Y., Li, S., Fang, J., Liang, R., Xie, L., Zhang, X., Shen, W., & Tian, Q. C. (2024) · ACM Transactions on Graphics · doi:10.1145/3687759 · Cited by 74
Freestyle Layout-to-Image Synthesis
Han, X., Huang, Z., Sun, Q., Song, L., & Zhang, W. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01370 · Cited by 80
SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Models
Guo, Y., Yang, C., Rao, A., Agrawala, M., Lin, D., & Dai, B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72946-1_19 · Cited by 149
Self-citationHIVE: Harnessing Human Feedback for Instructional Visual Editing
Zhang, S., Yang, X., Feng, Y., Qin, C., Chen, C.-C., Yu, N., Chen, Z., Wang, H., Savarese, S., Ermon, S., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00862 · Cited by 133
Prompt-Free Diffusion: Taking “Text” Out of Text-to-Image Diffusion Models
Xu, X., Guo, J., Wang, Z., Huang, G. M., Essa, I. A., & Shi, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00829 · Cited by 84
Applications of Generative AI (GAI) for Mobile and Wireless Networking: A Survey
Vu, T.-H., Jagatheesaperumal, S. K., Nguyen, M.-D., Huynh, N. V., Kim, S., & Pham, Q. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3487627 · Cited by 78
3D-GPT: Procedural 3D Modeling with Large Language Models
Sun, C., Han, J., Deng, W., Wang, X., Qin, Z., & Gould, S. J. (2025) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv66043.2025.00119 · Cited by 70
Expressive Text-to-Image Generation with Rich Text
Ge, S., Park, T., Zhu, J.-Y., & Huang, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00694 · Cited by 87
Alhabeeb, S., & Al-Shargabi, A. A. (2024) · IEEE Access · doi:10.1109/access.2024.3365043 · Cited by 77
NetDiffusion: Network Data Augmentation Through Protocol-Constrained Traffic Generation
Jiang, X., Liu, S., Gember-Jacobson, A., Bhagoji, A. N., Schmitt, P., Bronzino, F., & Feamster, N. (2024) · Proceedings of the ACM on Measurement and Analysis of Computing Systems · doi:10.1145/3639037 · Cited by 69
Implicit Style-Content Separation Using B-LoRA
Frenkel, Y., Vinker, Y., Shamir, A., & Cohen‐Or, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72684-2_11 · Cited by 69
UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANs
Xu, Y., Zhao, Y., Xiao, Z., & Hou, T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00783 · Cited by 112
VACE: All-in-One Video Creation and Editing
Jiang, Z., Han, Z., Mao, C., Zhang, J., Pan, Y., & Liu, Y. (2025) · 2025 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01597 · Cited by 68
GenAssist: Making Image Generation Accessible
Huh, M., Peng, Y.-H., & Pavel, A. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606735 · Cited by 67
“In addition to text input, we can also consider multimodal input from users in the future such as image prompts [54], sketch prompts [11, 82], or music prompts [55] to create an image for a music album cover, as desired by P6.”
BackgroundChangen2: Multi-Temporal Remote Sensing Generative Change Foundation Model
Zheng, Z., Ermon, S., Kim, D., Zhang, L., & Zhong, Y. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3475824 · Cited by 80
Self-citationRGB↔X: Image decomposition and synthesis using material- and lighting-aware diffusion models
Zeng, Z., Deschaintre, V., Georgiev, I., Hold-Geoffroy, Y., Hu, Y., Luan, F., Yan, L., & Hašan, M. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657445 · Cited by 67
Generative AI model privacy: a survey
Liu, Y., Huang, J., Li, Y., Wang, D., & Xiao, B. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-11024-6 · Cited by 74
ControlMat: A Controlled Generative Approach to Material Capture
Vecchio, G., Martin, R., Roullier, A., Kaiser, A., Rouffet, R., Deschaintre, V., & Boubekeur, T. (2024) · ACM Transactions on Graphics · doi:10.1145/3688830 · Cited by 76
Diffusion Models Meet Remote Sensing: Principles, Methods, and Perspectives
Liu, Y., Yue, J., Xia, S., Ghamisi, P., Xie, W., & Fang, L. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3464685 · Cited by 81
Arc2Face: A Foundation Model for ID-Consistent Human Faces
Papantoniou, F. P., Lattas, A., Moschoglou, S., Deng, J., Kainz, B., & Zafeiriou, S. P. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72913-3_14 · Cited by 64
Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generation
Wang, W., Yang, H., Tuo, Z., He, H., Zhu, J., Fu, J., & Liu, J. (2025) · International Journal of Computer Vision · doi:10.1007/s11263-025-02349-y · Cited by 159
Language as Reality: A Co-Creative Storytelling Game Experience in 1001 Nights Using Generative AI
Sun, Y., Li, Z., Fang, K., Lee, C. H., & Asadipour, A. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v19i1.27539 · Cited by 63
Denoising Diffusion Autoencoders are Unified Self-supervised Learners
Xiang, W., Hongyu, Y., Huang, D., & Wang, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01448 · Cited by 88
CoDeF: Content Deformation Fields for Temporally Consistent Video Processing
Ouyang, H., Wang, Q., Xiao, Y., Bai, Q., Zhang, J., Zheng, K., Zhou, X., Chen, Q., & Chen, Q. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00773 · Cited by 98
Liu, C., Chen, K., Zhao, R., Zou, Z., & Shi, Z. (2025) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2025.3560455 · Cited by 63
ChatPose: Chatting about 3D Human Pose
Feng, Y., Jing, L., Dwivedi, S. K., Sun, Y. M., Patel, P., & Black, M. J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00204 · Cited by 81
Rethinking Inductive Biases for Surface Normal Estimation
Bae, G., & Davison, A. J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00911 · Cited by 87
Dai, M.-G., Feng, Y., Wang, R., & Jung, J.-H. (2024) · Applied Sciences · doi:10.3390/app142311032 · Cited by 63
Prompt Tuning Inversion for Text-Driven Image Editing Using Diffusion Models
Dong, W., Xue, S., Duan, X., & Han, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00683 · Cited by 81
MotionLCM: Real-Time Controllable Motion Generation via Latent Consistency Model
Dai, W., Chen, L.-H., Wang, J., Liu, J., Dai, B., & Tang, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72640-8_22 · Cited by 61
MDNet: Mamba-Effective Diffusion-Distillation Network for RGB-Thermal Urban Dense Prediction
Zhou, W., Wu, H., & Jiang, Q. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3508058 · Cited by 67
Controllable Generation With Text-to-Image Diffusion Models: A Survey
Cao, P., Zhou, F., Song, Q., & Yang, L. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3646548 · Cited by 113
MapAnything: Universal Feed-Forward Metric 3D Reconstruction; map-anything.github.io
Keetha, N., Müller, N., Schönberger, J. L., Porzi, L., Zhang, Y., Fischer, T., Knapitsch, A., Zauss, D., Weber, E., Antunes, N., et al. (2026) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv69130.2026.00054 · Cited by 95
Chen, L., Xiao, S., Chen, Y., Song, Y., Wu, R., & Sun, L. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642229 · Cited by 60
“Moreover, considering the limitations in children’s drawing abilities and the aspiration for high-quality, tailored assets, we incorporated Stable Diffusion with ControlNet [55, 83].”
BackgroundMethodControlNet$$++$$: Improving Conditional Controls with Efficient Consistency Feedback
Li, M., Yang, T., Kuang, H., Wu, J., Wang, Z., Xiao, X., & Chen, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72667-5_8 · Cited by 97
SimDA: Simple Diffusion Adapter for Efficient Video Generation
Xing, Z., Dai, Q., Hu, H., Wu, Z., & Jiang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00748 · Cited by 92
Dream Video: Composing Your Dream Videos with Customized Subject and Motion
Wei, Y., Zhang, S., Qing, Z., Yuan, H., Liu, Z., Liu, Y., Zhang, Y., Zhou, J., & Shan, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00625 · Cited by 94
Diffusion Model as Representation Learner
Yang, X., & Wang, X. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01736 · Cited by 58
StyleShot: A Snapshot on Any Style
Gao, J., Sun, Y., Liu, Y., Tang, Y., Zeng, Y., Qi, D., Chen, K., & Zhao, C. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3610614 · Cited by 58
Speed Is All You Need: On-Device Acceleration of Large Diffusion Models via GPU-Aware Optimizations
Chen, Y., Sarokin, R., Lee, J., Tang, J., Chang, C.-L., Kulik, A., & Grundmann, M. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00490 · Cited by 66
Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model
Yang, K., Tao, J., Lyu, J., Ge, C., Chen, J., Shen, W., Zhu, X., & Li, X. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00854 · Cited by 109
Kim, T. S., Lee, Y., Chang, M., & Kim, J. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606833 · Cited by 56
SKED: Sketch-guided Text-based 3D Editing
Mikaeili, A., Perel, O., Safaee, M., Cohen‐Or, D., & Amiri, A. M. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01343 · Cited by 73
DisenDreamer: Subject-Driven Text-to-Image Generation With Sample-Aware Disentangled Tuning
Chen, H., Zhang, Y. J., Wang, X. Y., Duan, X., Zhou, Y., & Zhu, W. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3369757 · Cited by 59
Zhou, Y., Leng, H., Meng, S., Wu, H., & Zhang, Z. (2024) · Engineering Structures · doi:10.1016/j.engstruct.2024.118068 · Cited by 56
Zhang, J., Xiang, R., Kuang, Z., Wang, B., & Li, Y. (2024) · Heritage Science · doi:10.1186/s40494-024-01334-x · Cited by 60
VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation
Ku, M., Jiang, D., Wei, C., Yue, X., & Chen, W. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.663 · Cited by 56
FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Condition
Mo, S., Mu, F., Lin, K. H., Liu, Y., Guan, B., Li, Y., & Zhou, B. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00713 · Cited by 74
Metadata
- Authors
- Zhang, L., Rao, A., & Agrawala, M.
- Affiliations
- Stanford University (US)
- Venue
- 2023 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2023-10-01
- Pages
- 3813-3824
- OpenAlex
- W4390873054
- Cited by
- 4,547
- References
- 91
- Citations by year
- 2026: 1290 · 2025: 1847 · 2024: 1255 · 2023: 155
- FWCI
- 486.28
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Funders
- Brown Institute for Media Innovation
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- ControlNet, text-to-image diffusion models, spatial conditioning, conditional image generation, Stable Diffusion, parameter-efficient fine-tuning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
