Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models
Open accessAbstract
Recent text-to-image generative models have demonstrated an unparalleled ability to generate diverse and creative imagery guided by a target text prompt. While revolutionary, current state-of-the-art diffusion models may still fail in generating images that fully convey the semantics in the given text prompt. We analyze the publicly available Stable Diffusion model and assess the existence of catastrophic neglect , where the model fails to generate one or more of the subjects from the input prompt. Moreover, we find that in some cases the model also fails to correctly bind attributes ( e.g. , colors) to their corresponding subjects. To help mitigate these failure cases, we introduce the concept of Generative Semantic Nursing (GSN) , where we seek to intervene in the generative process on the fly during inference time to improve the faithfulness of the generated images. Using an attention-based formulation of GSN, dubbed Attend-and-Excite , we guide the model to refine the cross-attention units to attend to all subject tokens in the text prompt and strengthen --- or excite --- their activations, encouraging the model to generate all subjects described in the text prompt. We compare our approach to alternative approaches and demonstrate that it conveys the desired concepts more faithfully across a range of text prompts. Code is available at our project page: https://attendandexcite.github.io/Attend-and-Excite/.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
ISO/IEC DTS 4213 is an ongoing work item of ISO/IEC JTC 1/SC 42 on assessment of machine learning classification performance.
“ISO/IEC DTS (Draft Technical Specification) 4,213 “Information technology—Artificial Intelligence— Assessment of machine learning classification performance”12”
From Deepfake: definitions, performance metrics and standards, datasets, and a meta-review (2024) · cites this paper as [12]
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 15,880 · see citing works
Denoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712 · see citing works
AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks
Xu, T., Zhang, P., Huang, Q., Zhang, H., Gan, Z., Huang, X., & He, X. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00143 · Cited by 1,923 · see citing works
Image Generation from Scene Graphs
Johnson, J., Gupta, A., & Fei-Fei, L. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00133 · Cited by 836 · see citing works
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2021) · arXiv · doi:10.48550/arxiv.2112.10752 · Cited by 737 · see citing works
DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-To-Image Synthesis
Zhu, M., Pan, P., Chen, W., & Yang, Y. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00595 · Cited by 605 · see citing works
Gal, R., Patashnik, O., Maron, H., Bermano, A. H., Chechik, G., & Cohen‐Or, D. (2022) · ACM Transactions on Graphics · doi:10.1145/3528223.3530164 · Cited by 503 · see citing works
Self-citationDF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesis
Tao, M., Tang, H., Wu, F., Jing, X., Bao, B., & Xu, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01602 · Cited by 399 · see citing works
Cross-Modal Contrastive Learning for Text-to-Image Generation
Zhang, H., Koh, J. Y., Baldridge, J., Lee, H., & Yang, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00089 · Cited by 359 · see citing works
VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance
Crowson, K., Biderman, S., Kornis, D., Stander, D., Hallahan, E., Castricato, L., & Raff, E. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19836-6_6 · Cited by 316 · see citing works
Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers
Chefer, H., Gur, S., & Wolf, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00045 · Cited by 313 · see citing works
Self-citationCLIPstyler: Image Style Transfer with a Single Text Condition
Kwon, G., & Ye, J. C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01753 · Cited by 262 · see citing works
eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert DenoisersPreprint
Balaji, Y., Nah, S., Huang, X., Vahdat, A., Song, J., Zhang, Q., Kreis, K., Aittala, M., Aila, T., Laine, S., et al. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.01324 · Cited by 253 · see citing works
Zhao, B., Meng, L., Yin, W., & Sigal, L. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00878 · Cited by 212 · see citing works
Specifying Object Attributes and Relations in Interactive Scene Generation
Ashual, O., & Wolf, L. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00466 · Cited by 185 · see citing works
Self-citationImage-Based CLIP-Guided Essence Transfer
Chefer, H., Benaim, S., Paiss, R., & Wolf, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19778-9_40 · Cited by 35 · see citing works
Self-citationNo Token Left Behind: Explainability-Aided Image Classification and Generation
Paiss, R., Chefer, H., & Wolf, L. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19775-8_20 · Cited by 23 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
A Survey on Generative Diffusion Models
Cao, H., Tan, C., Gao, Z., Xu, Y., Chen, G., Heng, P., & Li, S. Z. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3361474 · Cited by 679 · see citing works
MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editing
Cao, M., Wang, X., Qi, Z., Shan, Y., Qie, X., & Zheng, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02062 · Cited by 443 · see citing works
Proceedings of the 32nd ACM International Conference on Multimedia
Pan, Q., Zhao, Z. W., Wang, Z., Long, S., Wu, Y., Ji, W., Liang, H., & Liang, R. (2024) · doi:10.1145/3664647 · Cited by 290 · see citing works
Training-Free Layout Control with Cross-Attention Guidance
Chen, M., Laina, I., & Vedaldi, A. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00526 · Cited by 180 · see citing works
BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusion
Xie, J., Li, Y., Huang, Y., Liu, H., Zhang, W., Zheng, Y., & Shou, M. Z. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00685 · Cited by 174 · see citing works
FastComposer: Tuning-Free Multi-subject Image Generation with Localized Attention
Xiao, G., Yin, T., Freeman, W. T., Durand, F., & Han, S. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02227-z · Cited by 189 · see citing works
Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap
Wu, X., Wu, S.-H., Wu, J., Feng, L., & Tan, K. C. (2024) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2024.3506731 · Cited by 177 · see citing works
Versatile Diffusion: Text, Images and Variations All in One Diffusion Model
Xu, X., Wang, Z., Zhang, E., Wang, K., & Shi, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00713 · Cited by 150 · see citing works
Multimodal Foundation Models: From Specialists to General-Purpose Assistants
Li, C., Gan, Z., Yang, Z., Yang, J., Li, L., Wang, L., & Gao, J. (2024) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000110 · Cited by 225 · see citing works
Style Aligned Image Generation via Shared Attention
Hertz, A., Voynov, A. S., Fruchter, S., & Cohen‐Or, D. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00457 · Cited by 153 · see citing works
Self-citationBreak-A-Scene: Extracting Multiple Concepts from a Single Image
Avrahami, O., Aberman, K., Fried, O., Cohen‐Or, D., & Lischinski, D. (2023) · SIGGRAPH Asia Conference Papers · doi:10.1145/3610548.3618154 · Cited by 147 · see citing works
Self-citationForget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models
Zhang, G., Wang, K., Xu, X., Wang, Z., & Shi, H. (2024) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00182 · Cited by 115 · see citing works
Dense Text-to-Image Generation with Attention Modulation
Kim, Y., Lee, J., Kim, J.-H., Ha, J.-W., & Zhu, J.-Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00708 · Cited by 115 · see citing works
TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Composition
Lu, S., Liu, Y., & Kong, A. W. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00218 · Cited by 105 · see citing works
Localizing Object-level Shape Variations with Text-to-Image Diffusion Models
Patashnik, O., Garibi, D., Azuri, I., Averbuch‐Elor, H., & Cohen‐Or, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02107 · Cited by 100 · see citing works
Self-citationInstanceDiffusion: Instance-Level Control for Image Generation
Wang, X., Darrell, T., Rambhatla, S. S., Girdhar, R., & Misra, I. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00596 · Cited by 100 · see citing works
On the design fundamentals of diffusion models: A survey
Chang, Z., Koulieris, G. A., Chang, H. J., & Shum, H. P. H. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111934 · Cited by 117 · see citing works
LEDITS++: Limitless Image Editing Using Text-to-Image Models
Brack, M., Friedrich, F., Kornmeier, K., Tsaban, L., Schramowski, P., Kersting, K., & Passos, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00846 · Cited by 93 · see citing works
Revolutionizing Visuals: The Role of Generative AI in Modern Image Generation
Bansal, G., Nawal, A., Chamola, V., & Herencsar, N. (2024) · ACM Transactions on Multimedia Computing, Communications, and Applications · doi:10.1145/3689641 · Cited by 91 · see citing works
MACE: Mass Concept Erasure in Diffusion Models
Lu, S., Wang, Z., Li, L., Liu, Y., & Adams Wai Kin, K. O. N. G. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00615 · Cited by 90 · see citing works
Grounded Text-to-Image Synthesis with Attention Refocusing
Phung, Q., Ge, S., & Huang, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00758 · Cited by 94 · see citing works
A Neural Space-Time Representation for Text-to-Image Personalization
Alaluf, Y., Richardson, E., Metzer, G., & Cohen‐Or, D. (2023) · ACM Transactions on Graphics · doi:10.1145/3618322 · Cited by 92 · see citing works
Self-citationCross-Image Attention for Zero-Shot Appearance Transfer
Alaluf, Y., Garibi, D., Patashnik, O., Averbuch‐Elor, H., & Cohen‐Or, D. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657423 · Cited by 79 · see citing works
Self-citationMIGC: Multi-Instance Generation Controller for Text-to-Image Synthesis
Zhou, D., Li, Y., Ma, F., Zhang, X.-T., & Yang, Y. (2024) · Computer Vision and Pattern Recognition · doi:10.1109/cvpr52733.2024.00651 · Cited by 175 · see citing works
Uni-paint: A Unified Framework for Multimodal Image Inpainting with Pretrained Diffusion Model
Yang, S., Chen, X., & Liao, J. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612200 · Cited by 69 · see citing works
Expressive Text-to-Image Generation with Rich Text
Ge, S., Park, T., Zhu, J.-Y., & Huang, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00694 · Cited by 69 · see citing works
Deepfake: definitions, performance metrics and standards, datasets, and a meta-review
Altuncu, E., Franqueira, V. N. L., & Li, S. (2024) · Frontiers in Big Data · doi:10.3389/fdata.2024.1400024 · Cited by 65 · see citing works
T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-Image Generation
Huang, K., Duan, C., Sun, K., Xie, E., Li, Z., & Liu, X. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3531907 · Cited by 74 · see citing works
UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANs
Xu, Y., Zhao, Y., Xiao, Z., & Hou, T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00783 · Cited by 68 · see citing works
Vox-E: Text-guided Voxel Editing of 3D Objects
Sella, E., Fiebelman, G., Hedman, P., & Averbuch‐Elor, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00046 · Cited by 68 · see citing works
Training-Free Consistent Text-to-Image Generation
Tewel, Y., Kaduri, O., Gal, R., Kasten, Y., Wolf, L., Chechik, G., & Atzmon, Y. (2024) · ACM Transactions on Graphics · doi:10.1145/3658157 · Cited by 68 · see citing works
Self-citationPrompt Tuning Inversion for Text-Driven Image Editing Using Diffusion Models
Dong, W., Xue, S., Duan, X., & Han, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00683 · Cited by 61 · see citing works
AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusion
Jiang, Y., Zhang, Z., Xue, T., & Gu, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73661-2_19 · Cited by 64 · see citing works
Zero-shot spatial layout conditioning for text-to-image diffusion models
Couairon, G., Careil, M., Cord, M., Lathuilière, S., & Verbeek, J. J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00207 · Cited by 55 · see citing works
SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generation
Zhang, Y., Song, Y., Liu, J., Wang, R., Yu, J., Tang, H., Li, H., Tang, X., Hu, Y., Pan, H. C., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00771 · Cited by 54 · see citing works
Create Your World: Lifelong Text-to-Image Diffusion
Sun, G., Liang, W., Dong, J., Li, J., Ding, Z., & Cong, Y. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3382753 · Cited by 53 · see citing works
Composing Object Relations and Attributes for Image-Text Matching
Pham, K., Huynh, C., Lim, S.-N., & Shrivastava, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01361 · Cited by 53 · see citing works
Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motion
Yang, S., Hou, L., Huang, H., Ma, C., Wan, P., Zhang, D., Chen, X., & Liao, J. (2024) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3641519.3657481 · Cited by 55 · see citing works
GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphs
Gao, G., Liu, W., Chen, A., Geiger, A., & Schölkopf, B. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02012 · Cited by 46 · see citing works
Let the Chart Spark: Embedding Semantic Context into Chart with Text-to-Image Generative Model
Xiao, S., Huang, S., Lin, Y., Ye, Y., & Zeng, W. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2023.3326913 · Cited by 44 · see citing works
Diffusion Model is Secretly a Training-Free Open Vocabulary Semantic Segmenter
Wang, J., Li, X., Zhang, J., Xu, Q., Zhou, Q., Yu, Q., Sheng, L., & Xu, D. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3551648 · Cited by 55 · see citing works
Compositional Text-to-Image Synthesis with Attention Map Control of Diffusion Models
Wang, R., Chen, Z., Chen, C., Ma, J., Lu, H., & Lin, X. S. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i6.28364 · Cited by 43 · see citing works
Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfer
Yatim, D., Fridman, R., Bar-Tal, O., Kasten, Y., & Dekel, T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00809 · Cited by 47 · see citing works
DGE: Direct Gaussian 3D Editing by Consistent Multi-view Editing
Chen, M., Laina, I., & Vedaldi, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72904-1_5 · Cited by 39 · see citing works
PortraitBooth: A Versatile Portrait Model for Fast Identity-Preserved Personalization
Xu, P., Zhu, J., Jiang, B., Tai, Y. Y., Luo, D., Zhang, J., Lin, W., Jin, T., Wang, C., & Ji, R. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02557 · Cited by 38 · see citing works
Initno: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization
Guo, X., Liu, J., Cui, M., Li, J., Hongyu, Y., & Huang, D. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00896 · Cited by 36 · see citing works
MosaicFusion: Diffusion Models as Data Augmenters for Large Vocabulary Instance Segmentation
Xie, J., Li, W., Li, X., Liu, Z., Ong, Y.-S., & Loy, C. C. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02223-3 · Cited by 40 · see citing works
A-STAR: Test-time Attention Segregation and Retention for Text-to-image Synthesis
Agarwal, A., Karanam, S., Joseph, K. J., Saxena, A., Goswami, K., & Srinivasan, B. V. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00217 · Cited by 35 · see citing works
Focus on Your Instruction: Fine-grained and Multi-instruction Image Editing by Attention Modulation
Guo, Q., & Lin, T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00667 · Cited by 36 · see citing works
Diffusion-Based Visual Art Creation: A Survey and New Perspectives
Wang, B., Chen, Q., & Wang, Z. (2025) · ACM Computing Surveys · doi:10.1145/3728459 · Cited by 40 · see citing works
Adversarial attacks and defenses on text-to-image diffusion models: A survey
Zhang, C., Hu, M., Li, W., & Wang, L. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102701 · Cited by 33 · see citing works
Controllable Generation With Text-to-Image Diffusion Models: A Survey
Cao, P., Zhou, F., Song, Q., & Yang, L. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3646548 · Cited by 61 · see citing works
ConvoFusion: Multi-Modal Conversational Diffusion for Co-Speech Gesture Synthesis
Mughal, M. H., Dabral, R., Habibie, I., Donatelli, L. E., Habermann, M., & Theobalt, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00138 · Cited by 34 · see citing works
Weed image augmentation by ControlNet-added stable diffusion for multi-class weed detection
Deng, B., & Lu, Y. (2025) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2025.110123 · Cited by 33 · see citing works
Ma, X., & Huo, Y. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103755 · Cited by 30 · see citing works
A Survey of Cross-Modal Visual Content Generation
Nazarieh, F., Feng, Z., Awais, M., Wang, W., & Kittler, J. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3351601 · Cited by 29 · see citing works
The Chosen One: Consistent Characters in Text-to-Image Diffusion Models
Avrahami, O., Hertz, A., Vinker, Y., Arar, M., Fruchter, S., Fried, O., Cohen-Or, D., & Lischinski, D. (2024) · Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers · doi:10.1145/3641519.3657430 · Cited by 29 · see citing works
Self-citationEchoScene: Indoor Scene Generation via Information Echo Over Scene Graph Diffusion
Zhai, G., Örnek, E. P., Chen, D. Z., Liao, R., Di, Y., Navab, N., Tombari, F., & Busam, B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72664-4_10 · Cited by 32 · see citing works
Generating Images of Rare Concepts Using Pre-trained Diffusion Models
Samuel, D., Ben‐Ari, R., Raviv, S., Darshan, N., & Chechik, G. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28270 · Cited by 31 · see citing works
Ranni: Taming Text-to-Image Diffusion for Accurate Instruction Following
Feng, Y., Gong, B., Chen, D., Shen, Y., Liu, Y., & Zhou, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00454 · Cited by 28 · see citing works
Be Yourself: Bounded Attention for Multi-subject Text-to-Image Generation
Dahary, O., Patashnik, O., Aberman, K., & Cohen‐Or, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72630-9_25 · Cited by 30 · see citing works
Self-citationZhang, X., Yang, B., Kampffmeyer, M., Zhang, W., Zhang, S., Lu, G., Lin, L., Xu, H., & Liang, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02116 · Cited by 24 · see citing works
Zhao, Y., & Lian, Z. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72751-1_13 · Cited by 27 · see citing works
Attention Calibration for Disentangled Text-to-Image Personalization
Zhang, Y., Yang, M., Zhou, Q., & Wang, Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00456 · Cited by 24 · see citing works
Unseen Visual Anomaly Generation
Sun, H., Cao, Y., Dong, H., & Fink, O. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.02375 · Cited by 23 · see citing works
Yang, Y., Wen, W. Y., Peng, L., Song, C., Chen, Y., Li, H., Yang, X., Lu, Q., Cai, D., He, X., et al. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3633153 · Cited by 37 · see citing works
SODA: Bottleneck Diffusion Models for Representation Learning
Hudson, D. A., Zoran, D., Malinowski, M., Lampinen, A. K., Jaegle, A., McClelland, J. L., Matthey, L., Hill, F., & Lerchner, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02181 · Cited by 24 · see citing works
Rethinking the Spatial Inconsistency in Classifier-Free Diffusion Guidance
Shen, D., Song, G., Xue, Z., Wang, F.-Y., & Liu, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00895 · Cited by 22 · see citing works
Prompting Hard or Hardly Prompting: Prompt Inversion for Text-to-Image Diffusion Models
Mahajan, S., Rahman, T., Yi, K. M., & Sigal, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00650 · Cited by 22 · see citing works
A Survey on Quality Metrics for Text-to-Image Generation
Hartwig, S., Engel, D., Sick, L., Kniesel, H., Payer, T., Poonam, P., Glöckler, M., Bäuerle, A., & Ropinski, T. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3585077 · Cited by 39 · see citing works
EasyControl: Adding Efficient and Flexible Control for Diffusion Transformer
Zhang, Y. S., Yuan, Y., Song, Y., Wang, H., & Liu, J. (2025) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51701.2025.01814 · Cited by 24 · see citing works
DMF-GAN: Deep Multimodal Fusion Generative Adversarial Networks for Text-to-Image Synthesis
Yang, B., Xiang, X., Kong, W., Zhang, J., & Peng, Y. K. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3358086 · Cited by 21 · see citing works
CONFORM: Contrast is All You Need For High-Fidelity Text-to-Image Diffusion Models
Meral, T. H. S., Simsar, E., Tombari, F., & Yanardag, P. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00860 · Cited by 20 · see citing works
Large-Scale Reinforcement Learning for Diffusion Models
Zhang, Y., Tzeng, E., Du, Y., & Kislyuk, D. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73036-8_1 · Cited by 19 · see citing works
CosmicMan: A Text-to-Image Foundation Model for Humans
Li, S., Fu, J., Liu, K., Wang, W., Lin, K.-Y., & Wu, W. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00664 · Cited by 20 · see citing works
Understanding the Impact of Negative Prompts: When and How Do They Take Effect?
Ban, Y., Wang, R., Zhou, T., Cheng, M., Gong, B., & Hsieh, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73024-5_12 · Cited by 18 · see citing works
ConceptLab: Creative Concept Generation using VLM-Guided Diffusion Prior Constraints
Richardson, E., Goldberg, K., Alaluf, Y., & Cohen‐Or, D. (2024) · ACM Transactions on Graphics · doi:10.1145/3659578 · Cited by 17 · see citing works
Self-citationDefectFill: Realistic Defect Generation with Inpainting Diffusion Model for Visual Inspection
Song, J., Park, D., Baek, K., Lee, S., Choi, J., Kim, E., & Yoon, S. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01744 · Cited by 18 · see citing works
TokenCompose: Text-to-Image Diffusion with Token-Level Supervision
Wang, Z., Sha, Z., Ding, Z., Wang, Y., & Tu, Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00817 · Cited by 21 · see citing works
GenZI: Zero-Shot 3D Human-Scene Interaction Generation
Li, L., & Dai, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01934 · Cited by 16 · see citing works
Monkey See, Monkey Do: Harnessing Self-attention in Motion Diffusion for Zero-shot Motion Transfer
Raab, S., Gat, I., Sala, N., Tevet, G., Shalev-Arkushin, R., Fried, O., Bermano, A. H., & Cohen‐Or, D. (2024) · SIGGRAPH Asia Conference Papers · doi:10.1145/3680528.3687579 · Cited by 16 · see citing works
Self-citationPALP: Prompt Aligned Personalization of Text-to-Image Models
Arar, M., Voynov, A. S., Hertz, A., Avrahami, O., Fruchter, S., Pritch, Y., Cohen‐Or, D., & Shamir, A. (2024) · SIGGRAPH Asia Conference Papers · doi:10.1145/3680528.3687604 · Cited by 16 · see citing works
Self-citationStableMakeup: When Real-World Makeup Transfer Meets Diffusion Model
Zhang, Y., Yuan, Y., Song, Y., & Liu, J. (2025) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/3721238.3730702 · Cited by 18 · see citing works
An Image is Worth Multiple Words: Multi-Attribute Inversion for Constrained Text-To-Image Synthesis
Agarwal, A., Karanam, S., Shukla, T., & Srinivasan, B. V. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00590 · Cited by 19 · see citing works
Wu, L. Y., Chau, K. T., Yahaya, W. A. J. W., Wang, S., & Wu, X. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2371692 · Cited by 14 · see citing works
KV Inversion: KV Embeddings Learning for Text-Conditioned Real Image Action Editing
Huang, J., Liu, Y., Qin, J., & Chen, S. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-8429-9_14 · Cited by 14 · see citing works
TIAM - A Metric for Evaluating Alignment in Text-to-Image Generation
Grimal, P., Borgne, H. L., Ferret, O., & Tourille, J. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00287 · Cited by 14 · see citing works
Prompt Highlighter: Interactive Control for Multi-Modal LLMs
Zhang, Y., Qian, S., Peng, B., Liu, S., & Jia, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01255 · Cited by 14 · see citing works
Tuning-Free Image Customization with Image and Text Guidance
Li, P., Nie, Q., Chen, Y., Jiang, X., Wu, K., Lin, Y., Liu, Y., Peng, J., Wang, C., & Zheng, F. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73116-7_14 · Cited by 15 · see citing works
Vision + Language Applications: A Survey
Zhou, Y., & Shimada, N. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00090 · Cited by 14 · see citing works
Few-Shot Defect Image Generation Based on Consistency Modeling
Shi, Q., Wei, J., Shen, F., & Zhang, Z. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73116-7_21 · Cited by 14 · see citing works
ConsistentID: Portrait Generation With Multimodal Fine-Grained Identity Preserving
Huang, J., Dong, X. M., Song, W., Zheng, C., Tang, Z., Zhou, J., Cheng, Y., Chen, L., Li, H., Yan, Y., et al. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3652557 · Cited by 21 · see citing works
Unsupervised Compositional Concepts Discovery with Text-to-Image Generative Models
Liu, N., Du, Y., Li, S., Tenenbaum, J. B., & Torralba, A. B. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00199 · Cited by 13 · see citing works
Generating Non-Stationary Textures Using Self-Rectification
Zhou, Y., Xiao, R., Lischinski, D., Cohen‐Or, D., & Huang, H. F. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00742 · Cited by 12 · see citing works
Self-citationCLiC: Concept Learning in Context
Safaee, M., Mikaeili, A., Patashnik, O., Cohen‐Or, D., & Amiri, A. M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00661 · Cited by 12 · see citing works
Self-citationPreciseControl: Enhancing Text-to-Image Diffusion Models with Fine-Grained Attribute Control
Parihar, R., Sachidanand, V. S., Mani, S., Karmali, T., & Babu, R. V. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73007-8_27 · Cited by 12 · see citing works
Enhancing Semantic Fidelity in Text-to-Image Synthesis: Attention Regulation in Diffusion Models
Zhang, Y., Tzun, T. T., Hern, L. W., & Kawaguchi, K. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73016-0_5 · Cited by 12 · see citing works
DiffUHaul: A Training-Free Method for Object Dragging in Images
Avrahami, O., Gal, R., Chechik, G., Fried, O., Lischinski, D., Vahdat, A., & Nie, W. (2024) · SIGGRAPH Asia Conference Papers · doi:10.1145/3680528.3687590 · Cited by 13 · see citing works
Rethinking local-to-global representation learning for rotation-invariant point cloud analysis
Wang, Z., Yu, Y., & Li, X. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2024.110624 · Cited by 11 · see citing works
Discriminative Probing and Tuning for Text-to-Image Generation
Qu, L., Wang, W., Li, Y., Zhang, H., Nie, L., & Chua, T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00710 · Cited by 16 · see citing works
- Authors
- Chefer, H., Alaluf, Y., Vinker, Y., Wolf, L., & Cohen-Or, D.
- Affiliations
- Tel Aviv University (IL)
- Tel Aviv University, Tel Aviv, Israel
- Venue
- ACM Transactions on Graphics
- Published
- 2023-07-26
- Volume
- 42
- Issue
- 4
- Pages
- 1-10
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W4385270985
- Cited by
- 461
- References
- 19
- Citations by year
- 2026: 120 · 2025: 143 · 2024: 166 · 2023: 32
- FWCI
- 41.84
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Transactions on Graphics (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- text-to-image generation, diffusion models, attention guidance, cross-attention, Stable Diffusion, inference-time intervention, semantic fidelity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review