DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation
Abstract
Large text-to-image models achieved a remarkable leap in the evolution of AI, enabling high-quality and diverse synthesis of images from a given text prompt. However, these models lack the ability to mimic the appearance of subjects in a given reference set and synthesize novel renditions of them in different contexts. In this work, we present a new approach for “personalization” of text-to-image diffusion models. Given as input just a few images of a subject, we fine-tune a pretrained text-to-image model such that it learns to bind a unique identifier with that specific subject. Once the subject is embedded in the output domain of the model, the unique identifier can be used to synthesize novel photorealistic images of the subject contextualized in different scenes. By leveraging the semantic prior embedded in the model with a new autogenous class-specific prior preservation loss, our technique enables synthesizing the subject in diverse scenes, poses, views and lighting conditions that do not appear in the reference images. We apply our technique to several previously-unassailable tasks, including subject recontextualization, text-guided view synthesis, and artistic rendering, all while preserving the subject's key features. We also provide a new dataset and evaluation protocol for this new task of subject-driven generation. Project page: https://dreambooth.github.io/
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Tune-A-Video allows extensions developed for text-to-image tasks such as ControlNet or Dreambooth to be utilized.
“extensions developed for text-to-image tasks such as ControlNet (Zhang & Agrawala, 2023) or Dreambooth (Ruiz et al., 2023) can be utilized.”
From Video Diffusion Models: A Survey (2024) · cites this paper as Ruiz et al., 2023
Cites
High-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 15,880 · see citing works
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714 · see citing works
Emerging Properties in Self-Supervised Vision Transformers
Caron, M., Touvron, H., Misra, I., Jegou, H., Mairal, J., Bojanowski, P., & Joulin, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00951 · Cited by 5,638 · cited 7× · see citing works
“We compute DINO and CLIP-I subject fidelity metrics and the CLIP-T prompt fidelity metric.”
BackgroundMethodInfluentialLarge Scale GAN Training for High Fidelity Natural Image Synthesis
Brock, A., Donahue, J., & Simonyan, K. (2018) · arXiv · doi:10.48550/arxiv.1809.11096 · Cited by 3,717 · see citing works
Diffusion Models Beat GANs on Image Synthesis
Dhariwal, P., & Nichol, A. (2021) · arXiv · doi:10.48550/arxiv.2105.05233 · Cited by 3,362 · see citing works
Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E., Ghasemipour, S. K. S., Ayan, B. K., Mahdavi, S. S., Lopes, R. G., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.11487 · Cited by 2,528 · see citing works
Hierarchical Text-Conditional Image Generation with CLIP Latents
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M. (2022) · arXiv · doi:10.48550/arxiv.2204.06125 · Cited by 2,511 · cited 9× · see citing works
“Next, we show comparisons of recontextualization of a subject clock, with distinctive features using our method and prompt engineering using vanilla Imagen [56] and the public API of DALL-E 2 [51].”
BackgroundMethodInfluentialHigh-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2021) · arXiv · doi:10.48550/arxiv.2112.10752 · Cited by 737 · see citing works
MirrorGAN: Learning Text-To-Image Generation by Redescription
Qiao, T., Zhang, J., Xu, D., & Tao, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00160 · Cited by 644 · cited 2× · see citing works
“There also exists work on text-to-image synthesis [14, 16, 19, 24, 27, 35, 36, 50, 51, 55, 58, 67, 74].”
BackgroundAn Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion
Gal, R., Alaluf, Y., Atzmon, Y., Patashnik, O., Bermano, A. H., Chechik, G., & Cohen-Or, D. (2022) · arXiv · doi:10.48550/arxiv.2208.01618 · Cited by 611 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Adding Conditional Control to Text-to-Image Diffusion Models
Zhang, L., Rao, A., & Agrawala, M. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00355 · Cited by 4,547 · see citing works
Diffusion Models: A Comprehensive Survey of Methods and Applications
Yang, L., Zhang, Z., Song, Y., Hong, S., Xu, R., Zhao, Y., Zhang, W., Cui, B., & Yang, M.-H. (2024) · ACM Computing Surveys · doi:10.1145/3626235 · Cited by 1,704 · see citing works
DIRE for Diffusion-Generated Image Detection
Wang, Z., Bao, J., Zhou, W., Wang, W., Hu, H., Chen, H., & Li, H. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02051 · Cited by 354 · see citing works
Generative AI in Higher Education: The ChatGPT Effect
Chan, C. K. Y., & Colloton, T. (2024) · Routledge · doi:10.4324/9781003459026 · Cited by 203 · see citing works
Draxler, F., Werner, A., Lehmann, F., Hoppe, M., Schmidt, A., Buschek, D., & Welsch, R. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3637875 · Cited by 186 · see citing works
DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animation
Shen, S., Zhao, W., Meng, Z., Li, W., Zhu, Z., Zhou, J., & Lu, J. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00197 · Cited by 133 · see citing works
Muse: Text-To-Image Generation via Masked Generative Transformers
Chang, H., Zhang, H., Barber, J., Maschinot, A., Lezama, J., Jiang, L., Yang, M.-H., Murphy, K., Freeman, W. T., Rubinstein, M., et al. (2023) · arXiv · doi:10.48550/arxiv.2301.00704 · Cited by 157 · see citing works
“Dreambooth (Ruiz et al., 2022) and Imagic (Kawar et al., 2022) involve fine-tuning of the generative models.”
BackgroundSelf-citationA vision–language foundation model for the generation of realistic chest X-ray images
Bluethgen, C., Chambon, P., Delbrouck, J.-B., van der Sluijs, R., Połacin, M., Zambrano Chaves, J. M., Abraham, T. M., Purohit, S., Langlotz, C. P., & Chaudhari, A. S. (2024) · Nature Biomedical Engineering · doi:10.1038/s41551-024-01246-y · Cited by 108 · see citing works
Revolutionizing Visuals: The Role of Generative AI in Modern Image Generation
Bansal, G., Nawal, A., Chamola, V., & Herencsar, N. (2024) · ACM Transactions on Multimedia Computing, Communications, and Applications · doi:10.1145/3689641 · Cited by 91 · see citing works
The Revolution of Multimodal Large Language Models: A Survey
Caffagni, D., Cocchi, F., Barsellotti, L., Moratelli, N., Sarto, S., Baraldi, L., Baraldi, L., Cornia, M., & Cucchiara, R. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.807 · Cited by 98 · see citing works
“DreamBench (Ruiz et al., 2023) is a benchmark that evaluates the generative capabilities of the models on subject-driven generation.”
Diffusion Facial Forgery Detection
Cheng, H., Guo, Y., Wang, T., Nie, L., & Kankanhalli, M. (2024) · Proceedings of the 32nd ACM International Conference on Multimedia · doi:10.1145/3664647.3680797 · Cited by 47 · see citing works
Diffusion-Based Visual Art Creation: A Survey and New Perspectives
Wang, B., Chen, Q., & Wang, Z. (2025) · ACM Computing Surveys · doi:10.1145/3728459 · Cited by 40 · see citing works
Inspecting the Geographical Representativeness of Images from Text-to-Image Models
Basu, A., Babu, R. V., & Pruthi, D. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00474 · Cited by 21 · see citing works
Deepfakes on Demand: The rise of accessible non-consensual deepfake image generators
Hawkins, W., Mittelstadt, B., & Russell, C. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732107 · Cited by 20 · see citing works
“Open models have been popular among development communities due to the ability to use and fine-tune models locally [18, 45, 62].”
PixArt-$α$: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis
Chen, J., Yu, J., Ge, C., Yao, L., Xie, E., Wu, Y., Wang, Z., Kwok, J., Luo, P., Lu, H., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.00426 · Cited by 30 · see citing works
“We extend the capabilities of P IX A RT - α by incorporating two commonly used customization methods: DreamBooth (Ruiz et al., 2022) and ControlNet (Zhang et al., 2023).”
MethodUnbounded: A Generative Infinite Game of Character Life Simulation
Li, J., Li, Y., Wadhwa, N., Pritch, Y., Jacobs, D. E., Rubinstein, M., Bansal, M., & Ruiz, N. (2024) · arXiv · doi:10.48550/arxiv.2410.18975 · Cited by 2 · see citing works
Self-citationVideo Diffusion Models: A Survey
Melnik, A., Ljubljanac, M., Lu, C., Yan, Q., Ren, W., & Ritter, H. (2024) · arXiv · doi:10.48550/arxiv.2405.03150 · Cited by 4 · see citing works
Glob-Diffusion: A Global Consistent Diffusion Model for Large-Scale Image Generation
Kang, Y., Shi, H., Liu, H., Xie, W., Fang, L., & Bruzzone, L. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3643743 · Cited by 1 · see citing works
Akdemir, K., & Yanardag, P. (2024) · arXiv · doi:10.48550/arxiv.2406.02820
“In addressing the challenge of achieving consistent character visualization across various applications of text-to-image diffusion models, the field has increasingly leaned on personalization techniques such as Dreambooth (Ruiz et al. 2022), Textual Inversion (Gal et al. 2022) or LoRAs (Hu et al. 2021).”
BackgroundMethodInfluentialKwon, P., & Chen, C. (2025) · arXiv · doi:10.48550/arxiv.2512.01686
“Early methods for subject customization were based on finetuning the model for each specific subject [16, 18, 24, 46], achieving strong consistency but requiring training for each instance.”
OmniGen2: Towards Instruction-Aligned Multimodal Generation
Wu, C., Zheng, P., Yan, R., Xiao, S., Luo, X., Wang, Y., Li, W., Jiang, X., Liu, Y., Zhou, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.18871
DiffSim: Taming Diffusion Models for Evaluating Visual Similarity
Song, Y., Liu, X., & Shou, M. Z. (2024) · arXiv · Cited by 1 · see citing works
“Customized generation methods enable flexible customization of concepts and styles by fine-tuning U-Net [55] or certain parameters [27, 37], alongside trainable tokens.”
- Authors
- Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., & Aberman, K.
- Affiliations
- Boston University (US)
- Google (United States) (US)
- Venue
- 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2023-06
- Pages
- 22500-22510
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4386072096
- Cited by
- 2,295
- References
- 10
- Citations by year
- 2026: 434 · 2025: 796 · 2024: 841 · 2023: 220 · 2022: 4
- FWCI
- 245.01
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- DreamBooth, text-to-image diffusion models, personalization, fine-tuning, artistic rendering, realistic image synthesis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
