The DOI matches its DataCite record; 69 of the 69 references it cites were found in a registry or an index.
Hierarchical Text-Conditional Image Generation with CLIP Latents
Open accessAbstract
Contrastive models like CLIP have been shown to learn robust representations of images that capture both semantics and style. To leverage these representations for image generation, we propose a two-stage model: a prior that generates a CLIP image embedding given a text caption, and a decoder that generates an image conditioned on the image embedding. We show that explicitly generating image representations improves image diversity with minimal loss in photorealism and caption similarity. Our decoders conditioned on image representations can also produce variations of an image that preserve both its semantics and style, while varying the non-essential details absent from the image representation. Moreover, the joint embedding space of CLIP enables language-guided image manipulations in a zero-shot fashion. We use diffusion models for the decoder and experiment with both autoregressive and diffusion models for the prior, finding that the latter are computationally more efficient and produce higher-quality samples.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While more sophisticated methods for partially editing individual frames during training exist (e.g., text-to-image models [13], [14]), they are slow, computationally expensive and error-prone.”
From Language-Based Augmentation to Address Shortcut Learning in Object-Goal Navigation (2023) · cites this paper as [13]
Experimental — accuracy not yet measuredIn the citing paper
“Generative machines, such as text-to-image algorithms, are contributing to the variety of cultural traits.”
From Machine culture (2023) · cites this paper as [20]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Part of this paper’s reference list could not be split into single entries.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“To this end, we generated 512 “artistic” captions using GPT-3 [4] by prompting it with captions for existing artwork (both real and AI generated).”
MethodSelf-citationHigh-Resolution Image Synthesis with Latent Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01042 · Cited by 20,302 · cited 2×
“[42] use diffusion models on the latents obtained from a VQGAN [14] like autoencoder.”
BackgroundMethodLearning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · cited 3×
“Recent progress in computer vision has been driven by scaling models on large datasets of captioned images collected from the internet [10, 44, 60, 39, 31, 16].”
BackgroundMethodSelf-citationLIII. On lines and planes of closest fit to systems of points in space
Pearson, K. (1901) · The London Edinburgh and Dublin Philosophical Magazine and Journal of Science · doi:10.1080/14786440109462720 · Cited by 12,864 · cited 4×
“After applying PCA, we order the principal components by decreasing eigenvalue magnitude, quantize each of the 319 dimensions into 1,024 discrete buckets, and predict the resulting sequence using a decoder-only Transformer [53] model with a causal attention mask.”
MethodInfluentialDecoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
Denoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712
Research works printed but not linked: 2
- [23][23] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium. Advances in Neural Information Processing Systems 30 (NIPS 2017), 2017.
- [32][32] Ryan Murdock. The Big Sleep. https://twitter.com/advadnoun/status/ 1351038053033406468, 2021. 20 https://openreview.net/forum?id=qw674L9PfQE https://openreview.net/forum?id=qw674L9PfQE https://arxiv.org/abs/2203.13131 https://arxiv.org/abs/2108.00946 https://arxiv.org/abs/2102.01645 https://arxiv.org/abs/1406.2661 https://arxiv.org/abs/2111.14822 https://openreview.net/forum?id=qw8AKxfYbI https://openreview.net/forum?id=qw8AKxfYbI https://arxiv.org/abs/2006.11239 https://arxiv.org/abs/2106.15282 https://arxiv.org/abs/1412.6980 https://arxiv.org/abs/1405.0312 https://arxiv.org/abs/1711.05101 https://github.com/openai/dalle-2-preview/blob/main/system-card.md https://github.com/openai/dalle-2-preview/blob/main/system-card.md https://arxiv.org/abs/2112.12750 https://twitter.com/advadnoun/status/1351038053033406468 https://twitter.com/advadnoun/status/1351038053033406468 · arXiv
Cited by
Sorted by citations
Items 601–700 of 1,051
Text-Image Alignment for Diffusion-Based Perception
Kondapaneni, N., Marks, M., Knott, M., Guimarães, R., & Perona, P. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01317 · Cited by 55
“Diffusion models have set the state-of-the-art (SOTA) for image generation [32, 35, 38, 52].”
BackgroundAttacks and Defenses for Generative Diffusion Models: A Comprehensive Survey
Truong, V. T., Dang, L., & Le, L. B. (2025) · ACM Computing Surveys · doi:10.1145/3721479 · Cited by 59
Diffusion models for robotic manipulation: a survey
Wolf, R., Shi, Y., Liu, S., & Rayyes, R. (2025) · Frontiers in Robotics and AI · doi:10.3389/frobt.2025.1606247 · Cited by 67
“…promising deep generative models in diverse domains, including computer vision (Ho et al., 2020; Song J. et al., 2021; Nichol and Dhariwal, 2021; Ramesh et al., 2022; Rombach et al., 2022a), natural language processing (Li et al., 2022; Zhang et al., 2023; Yu et al., 2022), and robotics (Chi et…”
The Emerging Social Status of Generative AI: Vocabularies of AI Competence in Public Discourse
Bran, E., Rughiniş, C., Nadoleanu, G., & Flaherty, M. G. (2023) · 2023 24th International Conference on Control Systems and Computer Science (CSCS) · doi:10.1109/cscs59211.2023.00068 · Cited by 36
Cross-modal text and visual generation: A systematic review. Part 1: Image to text
Żelaszczyk, M., & Mańdziuk, J. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.01.008 · Cited by 31
Ham2Pose: Animating Sign Language Notation into Pose Sequences
Shalev-Arkushin, R., Moryossef, A., & Fried, O. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02016 · Cited by 39
“Diffusion models [19,46] recently showed impressive results on image and video generation tasks [10,35,38].”
Towards machine learned generative design
Gradišar, L., Dolenc, M., & Klinc, R. (2024) · Automation in Construction · doi:10.1016/j.autcon.2024.105284 · Cited by 37
A-STAR: Test-time Attention Segregation and Retention for Text-to-image Synthesis
Agarwal, A., Karanam, S., Joseph, K. J., Saxena, A., Goswami, K., & Srinivasan, B. V. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00217 · Cited by 48
“While the recent class of pixel [20, 23] and latent [21] diffusion models have shown unprecedented image generation results, they have some key limitations.”
BackgroundLv, K., Yuan, L., & Ni, X. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3375973 · Cited by 35
“Since then, DM has achieved remarkable results in various multimodal generation tasks, including image generation [52], [53], audio generation [54], and time series prediction [55], [56].”
BackgroundBAGM: A Backdoor Attack for Manipulating Text-to-Image Generative Models
Vice, J., Akhtar, N., Hartley, R. I., & Mian, A. S. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3386058 · Cited by 43
“The hierarchical image generation architecture contains decoder and encoder processes, allowing for the production of semantically similar output images [13].”
BackgroundMethodInfluentialDiff2Lip: Audio Conditioned Diffusion Models for Lip-Synchronization
Mukhopadhyay, S., Suri, S., Gadde, R. T., & Shrivastava, A. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00521 · Cited by 49
“Meth-ods like GLIDE [32], DALLE-2 [36], Stable-Diffusion [37], and IMAGEN [40] leverage language models to generate photorealistic as well as many other styles of images just us-ing text prompt inputs.”
BackgroundHelpless infants are learning a foundation model
Cusack, R., Ranzato, M., & Charvet, C. J. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.05.001 · Cited by 37
Ferrante, M., Boccato, T., Bargione, S., & Toschi, N. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108701 · Cited by 36
Drive Anywhere: Generalizable End-to-end Autonomous Driving with Multi-modal Foundation Models
Wang, T.-H. J., Maalouf, A., Xiao, W., Ban, Y., Amini, A., Rosman, G., Karaman, S., & Rus, D. L. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10611590 · Cited by 52
“Moreover, these models have demonstrated their versatility across multiple data modalities [3], [25], [29]–[32] marking a new era of robots that can reason and interact wisely with the environment.”
A Representative Study on Human Detection of Artificially Generated Media Across Countries
Frank, J., Herbert, F., Ricker, J., Schönherr, L., Eisenhofer, T., Fischer, A., Dürmuth, M., & Holz, T. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00159 · Cited by 66
Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generation
Li, H., Shen, C., Torr, P. H. S., Volker Tresp, C., & Gu, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01141 · Cited by 56
“In particular, text-to-image diffusion models, which have exhibited remarkable performance in creating images from text prompts [13, 15, 20, 31, 35, 37, 49], raise concerns about the risks of generating inappropriate content.”
BackgroundFocus on Your Instruction: Fine-grained and Multi-instruction Image Editing by Attention Modulation
Guo, Q., & Lin, T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00667 · Cited by 52
“Recently, methods based on diffusion models [19, 49] have showcased unprecedented prowess in image generation and editing [5, 33, 41, 43, 45, 47, 63].”
BackgroundMethodDermawan, A. (2023) · The Journal of World Intellectual Property · doi:10.1111/jwip.12285 · Cited by 31
RIATIG: Reliable and Imperceptible Adversarial Text-to-Image Generation with Natural Prompts
Liu, H., Wu, Y., Zhai, S., Yuan, B., & Zhang, N. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01972 · Cited by 34
“Driven by recent advances in models trained with large datasets [42, 57] and multimodal learning [45] (e.g., diffusion models [17]), text-to-image generation has made significant progress in synthesizing high-fidelity and photorealistic images, such as DALL · E [43], DALL · E 2 [42] and Imagen [45].”
BackgroundMethodLayoutDiffusion: Improving Graphic Layout Generation by Discrete Diffusion Probabilistic Models
Zhang, J., Guo, J., Sun, S., Lou, J.-G., & Zhang, D. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00664 · Cited by 35
“Meanwhile, the emerging diffusion probabilistic model (DDPM) [13, 38] achieves amazing performance on many generation tasks [12, 21, 33–36 , 44 ].”
BackgroundDS-Fusion: Artistic Typography via Discriminated and Stylized Diffusion
Tanveer, M., Wang, Y., Amiri, A. M., & Zhang, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00041 · Cited by 39
“Qualitative and quantitative comparisons are made to the closest alternatives that can be adapted to produce similar forms of typographies, including DALL-E 2 [20], Stable Diffusion (SD) [21], CLIPDraw [7], and a Google search baseline.”
BackgroundMethodResultInfluentialPutting People in Their Place: Affordance-Aware Human Insertion into Scenes
Kulal, S., Brooks, T., Aiken, A., Wu, J., Yang, J., Lu, J., Efros, A. A., & Singh, K. K. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01639 · Cited by 43
“With a straightforward architecture, they achieve promising performance in several text-to-image [44, 49, 50, 54], video [29, 57], and 3D synthesis [48] tasks.”
InfluentialWu, Q., Liu, Y., Zhao, H., Bui, T., Lin, Z., Zhang, Y., & Chang, S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00714 · Cited by 40
“Particularly, text-to-image diffusion models [2, 33, 36, 40, 42] allow generating images conditioned on a text description, which enables generation of creative images due to the expressiveness of natural language.”
BackgroundX-Mesh: Towards Fast and Accurate Text-driven 3D Stylization via Dynamic Textual Guidance
Ma, Y., Wang, H., Zhang, X., Jiang, G., Sun, X., Zhuang, W., Ji, J., & Ji, R. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00258 · Cited by 31
“DALL-E2 [49] generates original and realistic images given a text prompt by encoding image features according to the text features of CLIP and then decod-2750 ing them via a diffusion model.”
MethodPrompt learning in computer vision: a survey
Lei, Y., Li, J., Li, Z., Cao, Y., & Shan, H. (2024) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2300389 · Cited by 30
ControlRoom3D: Room Generation Using Semantic Proxy Rooms
Schult, J., Tsai, S. S., Höllein, L., Wu, B., Wang, J., Ma, C., Li, K., Wang, X., Wimbauer, F., He, Z., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00593 · Cited by 58
“Fueled by the availability of large-scale vision-text datasets [22, 32, 43–45, 52, 57] and powerful vision-language models [35], recent advancements in 2D diffusion models [5, 11, 31, 36, 40, 41] have significantly contributed to the ongoing progress in generating 3D objects and scenes.”
BackgroundDiffusion Models Without Attention
Yan, J. N., Gu, J., & Rush, A. M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00787 · Cited by 62
“Diffusion and score-based generative models [26, 58–61] have shown considerable improvements, especially in image generation tasks [46–48].”
BackgroundInitno: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization
Guo, X., Liu, J., Cui, M., Li, J., Hongyu, Y., & Huang, D. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00896 · Cited by 66
“The integration of large-scale vision-language models [2, 11, 14, 17, 30–32, 40, 41] has propelled noteworthy advancements in the text-to-image domain.”
BackgroundCCEdit: Creative and Controllable Video Editing via Diffusion Models
Feng, R., Weng, W., Wang, Y., Yuan, Y., Bao, J., Luo, C., Chen, Z., & Guo, B. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00641 · Cited by 67
“A large body of prior research has demonstrated the exceptional capabilities of diffusion models in generating diverse and high-quality images [23, 27, 30] and videos [3, 12, 33], conditioned by text prompts.”
PADL: Language-Directed Physics-Based Character Control
Juravsky, J., Guo, Y., Fidler, S., & Peng, X. B. (2022) · SIGGRAPH Asia Conference Papers · doi:10.1145/3550469.3555391 · Cited by 40
“…for integrating natural language interfaces for a wide range of downstream applications [Brown et al. 2020; Devlin et al. 2018; Radford et al. 2021], such as generating functional code and realistic images from natural language descriptions [Chen et al. 2021; Ramesh et al. 2022; Tan et al. 2018].”
BackgroundMethodPhotoMat: A Material Generator Learned from Single Flash Photos
Zhou, X., Hašan, M., Deschaintre, V., et al. (2023) · doi:10.1145/3588432.3591535 · Cited by 31
“Scaling up our dataset size is likely to lead to the most dramatic improvements, much like with generative models in other domains [Ramesh et al. 2022].”
MethodDiffusion Models for Non-autoregressive Text Generation: A Survey
Li, Y., Zhou, K., Zhao, W. X., & Wen, J.-R. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/750 · Cited by 67
Contrastive Transformer Learning With Proximity Data Generation for Text-Based Person Search
Wu, H., Chen, W., Liu, Z., Chen, T., Chen, Z., & Lin, L. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3329220 · Cited by 34
“Since Transformer is more data-hungry due to lack of inductive bias, BeiT [27] proposes a distillation framework to train the ViT without the need of large-scale datasets.”
BackgroundSeger, E., Dreksler, N., Moulange, R., Dardaman, E., Schuett, J., Wei, K., Winter, C., Arnold, M., Ó hÉigeartaigh, S., Korinek, A., et al. (2023) · doi:10.2139/ssrn.4596436 · Cited by 31
“For instance, Stable Diffusion [27] and DALL·E 2 [28] combine natural language processing capabilities with image generation capabilities to translate natural language prompts into image outputs.”
BackgroundControllable Person Image Synthesis with Pose-Constrained Latent Diffusion
Han, X., Zhu, X., Deng, J., Song, Y.-Z., & Xiang, T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02081 · Cited by 37
“Inspired by the massive advance of image generation and editing [6, 29, 32, 33], diffusion models have been recently exploited for CPIS with the best-ever results achieved [2].”
MethodRAVE: Randomized Noise Shuffling for Fast and Consistent Video Editing with Diffusion Models
Kara, Ö., Kurtkaya, B., Yesiltepe, H., Rehg, J. M., & Yanardag, P. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00622 · Cited by 63
“Diffusion-based generative models [16, 43] demonstrate outstanding capabilities in generating and editing diverse and high-quality images [36, 37, 40] guided by text prompts.”
BackgroundSmooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Models
Guo, J., Xu, X., Pu, Y., Ni, Z., Wang, C., Vasu, M., Song, S., Huang, G. M., & Shi, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00721 · Cited by 43
“Recently, generating images from text prompts (T2I) become an emerging field [11, 19, 28, 47, 62], among which diffusion models [16, 49, 59, 62, 64] have become quite visible to the public.”
BackgroundArtAdapter: Text-to-Image Style Transfer using Multi-Level Style Encoder and Explicit Adaptation
Chen, D.-Y., Tennent, H., & Hsu, C.-W. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00823 · Cited by 41
“Subsequent advancements with diffusion models [19] further refined the generation with a gradual denoising process, offering enhanced control and fidelity in outputs, as showcased by DALL-E2 [35], Imagen [41], and Stable Diffusion [37].”
MethodShan, S., & Li, Y. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2423335 · Cited by 36
“DALL-E 2 further scaled up the model size and training data to generate more realistic and diverse images (Ramesh et al., 2022).”
Closer Worlds: Using Generative AI to Facilitate Intimate Conversations
Chen, T., Lee, C., Mindel, J. R., Elhaouij, N., & Picard, R. W. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3585651 · Cited by 33
“Generative AI tools have made creative exploration more accessible: recent rapid advances in difusion models [45] allow users to craft coherent and emotionally resonant images from text alone, granting users easy access to the Internet’s collective imagination.”
BackgroundCatch Missing Details: Image Reconstruction with Frequency Augmented Variational Autoencoder
Lin, X., Li, Y., Hsiao, J.-H., Ho, C., & Kong, Y. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00173 · Cited by 33
“Diffusionbased models [5, 7, 10, 27, 28, 37, 40, 41] generate images from Gaussian noise through a reverse diffusion process can often require substantial training and large datasets.”
BackgroundDomain-specific chatbots for science using embeddings
Yager, K. G. (2023) · Digital Discovery · doi:10.1039/d3dd00112a · Cited by 35
Bian, C., Wang, X., Huang, Y., Zhou, S., & Lu, W. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-05860-2 · Cited by 30
A Survey of Cross-Modal Visual Content Generation
Nazarieh, F., Feng, Z., Awais, M., Wang, W., & Kittler, J. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3351601 · Cited by 32
“The architectures of DALL-E2 [57] and stable diffusion [36]. of CLIP embeddings, identity information related to the target domain can be learned as well.”
BackgroundHierarchical Generation of Human-Object Interactions with Diffusion Probabilistic Models
Pi, H., Peng, S., Yang, M., Zhou, X., & Bao, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01383 · Cited by 37
“Later, the classifier-free model [24] achieves better results [48] in text-conditioned image generation.”
Follow Anything: Open-Set Detection, Tracking, and Following in Real-Time
Maalouf, A., Jadhav, N., Murthy, J. K., Chahine, M., Vogt, D. M., Wood, R. J., Torralba, A. B., & Rus, D. L. (2024) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2024.3366013 · Cited by 38
“Thus, to automatically identify the required object to follow, numerous detection approaches have been suggested [17], [18] such as RCNN and its variants [19], [20], [21], YOLO and its variants [22], [23], [24], and more [25], [26].”
MethodDreamStyler: Paint by Style Inversion with Text-to-Image Diffusion Models
Ahn, N., Lee, J., Lee, C., Kim, K., Kim, D., Nam, S.-H., & Hong, K. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i2.27824 · Cited by 37
Multi-Class Segmentation from Aerial Views using Recursive Noise Diffusion
Kolbeinsson, B., & Mikolajczyk, K. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00825 · Cited by 34
“Thus, they have recently become exceedingly popular for image generation [17, 40, 41, 43].”
BackgroundLatent Feature-Guided Diffusion Models for Shadow Removal
Mei, K., Figueroa, L., Lin, Z., Ding, Z., Cohen, S. D., & Patel, V. M. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00426 · Cited by 43
“Typical guidance for diffusion models includes class information [4], text description [35, 38], and even gradients [18].”
BackgroundDeep generative models for detector signature simulation: A taxonomic review
Hashemi, H., & Krause, C. (2024) · Reviews in Physics · doi:10.1016/j.revip.2024.100092 · Cited by 56
“Diffusion Models Diffusion models or deep score-based generative models [74–78] are a class of generative processes inspired by non-equilibrium thermodynamics [79] and PDEs [80].”
BackgroundLee, J.-K., Jeong, H., Kim, Y., & Cha, S. H. (2024) · Advanced Engineering Informatics · doi:10.1016/j.aei.2024.102686 · Cited by 37
EDMP: Ensemble-of-costs-guided Diffusion for Motion Planning
Saha, K., Mandadi, V., Reddy, J., Srikanth, A., Agarwal, A., Sen, B., Singh, A. K., & Krishna, M. (2024) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra57147.2024.10610519 · Cited by 49
Diffusion-Based Cloud-Edge-Device Collaborative Learning for Next POI Recommendations
Long, J., Ye, G., Chen, T., Wang, Y., Wang, M., & Yin, H. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671743 · Cited by 35
“Diffusion models have revolutionized generative tasks across fields like computer vision (CV), natural language processing (NLP), and others [1, 17, 32, 45], with Denoising Diffusion Probabilistic Models (DDPMs) [11] excelling in creating high-quality images.”
It's All About Your Sketch: Democratising Sketch Control in Diffusion Models
Koley, S., Bhunia, A. K., Sekhri, D., Sain, A., Chowdhury, P. N., Xiang, T., & Song, Y.-Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00688 · Cited by 35
“The unconditional denoising diffusion process could be made “conditional” by influencing the F θ with auxiliary conditioning signals d ( e.g ., textual description [58, 61, 64], etc.).”
Diff-Plugin: Revitalizing Details for Diffusion-Based Low-Level Tasks
Liu, Y., Ke, Z., Liu, F., Zhao, N., & Lau, R. W. H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00402 · Cited by 41
“With extensive text-image data [59] and large-scale language models [49, 50], diffusion-based text-guided image synthesis [2, 42, 51, 54, 57] has become even more compelling.”
BackgroundConvoFusion: Multi-Modal Conversational Diffusion for Co-Speech Gesture Synthesis
Mughal, M. H., Dabral, R., Habibie, I., Donatelli, L. E., Habermann, M., & Theobalt, C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00138 · Cited by 43
“Diffusion models [30, 66] have demonstrated remarkable potential in the field of generative modeling, consistently delivering impressive results in various synthesis applications [14, 34, 60, 63, 67, 70, 79].”
BackgroundEpiDiff: Enhancing Multi-View Synthesis via Localized Epipolar-Constrained Diffusion
Huang, Z., Wen, H., Dong, J., Wang, Y., Li, Y., Chen, X., Cao, Y., Ding, L., Qiao, Y., Dai, B., et al. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00934 · Cited by 57
“3D generation guided by 2D prior Given the robust generative capabilities of 2D diffusion models [39, 41, 42], some research efforts directly leverage pretrained image diffusion models.”
MethodContrastive Denoising Score for Text-Guided Latent Diffusion Image Editing
Nam, H., Kwon, G., Park, G. Y., & Ye, J. C. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00878 · Cited by 41
“Evolving from basic models, recent Latent Diffusion Model(LDM) showed notable efficacy in T2I task [23–25].”
BackgroundMethodBuehler, M. J. (2023) · Journal of materials research/Pratt's guide to venture capital sources · doi:10.1557/s43578-023-00892-3 · Cited by 32
“, VQGAN-CLIP [20, 30], GLIDE [31], DALL-E 2 [32, 33], or Imagen [34]).”
Survey on leveraging pre-trained generative adversarial networks for image editing and restoration
Liu, M., Wei, Y., Wu, X., Zuo, W., & Zhang, L. (2023) · Science China Information Sciences · doi:10.1007/s11432-022-3679-0 · Cited by 38
“They are also successfully applied in several large-scale text-guided image generation models like DALL · E 2 [191] and Imagen [192].”
BackgroundMethodPrompt Evolution for Generative AI: A Classifier-Guided Approach
Wong, M., Ong, Y.-S., Gupta, A., Bali, K. K., & Chen, C. (2023) · IEEE Conference on Artificial Intelligence (CAI) · doi:10.1109/cai54212.2023.00105 · Cited by 36
“Research addressing this limitation has mainly focused on enhancing the prompts before output generation [4]–[7] or improving the model’s performance beforehand [8]–[10].”
Character-Aware Models Improve Visual Text Rendering
Liu, R., Garrette, D., Saharia, C., Chan, W., Roberts, A., Narang, S., Blok, I., Mical, R., Norouzi, M., & Constant, N. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.900 · Cited by 76
MaskSketch: Unpaired Structure-guided Masked Image Generation
Bashkirova, D., Lezama, J., Sohn, K., Saenko, K., & Essa, I. A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00187 · Cited by 35
“While there is a vast volume of literature on image generative models thanks to recent progress ranging from generative adversarial networks [2, 18, 27] generative transformers [4, 13, 54] and diffusion models [11, 35, 40, 46], in this section, we focus on reviewing image-conditioned image generation, also known as im…”
BackgroundGeneralized Deep 3D Shape Prior via Part-Discretized Diffusion Process
Li, Y., Dou, Y., Chen, X., Ni, B., Sun, Y., Liu, Y., & Wang, F. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01610 · Cited by 33
Beyond Deepfake Images: Detecting AI-Generated Videos
Vahdati, D. S., Nguyen, T. D., Azizpour, A., & Stamm, M. C. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00443 · Cited by 37
SemEval-2023 Task 1: Visual Word Sense Disambiguation
Raganato, A., Calixto, I., Ushio, A., Camacho-Collados, J., & Pilehvar, M. T. (2023) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2023.semeval-1.308 · Cited by 28
“This is relevant given the well-known disambiguation issues for image recognition models, as also observed for recent image generation systems, such as DALLE-2 (Ramesh et al., 2022; Rassin et al., 2022).”
Hyperspectral and Panchromatic Images Fusion Based on the Dual Conditional Diffusion Models
Li, S., Li, S., & Zhang, L. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3321318 · Cited by 30
“DDPM can closely learn the distribution of the training data through the inter-pretable VLB loss function and provides excellent generative performance in image generation [26], [27], [28], super-resolution [22], [23], [29], [30], [31], repair [32], and so on.”
BackgroundFrom Paper to Card: Transforming Design Implications with Generative AI
Shin, D.-H., Wang, L. L., & Hsieh, G. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642266 · Cited by 35
“A well-known example of a text-to-image model is DALL-E 2 [52], a transformer-based diffusion model that produces images based on descriptive textual prompts.”
BackgroundMethodInfluentialStyle2Talker: High-Resolution Talking Head Generation with Emotion Style and Art Style
Tan, S., Ji, B., & Pan, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i5.28313 · Cited by 41
“Intrigued by the success of diffusion prior in DALL · E2 (Ramesh et al. 2022), µ θ directly predicts emotional motion sequence ˆ 1: N 0 = µ θ ( β 1: N t , c , t ) , instead of predicting the noise as in the vanilla DDPM (Ho, Jain, and Abbeel 2020).”
BackgroundAnomaly Detection for Telemetry Time Series Using a Denoising Diffusion Probabilistic Model
Sui, J., Yu, J., Song, Y., & Zhang, J. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3383416 · Cited by 32
“Recently, the denoising diffusion probabilistic model (DDPM) emerged as a new family of deep generative models and has beaten AE and GAN in the challenge of image synthesis [37], [38], [39].”
BackgroundTraining-free Content Injection using h-space in Diffusion Models
Jeong, J., Kwon, M., & Uh, Y. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00507 · Cited by 60
“In this sense, text-to-image DMs provide a way to reflect a given text for generating diverse images using classifier-free guidance [3,20,53,59,60,65].”
BackgroundOrthogonal Adaptation for Modular Customization of Diffusion Models
Po, R., Yang, G., Aberman, K., & Wetzstein, G. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00761 · Cited by 52
“Mix-of-Show (FedAvg) Orthogonal Adaptation (Ours)”
One-dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applications
Lyu, M., Yang, Y., Hong, H., Chen, H., Jin, X., He, Y., Xue, H., Han, J., & Ding, G. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00722 · Cited by 67
“Text-to-image diffusion models (DMs) [3, 12, 13, 26– 28, 33, 36, 38, 39, 47, 48, 50] have shown appealing advancement in high-quality image creation in the span of seconds, powered by pre-training on web-scale datasets.”
BackgroundRelightful Harmonization: Lighting-Aware Portrait Background Replacement
Ren, M., Xiong, W., Yoon, J. S., Shu, Z., Zhang, J., Jung, H., Gerig, G., & Zhang, H. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00617 · Cited by 52
“Diffusion Models [11, 19, 51, 52] have significantly advanced the image and video synthesis quality [18, 20, 24, 25, 45, 47].”
On the Content Bias in Fréchet Video Distance
Ge, S., Mahapatra, A., Parmar, G., Zhu, J.-Y., & Huang, J.-B. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00695 · Cited by 50
“Following the recent success of text-to-image Diffusion Models [49, 51, 53], several works aim to achieve high-quality results for the text-to-video task.”
BackgroundFine-grained Audible Video Description
Shen, X., Li, D., Zhou, J., Qin, Z., He, B., Han, X., Li, A., Dai, Y., Kong, L., Wang, M., et al. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01020 · Cited by 34
“…to other modalities, including image/video captioning [16, 32, 45, 48, 51, 64, 65, 75, 76, 85, 92], text-to-image/video synthesis [15,19–21, 58, 59, 62, In this paper, we consider language as a tool for conveying information from other modalities in multi-modal modeling and propose a new…”
MethodExposing fake images generated by text-to-image diffusion models
Xu, Q., Wang, H., Meng, L., Mi, Z., Yuan, J., & Yan, H. (2023) · Pattern Recognition Letters · doi:10.1016/j.patrec.2023.10.021 · Cited by 28
Practical Membership Inference Attacks Against Large-Scale Multi-Modal Models: A Pilot Study
Ko, M., Jin, M., Wang, C., & Jia, R. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00449 · Cited by 33
“We use CLIP as an example for the extensive evaluation, due to the following reasons: First, CLIP is widely used for zero-shot learning in various computer vision tasks [25, 24, 8], which entails the understanding of its privacy risk.”
MethodInfluentialFeatureNeRF: Learning Generalizable NeRFs by Distilling Foundation Models
Ye, J., Wang, N., & Wang, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00823 · Cited by 45
“After demonstrating huge impacts in NLP [4, 15, 41], a large family of vision foundation models [5,40,42,43,47,58,62,67] have been proposed and effectively transferred to various vision tasks.”
BackgroundShadows Don't Lie and Lines Can't Bend! Generative Models Don't know Projective Geometry…for Now
Sarkar, A., Mai, H., Mahapatra, A., Lazebnik, S., Forsyth, D. A., & Bhattad, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02658 · Cited by 43
“Pioneering efforts by Karras et al. [22–24] with StyleGAN, and the emergence of diffusion models [36–38], have set new benchmarks in realism.”
BackgroundOpen data and algorithms for open science in AI-driven molecular informatics
Brinkhaus, H. O., Rajan, K., Schaub, J., Zielesny, A., & Steinbeck, C. (2023) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2023.102542 · Cited by 29
Diffusing the Creator: Attributing Credit for Generative AI Outputs
Khosrowi, D., Finn, F., & Clark, E. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604716 · Cited by 28
“Unlike earlier systems based on generative adversarial networks [28, 53], many recent GAI systems are based on encoder-decoder deep neural network (DNN) architectures and involve diffusion models as decoders [54, 56].”
Face manipulated deepfake generation and recognition approaches: a survey
Rehaan, M., Kaur, N., & Kingra, S. (2023) · Smart Science · doi:10.1080/23080477.2023.2268380 · Cited by 34
“In recent study by Ramesh et al. [25], it was found that models like Contrastive Language-Image Pretraining (CLIP) can represent images well, capturing both their meaning and style.”
MethodShentu, J., & Moubayed, N. A. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00513 · Cited by 30
“On the other hand, denoising diffusion models are proposed recently, which avoid these problems by adopting likelihood-based models and have been verified to outperform GANs in terms of the generation quality in general fields [10, 14, 32, 36].”
BackgroundZeng, X., Gao, Z., Ye, Y., & Zeng, W. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642165 · Cited by 35
“Recent advancements in pre-trained text-to-image generative models, such as Stable Diffusion [41] and DALL-E-2 [39], have facilitated the generation of high-quality images from natural language descriptions ( i.e. , prompts) [44].”
Amodal Completion via Progressive Mixed Context Diffusion
Xu, K. H., Zhang, L., & Shi, J. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00869 · Cited by 40
“Inspired by non-equilibrium thermo-dynamics [41], diffusion models achieve remarkable results for text-to-image and image inpainting tasks [31, 37, 38, 40], often outperforming GANs in generating photorealistic and diverse images [7, 14, 30].”
BackgroundInstaGen: Enhancing Object Detection by Training on Synthetic Dataset
Feng, C., Zhong, Y., Jie, Z., Xie, W., & Ma, L. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01339 · Cited by 30
“DALL-E 2 [25] is trained to generate images by inverting the CLIP image space, while Ima-gen [29] explores the advantages of using pre-trained language models.”
MethodVlogger: Make Your Dream A Vlog
Zhuang, S., Li, K., Chen, X., Wang, Y., Liu, Z., Qiao, Y., & Wang, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00841 · Cited by 62
“Based on the success of diffusion models in the Text-to-Image (T2I) generation [4, 6, 18, 40, 45, 51, 52, 54], a series of such works have been recently transferred to T2V generation [8, 25, 33, 57, 70, 71, 76, 87].”
UniAnimate: taming unified video diffusion models for consistent human image animation
Wang, X., Zhang, S., Gao, C., Wang, J., Zhou, X., Zhang, Y., Yan, L., & Sang, N. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4592-3 · Cited by 103
Yager, K. G., Majewski, P. W., Noack, M. M., & Fukuto, M. (2023) · Nanotechnology · doi:10.1088/1361-6528/acd25a · Cited by 29
“For instance, AI text-to-image systems (such as DALL-E 2 [42], Midjourney [43], or Stable Diffusion [44]) are designed as an aid to human imagination and creativity, where the user inputs and iteratively adjusts text prompts and settings to create desired output.”
Creative AI for HRI Design Explorations
Hoggenmüeller, M., Lupetti, M. L., van der Maden, W., & Grace, K. S. (2023) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3568294.3580035 · Cited by 33
“The rapid development and increasing accessibility of generative AI systems, such as Stable Diffusion [62] and DALL-E [59], brought forward a range of examples that demonstrate how these tools can”
BackgroundInfluentialWho is Alyx? A new behavioral biometric dataset for user identification in XR
Räck, C., Fernando, T., Yalcin, M., Andreas Hotho, A., & Latoschik, M. E. (2023) · Frontiers in Virtual Reality · doi:10.3389/frvir.2023.1272234 · Cited by 30
“, 2021) or the recent DALL-E2 (text to image) (Ramesh et al., 2022) and ChatGPT (OpenAI, 2022).”
BackgroundAre Deep Neural Networks SMARTer Than Second Graders?
Cherian, A., Peng, K., Lohit, S., Smith, K. A., & Tenenbaum, J. B. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01043 · Cited by 44
“, the recent DALLE-2 [54] and ImageGen [56]), creative story writing (e.”
BackgroundSocial Diffusion: Long-term Multiple Human Motion Anticipation
Tanke, J., Zhang, L., Zhao, A. T., Tang, C., Cai, Y., Wang, L., Wu, P.-C., Gall, J., & Keskin, C. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00880 · Cited by 30
“Following [24, 62, 51], the loss during training is defined by: For inference, we reverse-iterate over Equation (3), starting at sampling step T and latent representation X T ∼ N ( 0 , 1 ) .”
MethodRe-scoring using image-language similarity for few-shot object detection
Jung, M. J., Han, S. D., & Kim, J. (2024) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2024.103956 · Cited by 28
“In the text-to-image generation domain [38, 39], it is used to find an image embedding that is located in the same position as the text in the pre-trained embedding space to generate an image related to the text.”
MethodDeep Image Fingerprint: Towards Low Budget Synthetic Image Detection and Model Lineage Analysis
Sinitsa, S., & Fried, O. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00402 · Cited by 46
“We observe that the accuracy remains stable across most of the models and starts to degrade drastically at N S = 128 , where we observe a 5% drop for the majority of the models.”
BackgroundAIGCOIQA2024: Perceptual Quality Assessment of AI Generated Omnidirectional Images
Yang, L., Duan, H., Teng, L., Zhu, Y., Liu, X., Hu, M., Min, X., Zhai, G., & Callet, P. L. (2024) · arXiv · doi:10.48550/arxiv.2404.01024 · Cited by 27
Repository-Level Prompt Generation for Large Language Models of Code
Shrivastava, D., Larochelle, H., & Tarlow, D. (2022) · arXiv · doi:10.48550/arxiv.2206.12839 · Cited by 27
“Large Language Models (LLMs) have demonstrated remarkable performance in natural language processing tasks (Brown et al., 2020; Chowdhery et al., 2022), text-to-image generation (Ramesh et al., 2022; Rombach et al., 2022) and even as a generalized agent (Reed et al., 2022).”
BackgroundIn silico ADME/tox comes of age: twenty years later
Ekins, S., Lane, T. R., Urbina, F., & Puhl, A. C. (2023) · Xenobiotica · doi:10.1080/00498254.2023.2245049 · Cited by 30
“Outside of drug discovery, recent advances in machine learning in image generation (Ramesh et al. 2022) and the advent of Transformer-based large language models (LLMs) (Vaswani et al. 2017) have shifted the landscape of machine learning entirely, with accelerated research into the scaling of both…”
Tell2Design: A Dataset for Language-Guided Floor Plan Generation
Leng, S., Zhou, Y. M., Dupty, M. H., Lee, W. S., Joyce, S. C., & Lu, J. W. Z. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.820 · Cited by 41
“Recently, text-conditional generative AI models (Nichol et al., 2022; Saharia et al., 2022b; Ramesh et al., 2022; Dhariwal and Nichol, 2021; Ho et al., 2022) have demonstrated impressive results in generating high-fidelity images.”
Background
Metadata
- Authors
- Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., & Chen, M.
- ORCID
- Chen, Mark
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2204.06125
- OpenAlex
- W4224035735
- Cited by
- 2,511
- References
- 66
- Citations by year
- 2026: 46 · 2025: 429 · 2024: 1113 · 2023: 859 · 2022: 63
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Keywords
- text-to-image generation, diffusion models, image priors, image variation, photorealism
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
