Cited by 71Generative Adversarial Networks and Image SynthesisDigital Media and Visual ArtMultimodal Machine Learning Applications

Best Prompts for Text-to-Image Models and How to Find Them

Open access
Pavlichenko, Nikita, Ustalov, Dmitry·2023·Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

Abstract

Advancements in text-guided diffusion models have allowed for the creation of visually appealing images similar to those created by professional artists. The effectiveness of these models depends on the composition of the textual description, known as the prompt, and its accompanying keywords. Evaluating aesthetics computationally is difficult, so human input is necessary to determine the ideal prompt formulation and keyword combination. In this study, we propose a human-in-the-loop method for discovering the most effective combination of prompt keywords using a genetic algorithm. Our approach demonstrates how this can lead to an improvement in the visual appeal of images generated from the same description.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3539618.3592000

Cites

14 of 14 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 11

  1. Tom Brown et al. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems 33 (NeurIPS 2020).
  2. Curran Associates, Inc., Montréal, QC, Canada, 1877-1901. https://proceedings.neurips.cc/paper/2020/ file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf
  3. Tianyu Gao, Adam Fisch, and Danqi Chen. 2021. Making Pre-trained Lan- guage Models Better Few-shot Learners. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (ACL- IJCNLP 2021). Association for Computational Linguistics, Online, 3816-3830. https://doi.org/10.18653/v1/2021.acl-long.295 · DOI · Paper page
  4. Nikita Pavlichenko and Dmitry Ustalov. 2021. IMDB-WIKI-SbS: An Evaluation Dataset for Crowdsourced Pairwise Comparisons. , 5 pages. arXiv:2110.14990. · arXiv
  5. Fabio Petroni, Patrick Lewis, Aleksandra Piktus, Tim Rocktäschel, Yuxiang Wu, Alexander H. Miller, and Sebastian Riedel. 2020. How Context Affects Language Models' Factual Predictions. arXiv:2005.04611. · arXiv
  6. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of the 38th International Conference on Machine Learning (ICML 2021, Vol. 139). PMLR, Virtual Only, 8748- 8763.
  7. Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022. Hierarchical Text-Conditional Image Generation with CLIP Latents. arXiv:2204.06125. · arXiv
  8. Ohad Rubin, Jonathan Herzig, and Jonathan Berant. 2022. Learning To Retrieve Prompts for In-Context Learning. , 2655-2671 pages. https://doi.org/10.18653/ v1/2022.naacl-main.191 · DOI · Paper page
  9. Jiaming Song, Chenlin Meng, and Stefano Ermon. 2020. Denoising Diffusion Implicit Models. arXiv:2010.02502. · arXiv
  10. Dmitry Ustalov, Nikita Pavlichenko, and Boris Tseitlin. 2023. Learning from Crowds with Crowd-Kit. arXiv:2109.08584 [cs.HC] https://arxiv.org/abs/2109. 08584 · arXiv
  11. Bo Zhang, Li Niu, and Liqing Zhang. 2021. Image Composition Assessment with Saliency-augmented Multi-pattern Pooling. arXiv:2104.03133. · arXiv

Cited by

Sorted by citations

Authors
Pavlichenko, N., & Ustalov, D.
Affiliations
Toloka, Belgrade, Serbia
Venue
Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval
Publisher
ACM
Published
2023-07-18
Pages
2067-2071
OpenAlex
W4379466097
Cited by
71
References
12
Citations by year
2026: 7 · 2025: 16 · 2024: 33 · 2023: 15
FWCI
7.61
Type
conference paper
Language
en
Open access
Locations
  • International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Digital Media and Visual Art
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
text-to-image generation, prompt engineering, diffusion models, human-in-the-loop evaluation, genetic algorithm, image aesthetics, visual appeal

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review