Unleashing the potential of prompt engineering for large language models
Open accessThe DOI matches its Crossref record; 295 of the 295 references it cites were found in a registry or an index.
Abstract
This review explores the role of prompt engineering in unleashing the capabilities of large language models (LLMs). Prompt engineering is the process of structuring inputs, and it has emerged as a crucial technique for maximizing the utility and accuracy of these models. Both foundational and advanced prompt engineering methodologies-including techniques such as self-consistency, chain of thought, and generated knowledge, which can significantly enhance the performance of models-are explored in this paper. Additionally, the prompt methods for vision language models (VLMs) are examined in detail. Prompt methods are evaluated with subjective and objective metrics, ensuring a robust analysis of their efficacy. Critical to this discussion is the role of prompt engineering in artificial intelligence (AI) security, particularly in terms of defending against adversarial attacks that exploit vulnerabilities in LLMs. Strategies for minimizing these risks and improving the robustness of models are thoroughly reviewed. Finally, we provide a perspective for future research and applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–295 of 295
InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents
Zhan, Q., Liang, Z., Ying, Z., & Kang, D. (2024) · arXiv · doi:10.48550/arxiv.2403.02691 · Cited by 143
Adversarial Demonstration Attacks on Large Language ModelsPreprint
Wang, J., Liu, Z., Park, K. H., Jiang, Z., Zheng, Z., Wu, Z., Chen, M., & Xiao, C. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.14950 · Cited by 139
A Diverse Corpus for Evaluating and Developing English Math Word Problem Solvers
Miao, S.-Y., Liang, C.-C., & Su, K.-Y. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.92 · Cited by 135
Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models
Logan IV, R., Balazevic, I., Wallace, E., Petroni, F., Singh, S., & Riedel, S. (2022) · Findings of the Association for Computational Linguistics: ACL 2022 · doi:10.18653/v1/2022.findings-acl.222 · Cited by 133
Evaluating Evaluation Methods for Generation in the Presence of Variation
Stent, A., Marge, M., & Singhai, M. (2005) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-30586-6_38 · Cited by 126
Chen, W., Ma, X., Wang, X., & Cohen, W. W. (2022) · arXiv · doi:10.48550/arxiv.2211.12588 · Cited by 125
ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language
Tafjord, O., Mishra, B. D., & Clark, P. (2020) · arXiv · doi:10.48550/arxiv.2012.13048 · Cited by 122
Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference
Chiang, W.-L., Zheng, L., Sheng, Y., Angelopoulos, A. N., Li, T., Li, D., Zhang, H., Zhu, B., Jordan, M., Gonzalez, J. E., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.04132 · Cited by 118
Thieves on Sesame Street! Model Extraction of BERT-based APIs
Krishna, K., Tomar, G. S., Parikh, A. P., Papernot, N., & Iyyer, M. (2019) · arXiv · doi:10.48550/arxiv.1910.12366 · Cited by 111
Ignore Previous Prompt: Attack Techniques For Language Models
Perez, F., & Ribeiro, I. (2022) · arXiv · doi:10.48550/arxiv.2211.09527 · Cited by 110
Zhang, Z., Gao, J., Dhaliwal, R. S., & Li, T. J.-J. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606800 · Cited by 108
Self-citationWebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents
Yao, S., Chen, H., Yang, J., & Narasimhan, K. (2022) · arXiv · doi:10.48550/arxiv.2207.01206 · Cited by 107
WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents
Yao, S., Chen, H., Yang, J., & Narasimhan, K. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1508 · Cited by 107
Understanding and Mitigating Overfitting in Prompt Tuning for Vision-Language Models
Ma, C., Liu, Y., Deng, J., Xie, L., Dong, W., & Xu, C. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3245584 · Cited by 104
Prompt Injection attack against LLM-integrated Applications
Liu, Y., Deng, G., Li, Y., Wang, K., Wang, Z., Wang, X., Zhang, T., Liu, Y., Wang, H., Zheng, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05499 · Cited by 100
Securing Large Language Models: Threats, Vulnerabilities and Responsible PracticesPreprint
Abdali, S., Anarfi, R., Barberan, C., He, J., & Shayegani, E. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.12503 · Cited by 100
Unsolved Problems in ML Safety
Hendrycks, D., Carlini, N., Schulman, J., & Steinhardt, J. (2021) · arXiv · doi:10.48550/arxiv.2109.13916 · Cited by 95
Mitchell, E., Lin, C., Bosselut, A., Finn, C., & Manning, C. D. (2021) · arXiv · doi:10.48550/arxiv.2110.11309 · Cited by 95
Wu, Y., Rabe, M. N., Hutchins, D., & Szegedy, C. (2022) · arXiv · doi:10.48550/arxiv.2203.08913 · Cited by 90
Unit Dependency Graph and Its Application to Arithmetic Word Problem Solving
Roy, S., & Roth, D. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10959 · Cited by 90
A very preliminary analysis of DALL-E 2
Marcus, G., Davis, E., & Aaronson, S. (2022) · arXiv · doi:10.48550/arxiv.2204.13807 · Cited by 82
Machine Learning and Hebrew NLP for Automated Assessment of Open-Ended Questions in Biology
Ariely, M., Nazaretsky, T., & Alexandron, G. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00283-x · Cited by 76
Agent AI: Surveying the Horizons of Multimodal Interaction
Durante, Z., Huang, Q., Wake, N., Gong, R., Park, J. S., Sarkar, B., Taori, R., Noda, Y., Terzopoulos, D., Choi, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.03568 · Cited by 75
Making Language Models Better Reasoners with Step-Aware Verifier
Li, Y., Lin, Z., Zhang, S., Fu, Q., Chen, B., Lou, J.-G., & Chen, W. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.291 · Cited by 69
Self-citationASETF: A Novel Method for Jailbreak Attack on LLMs through Translate Suffix Embeddings
Wang, H., Li, H., Huang, M., & Sha, L. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.157 · Cited by 68
Prompt Engineering for Large Language Models
Gao, A. (2023) · doi:10.2139/ssrn.4504303 · Cited by 68
Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., & Cobbe, K. (2023) · arXiv · doi:10.48550/arxiv.2305.20050 · Cited by 65
A Comprehensive Overview of Backdoor Attacks in Large Language Models Within Communication Networks
Yang, H., Xiang, K., Ge, M., Li, H., Lu, R., & Yu, S. (2024) · IEEE Network · doi:10.1109/mnet.2024.3367788 · Cited by 65
Internet-augmented language models through few-shot prompting for open-domain question answering
Lazaridou, A., Gribovskaya, E., Stokowiec, W., & Grigorev, N. (2022) · arXiv · doi:10.48550/arxiv.2203.05115 · Cited by 64
Evaluating Human-Language Model Interaction
Lee, M., Srivastava, M., Hardy, A., Thickstun, J., Durmus, E., Paranjape, A., Gerard-Ursin, I., Li, X. L., Ladhak, F., Rong, F., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.09746 · Cited by 61
2023 ACM Conference on Fairness Accountability and Transparency
(2023) · doi:10.1145/3593013 · Cited by 61
Adapting Language Models to Compress Contexts
Chevalier, A., Wettig, A., Ajith, A., & Chen, D. (2023) · arXiv · Cited by 59
Identifying and Extracting Rare Diseases and Their Phenotypes with Large Language Models
Shyr, C., Hu, Y., Bastarache, L., Cheng, A., Hamid, R., Harris, P., & Xu, H. (2024) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-023-00155-0 · Cited by 58
ExpertPrompting: Instructing Large Language Models to be Distinguished Experts
Xu, B., An, Y., Lin, J., Wang, Q., Zhou, C., Zhang, Y., & Mao, Z. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.14688 · Cited by 55
Schulhoff, S., Pinto, J., Khan, A., Bouchard, L.-F., Si, C., Anati, S., Tagliabue, V., Kost, A., Carnahan, C., & Boyd-Graber, J. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.302 · Cited by 54
PoisonBench: Assessing Large Language Model Vulnerability to Data PoisoningPreprint
Fu, T., Sharma, M., Torr, P. H. S., Cohen, S. B., Krueger, D., & Barez, F. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.08811 · Cited by 53
Wu, S., Shen, E. M., Badrinath, C., Ma, J., & Lakkaraju, H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.13339 · Cited by 52
QuALITY: Question Answering with Long Input Texts, Yes!
Pang, R. Y., Parrish, A., Joshi, N., Nangia, N., Phang, J., Chen, A., Padmakumar, V., Ma, J., Thompson, J., He, H., et al. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.391 · Cited by 51
EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers
Guo, Q., Wang, R., Guo, J., Li, B., Song, K., Tan, X., Liu, G., Bian, J., & Yang, Y. (2023) · arXiv · doi:10.48550/arxiv.2309.08532 · Cited by 50
Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., & Fritz, M. (2023) · arXiv · doi:10.48550/arxiv.2302.12173 · Cited by 50
Large Language Models Perform Diagnostic ReasoningPreprint
Wu, C.-K., Chen, W., & Chen, H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.08922 · Cited by 49
Towards Trustworthy AutoGrading of Short, Multi-lingual, Multi-type Answers
Schneider, J., Richner, R., & Riser, M. (2023) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-022-00289-z · Cited by 48
Dehghani, M., Tay, Y., Gritsenko, A. A., Zhao, Z., Houlsby, N., Diaz, F., Metzler, D., & Vinyals, O. (2021) · arXiv · doi:10.48550/arxiv.2107.07002 · Cited by 48
SummScreen: A Dataset for Abstractive Screenplay Summarization
Chen, M., Chu, Z., Wiseman, S., & Gimpel, K. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.589 · Cited by 46
Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought
Saparov, A., & He, H. (2022) · arXiv · doi:10.48550/arxiv.2210.01240 · Cited by 46
Zhao, S., Jia, M., Guo, Z., Gan, L., Xu, X., Wu, X., Fu, J., Feng, Y., Pan, F., & Tuan, L. A. (2024) · doi:10.36227/techrxiv.172832726.62863760/v1 · Cited by 42
SoK: Prompt Hacking of Large Language Models
Rababah, B., Wu, S. T., Kwiatkowski, M. R., Leung, C. K.-S., & Akçora, C. G. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825103 · Cited by 42
Bring Your Own Data! Self-Supervised Evaluation for Large Language ModelsPreprint
Jain, N., Saifullah, K., Wen, Y., Kirchenbauer, J., Shu, M., Saha, A., Goldblum, M., Geiping, J., & Goldstein, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.13651 · Cited by 41
Smith, E., Hsu, O., Qian, R., Roller, S., Boureau, Y.-L., & Weston, J. (2022) · Proceedings of the 4th Workshop on NLP for Conversational AI · doi:10.18653/v1/2022.nlp4convai-1.8 · Cited by 40
2019 IEEE International Congress on Internet of Things (ICIOT)
(2019) · doi:10.1109/iciot48240.2019 · Cited by 39
Risk or Chance? Large Language Models and Reproducibility in HCI Research
Kosch, T., & Feger, S. S. (2024) · interactions · doi:10.1145/3695765 · Cited by 39
Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation
Ning, X., Lin, Z., Zhou, Z., Wang, Z., Yang, H., & Wang, Y. (2023) · arXiv · doi:10.48550/arxiv.2307.15337 · Cited by 36
Research in Attacks, Intrusions, and Defenses
Antonakakis, M., Polychronakis, M., Bailey, M., & Daciér, M. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66332-6 · Cited by 36
Exploring Prompt Engineering Practices in the EnterprisePreprint
Desmond, M. J., & Brachman, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.08950 · Cited by 36
Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
Burns, C., Izmailov, P., Kirchner, J. H., Baker, B., Gao, L., Aschenbrenner, L., Chen, Y., Ecoffet, A., Joglekar, M., Leike, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.09390 · Cited by 34
PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization
Wang, Y., Yu, Z., Zeng, Z., Yang, L., Wang, C., Chen, H., Jiang, C., Xie, R., Wang, J., Xie, X., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05087 · Cited by 34
Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks
Shayegani, E., Mamun, M. A. A., Fu, Y., Zaree, P., Dong, Y., & Abu‐Ghazaleh, N. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.10844 · Cited by 34
LLM Multi-Agent Systems: Challenges and Open Problems
Han, S., Zhang, Q., Jin, W., & Xu, Z. (2024) · arXiv · doi:10.48550/arxiv.2402.03578 · Cited by 33
Exploring Human-Like Reading Strategy for Abstractive Text Summarization
Yang, M., Qu, Q., Tu, W., Shen, Y., Zhao, Z., & Chen, X. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33017362 · Cited by 33
Exploring Vulnerabilities and Protections in Large Language Models: A SurveyPreprint
Liu, F. W., & Hu, C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.00240 · Cited by 33
Repository-Level Prompt Generation for Large Language Models of Code
Shrivastava, D., Larochelle, H., & Tarlow, D. (2022) · arXiv · doi:10.48550/arxiv.2206.12839 · Cited by 27
Solving math word problems with process- and outcome-based feedback
Uesato, J., Kushman, N., Kumar, R., Song, F., Siegel, N., Wang, L., Creswell, A., Irving, G., & Higgins, I. (2022) · arXiv · doi:10.48550/arxiv.2211.14275 · Cited by 25
Li, D., Huang, S., Wei, F., et al. (2024) · doi:10.18653/v1/2024.eacl-long · Cited by 24
Addressing Adversarial Machine Learning Attacks in Smart Healthcare Perspectives
Selvakkumar, A., Pal, S., & Jadidi, Z. (2022) · Lecture notes in electrical engineering · doi:10.1007/978-3-030-98886-9_21 · Cited by 24
AQuA: A Benchmarking Tool for Label Quality Assessment
Goswami, M., Sanil, V., Choudhry, A., Srinivasan, A., Udompanyawit, C., & Dubrawski, A. W. (2023) · neural information processing systems · doi:10.52202/075280-3494 · Cited by 24
Investigating Prompting Techniques for Zero- and Few-Shot Visual Question AnsweringPreprint
Awal, R., Zhang, L., & Agrawal, A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.09996 · Cited by 23
Post Hoc Explanations of Language Models Can Improve Language ModelsPreprint
Krishna, S., Ma, J., Slack, D., Ghandeharioun, A., Singh, S., & Lakkaraju, H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.11426 · Cited by 21
Scaling Transformer to 1M tokens and beyond with RMT
Bulatov, A., Kuratov, Y., Kapushev, Y., & Burtsev, M. S. (2023) · arXiv · doi:10.48550/arxiv.2304.11062 · Cited by 19
Active Retrieval Augmented Generation
Jiang, Z., Xu, F. F., Gao, L., Sun, Z., Liu, Q., Dwivedi-Yu, J., Yang, Y., Callan, J., & Neubig, G. (2023) · arXiv · doi:10.48550/arxiv.2305.06983 · Cited by 18
Öztürk, D. (2021) · Accounting, finance, sustainability, governance & fraud · doi:10.1007/978-981-33-6811-8_14 · Cited by 16
Chain-of-Thought Reasoning is a Policy Improvement OperatorPreprint
Zhang, H., & Parkes, D. C. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.08589 · Cited by 15
Less Likely Brainstorming: Using Language Models to Generate Alternative Hypotheses
Tang, L., Peng, Y., Wang, Y., Ding, Y., Durrett, G., & Rousseau, J. F. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.794 · Cited by 14
RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning
Deng, M., Wang, J., Hsieh, C.-P., Wang, Y., Guo, H., Shu, T., Song, M., Xing, E. P., & Hu, Z. (2022) · arXiv · doi:10.48550/arxiv.2205.12548 · Cited by 14
Proceedings of the 4th New Frontiers in Summarization Workshop
(2023) · doi:10.18653/v1/2023.newsum-1 · Cited by 12
Beyond Attention: Breaking the Limits of Transformer Context Length with Recurrent Memory
Bulatov, A., Kuratov, Y., Kapushev, Y., & Burtsev, M. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29722 · Cited by 12
Proceedings of the Workshop on Stylistic Variation
(2017) · doi:10.18653/v1/w17-49 · Cited by 10
QuALITY: Question Answering with Long Input Texts, Yes!
Pang, R. Y., Parrish, A., Joshi, N., Nangia, N., Phang, J., Chen, A., Padmakumar, V., Ma, J., Thompson, J., He, H., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.08608 · Cited by 10
Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading
Chen, H., Pasunuru, R., Weston, J., & Celikyilmaz, A. (2023) · arXiv · doi:10.48550/arxiv.2310.05029 · Cited by 9
If Our Aim Is to Build Morality Into an Artificial Agent, How Might We Begin to Go About Doing So?
Seeamber, R., & Badea, C. (2023) · IEEE Intelligent Systems · doi:10.1109/mis.2023.3320875 · Cited by 9
AQuA: A Benchmarking Tool for Label Quality Assessment
Goswami, M., Sanil, V., Choudhry, A., Srinivasan, A., Udompanyawit, C., & Dubrawski, A. (2023) · arXiv · doi:10.48550/arxiv.2306.09467 · Cited by 8
Skeleton-of-thought: Prompting LLMs for efficient parallel generation
Ning, X., Lin, Z., Zhou, Z., Wang, Z., Yang, H., & Wang, Y. (2024) · Lirias · Cited by 7
Enhancing Large Language Models Against Inductive Instructions with Dual-critique Prompting
Wang, R., Wang, H., Mi, F., Xue, B., Chen, Y., Wong, K.-F., & Xu, R. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.299 · Cited by 4
Schulhoff, S., Pinto, J., Khan, A., Bouchard, L.-F., Si, C., Anati, S., Tagliabue, V., Kost, A. L., Carnahan, C., & Boyd-Graber, J. (2023) · arXiv · doi:10.48550/arxiv.2311.16119 · Cited by 4
TreePrompt: Learning to Compose Tree Prompts for Explainable Visual GroundingPreprint
Zhang, C., Xiao, J., Chen, L., Shao, J., & Chen, L. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.11497 · Cited by 3
Enhancing Large Language Models Against Inductive Instructions with Dual-critique Prompting
Wang, R., Wang, H., Mi, F., Chen, Y., Xue, B., Wong, K.-F., & Xu, R. (2023) · arXiv · doi:10.48550/arxiv.2305.13733 · Cited by 2
Hierarchical prompting assists large language model on web navigation
Sridhar, A., Lo, R., Xu, F. F., Zhu, H., & Zhou, S. (2023) · arXiv · doi:10.48550/arxiv.2305.14257 · Cited by 2
Teaching Algorithmic Reasoning via In-context Learning
Zhou, H., Nova, A., Larochelle, H., Courville, A., Neyshabur, B., & Sedghi, H. (2022) · arXiv · doi:10.48550/arxiv.2211.09066
Large Language Model Guided Tree-of-Thought
Long, J. (2023) · arXiv · doi:10.48550/arxiv.2305.08291
Teaching Arithmetic to Small Transformers
Lee, N., Sreenivasan, K., Lee, J. D., Lee, K., & Papailiopoulos, D. (2023) · arXiv · doi:10.48550/arxiv.2307.03381
Research works printed but not linked: 13
- Achiam, 2024, GPT-4 technical report, arXiv
- Dubey, 2024, The Llama 3 herd of models, arXiv
- Sarkhel, 2023, Self-training for label-efficient information extraction from semi-structured web-pages, Proceedings VLDB Endowment, 16, 3098, 10.14778/3611479.3611511 · DOI
- Arias, 2024, Decoding decoded: understanding hyperparameter effects in open-ended text generation, arXiv
- Van Buren, 2023, Guided scenarios with simulated expert personae: a remarkable strategy to perform cognitive work, arXiv
- Lee, K., Firat, O., Agarwal, A., Fannjiang, C., and Sussillo, D. Hallucinations in neural machine translation (2018). https://openreview.net/forum?id=SkxJ-309FQ
- Gao, 2024, Retrieval-sugmented generation for large language models: a survey, arXiv
- Paranjape, 2023, automatic multi-step reasoning and tool-use for large language models, arXiv
- Hu, H., Lu, H., Zhang, H., Song, Y.-Z., Lam, W., and Zhang, Y. (2024). Chain-of-symbol prompting for spatial relationships in large language models. https://openreview.net/forum?id=B0wJ5oCPdB
- Steinhardt, 2017, Certified defenses for data poisoning attacks, Adv. Neural Inf. Process. Syst., 30
- Wang, 2024, From noise to clarity: unraveling the adversarial suffix of large language model attacks via translation of text embeddings, arXiv
- Sha, 2024, Prompt stealing attacks against large language models, arXiv
- Bertucci, 2022, Deep learning in finance: from implementation to regulation, arXiv
Cited by
Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents
Zhang, J., Hu, S., Lu, C., Lange, R., & Clune, J. (2025) · arXiv · doi:10.48550/arxiv.2505.22954 · Cited by 344
Generative AI Literacy: Twelve Defining Competencies
Annapureddy, R., Fornaroli, A., & Gatica-Perez, D. (2025) · Digital Government: Research and Practice · doi:10.1145/3685680 · Cited by 333
Gödel Agent: A Self-Referential Agent Framework for Recursively Self-Improvement
Yin, X., Wang, X., Pan, L., Lin, L., Wan, X., & Wang, W. Y. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.1354 · Cited by 105
Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL
Hong, Z., Yuan, Z., Zhang, Q., Chen, H., Dong, J., Huang, F., & Huang, X. (2024) · arXiv · doi:10.48550/arxiv.2406.08426 · Cited by 100
Large language models for life cycle assessments: Opportunities, challenges, and risks
Preuss, N., Alshehri, A. S., & You, F. (2024) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2024.142824 · Cited by 71
Metadata
- Authors
- Chen, B., Zhang, Z., Langrené, N., & Zhu, S.
- Venue
- Patterns
- Publisher
- Elsevier BV
- Published
- 2025-06
- Volume
- 6
- Issue
- 6
- Pages
- 101260
- ISSN
- 2666-3899 (ISSN-L)
- OpenAlex
- W4410202262
- Cited by
- 490
- References
- 295
- FWCI
- 186.42
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- creativecommons.org/licenses/by/4.0/ (vor)
- Funders
- National Natural Science Foundation of China (12271047)
- Hong Kong Baptist University
- Beijing Normal University
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.