Phishing and Social Engineering in the Age of LLMs
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
The human factor remains a major vulnerability in cybersecurity. This chapter explores the escalating threats that Large Language Models (LLMs) pose in the field of cybercrime, particularly in phishing and social engineering. Due to their ability to generate highly convincing and individualized content, LLMs enhance the effectiveness and scale of phishing attacks, making them increasingly difficult to detect. The integration of multimodal generative models allows malicious actors to leverage AI-generated text, images, and audio, increasing attack avenues and making attacks more convincing. Two case studies provide a comprehensive look, examining how AI technology orchestrates a phishing attack posing as a typical e-commerce transaction and how an LLM was used in a romance-themed cryptocurrency scam. Both scenarios underline the need for increased awareness and improved defenses against these novel and sophisticated cyber threats.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Red Teaming Language Models with Language Models
Perez, E., Huang, S., Song, F., Cai, T., Ring, R., Aslanides, J., Glaese, A., McAleese, N., & Irving, G. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.225 · Cited by 363
Human Factors in Phishing Attacks: A Systematic Literature Review
Desolda, G., Ferro, L. S., Marrella, A., Catarci, T., & Costabile, M. F. (2022) · ACM Computing Surveys · doi:10.1145/3469886 · Cited by 148
Spear Phishing With Large Language Models
Hazell, J. (2023) · arXiv · doi:10.48550/arxiv.2305.06972 · Cited by 28
Research works printed but not linked: 4
- Jaron Mink et al. {DeepPhish}: Understanding user trust towards artificially generated profiles in online social networks. In 31st USENIX Security Symposium (USENIX Security 22), pages 1669–1686, 2022.
- Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043, 2023. · arXiv
- Chengyi Wang et al. Neural codec language models are zero-shot text to speech synthesizers. arXiv preprint arXiv:2301.02111, 2023. · arXiv
- Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499, 2016. · arXiv
Cited by
International AI Safety Report 2026Preprint
Bengio, Y., Clare, S., Prunkl, C., Andriushchenko, M., Bucknall, B., Murray, M., Bommasani, R., Casper, S., Davidson, T., Douglas, R., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.21012 · Cited by 177
A State-of-the-Art Review on Phishing Website Detection Techniques
Li, W., Manickam, S., Chong, Y.-W., Leng, W., & Nanda, P. (2024) · IEEE Access · doi:10.1109/access.2024.3514972 · Cited by 88
People cannot distinguish GPT-4 from a human in a Turing test
Jones, C. R., Rathi, I., Taylor, S., & Bergen, B. K. (2025) · Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3715275.3732108 · Cited by 76
People cannot distinguish GPT-4 from a human in a Turing test
Jones, C. R., & Bergen, B. K. (2024) · arXiv · doi:10.48550/arxiv.2405.08007 · Cited by 76
Jones, C., & Bergen, B. K. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11517-6 · Cited by 63
Metadata
- Authors
- Gallagher, S., Gelman, B., Taoufiq, S., Vörös, T., Lee, Y., Kyadige, A., & Bergeron, S.
- Corresponding author
- Gallagher, Sean
- Venue
- Large Language Models in Cybersecurity
- Publisher
- Springer Nature Switzerland
- Published
- 2024
- Pages
- 81-86
- OpenAlex
- W4399268578
- Cited by
- 20
- References
- 2
- Citations by year
- 2026: 5 · 2025: 11 · 2024: 4
- FWCI
- 14.56
- Type
- book chapter
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor, 102 d)
- Locations
- Springer eBooks (publishedVersion, cc-by)
- Funders
- armasuisse
- Topics
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Social Sciences › Law › Cybercrime and Law Enforcement Studies
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- phishing, social engineering, large language models, AI-generated content, multimodal generative models, cybercrime, cryptocurrency scams, romance scams, e-commerce fraud, human factors in cybersecurity, phishing detection, cybersecurity awareness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.