ChatPhishDetector: Detecting Phishing Sites Using Large Language Models
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Large Language Models (LLMs), such as ChatGPT, are significantly impacting various fields. While LLMs have been extensively studied for code generation and text synthesis, their application in detecting malicious web content, particularly phishing sites, remains largely unexplored. To counter the increasing cyber-attacks that leverage LLMs for creating more sophisticated and convincing phishing content, it is crucial to automate detection by harnessing LLMs’ advanced capabilities. This paper introduces ChatPhishDetector, a novel system that employs LLMs to identify phishing sites. Our approach involves using a web crawler to collect website information, generating prompts for LLMs based on the gathered data, and extracting detection results from LLM responses. This system enables accurate detection of multilingual phishing sites by identifying impersonated brands and social engineering techniques within the entire website context, without requiring machine learning model training. We evaluated our system’s performance using our own dataset and compared it with baseline systems and several LLMs. Experiments using GPT-4V showed exceptional results, achieving 98.7% precision and 99.6% recall, surpassing the detection performance of other LLMs and existing systems. These findings highlight the potential of LLMs for protecting users from online fraudulent activities and provide crucial insights for strengthening defenses against phishing attacks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Training Language Models to Follow Instructions with Human Feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2011 · Cited by 3,117
Large language models are zero-shot reasoners
Kojima, T., Gu, S. (., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1613 · Cited by 2,013
Large Language Models are Zero-Shot Reasoners
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · arXiv · doi:10.48550/arxiv.2205.11916 · Cited by 2,013
Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., & Fritz, M. (2023) · Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security · doi:10.1145/3605764.3623985 · Cited by 667 · cited 1×
“Methods that use LLMs, including our system, may be susceptible to prompt injection attacks [58], [59], where the original prompt is overwritten and malicious content is inserted into the response.”
MethodPhishStorm: Detecting Phishing With Streaming Analytics
Marchal, S., Francois, J., State, R., & Engel, T. (2014) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2014.2377295 · Cited by 271 · cited 2×
“We chose dnstwist [33], Phishpedia [15], and Phish-Storm [49] as baseline systems because they have been widely used in previous studies [16], [34], [52], [53], [54] for phishing detection.”
MethodLarge Language Model guided Protocol Fuzzing
Meng, R., Mirchev, M., Böhme, M., & Roychoudhury, A. (2024) · Proceedings 2024 Network and Distributed System Security Symposium · doi:10.14722/ndss.2024.24556 · Cited by 246 · cited 1×
“Meng et al. [38] proposed CHATAFL, an LLM-guided protocol fuzzing engine that outperformed state-of-the-art fuzzers in terms of state and code coverage and discovered nine previously unknown vulnerabilities in widely-used protocol implementations.”
Background(2017) · doi:10.1145/3133956.3134067 · Cited by 187 · cited 1×
“These websites frequently impersonate legitimate platforms such as online banking services [17], e-commerce sites [18], and social media platforms [19].”
BackgroundSeven Months' Worth of Mistakes: A Longitudinal Study of Typosquatting Abuse
Agten, P., Joosen, W., Piessens, F., & Nikiforakis, N. (2015) · Proceedings 2015 Network and Distributed System Security Symposium · doi:10.14722/ndss.2015.23058 · Cited by 140
Cited by
Large Language Models for Cyber Security: A Systematic Literature Review
Xu, H., Wang, S., Li, N., Wang, K., Zhao, Y., Chen, K., Yu, T., Liu, Y., & Wang, H. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3769676 · Cited by 189
Jelodar, H., Bai, S., Hamedi, P., Mohammadian, H., Razavi-Far, R., & Ghorbani, A. (2026) · Journal of Information Security and Applications · doi:10.1016/j.jisa.2026.104390 · Cited by 27
A Multi-Agent System for Cybersecurity Threat Detection and Correlation Using Large Language Models
Hmimou, Y., Tabaa, M., Khiat, A., & Hidila, Z. (2025) · IEEE Access · doi:10.1109/access.2025.3602681 · Cited by 25
Large Language Models for Security Operations Centers: A Comprehensive Survey
Habibzadeh, A., Feyzi, F., & Atani, R. E. (2026) · Journal of Electrical and Computer Engineering · doi:10.1155/jece/3383674 · Cited by 13
“Models adopting this approach seek to overcome the limitations of traditional methods, which often depend on static, manually curated knowledge bases or are constrained to a single input modality, such as images or text [246, 235, 247, 248].”
Metadata
- Authors
- Koide, T., Nakano, H., & Chiba, D.
- Venue
- IEEE Access
- Published
- 2024
- Volume
- 12
- Pages
- 154381-154400
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W4403600666
- Cited by
- 48
- References
- 13
- FWCI
- 18.37
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: ChatPhishDetector: Detecting Phishing Sites Using Large Language Models. https://app.cochat.ai/papers/10.1109/access.2024.3483905
- The paper
- Koide, T., Nakano, H., & Chiba, D. (2024). ChatPhishDetector: Detecting Phishing Sites Using Large Language Models. IEEE Access. https://doi.org/10.1109/access.2024.3483905
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.