PhishStorm: Detecting Phishing With Streaming Analytics
Open accessThe DOI matches its Crossref record; 49 of the 50 references it cites were found in a registry or an index.
Abstract
Despite the growth of prevention techniques, phishing remains an important threat since the principal countermeasures in use are still based on reactive URL blacklisting. This technique is inefficient due to the short lifetime of phishing Web sites, making recent approaches relying on real-time or proactive phishing URL detection techniques more appropriate. In this paper, we introduce PhishStorm, an automated phishing detection system that can analyze in real time any URL in order to identify potential phishing sites. PhishStorm can interface with any email server or HTTP proxy. We argue that phishing URLs usually have few relationships between the part of the URL that must be registered (low-level domain) and the remaining part of the URL (upper-level domain, path, query). We show in this paper that experimental evidence supports this observation and can be used to detect phishing sites. For this purpose, we define the new concept of intra-URL relatedness and evaluate it using features extracted from words that compose a URL based on query data from Google and Yahoo search engines. These features are then used in machine-learning-based classification to detect phishing URLs from a real dataset. Our technique is assessed on 96 018 phishing and legitimate URLs that result in a correct classification rate of 94.91% with only 1.44% false positives. An extension for a URL phishingness rating system exhibiting high confidence rate ( $>$ 99%) is proposed. We discuss in this paper efficient implementation patterns that allow real-time analytics using Big Data architectures such as STORM and advanced data structures based on the Bloom filter.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · cited 10×
“The detailed classification metrics for the Random Forest algorithm with a 0.76 discrimination threshold are given in Table IX.”
MethodInfluentialThe WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750 · cited 2×
“Since there is a wide range of supervised classification algorithms, we assessed our dataset according to several classifiers using Weka [27].”
MethodWordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015 · cited 1×
“…obfuscation techniques [15], with examples given in Table I for the domain paypal.com: • Type I: URL obfuscation with other domain: The mld.ps is a real domain name, usually registered by the phisher, while the original domain being phished is part of the path, the query or the upper level domain.”
BackgroundSpace/time trade-offs in hash coding with allowable errors
Bloom, B. H. (1970) · Communications of the ACM · doi:10.1145/362686.362692 · Cited by 7,492 · cited 1×
“As a result we implement all the word set previously defined with an efficient data structure: the Bloom filter [20].”
MethodLandauer, T. K., & Dumais, S. T. (1997) · Psychological Review · doi:10.1037/0033-295x.104.2.211 · Cited by 6,282 · cited 2×
“[18] T. K. Landauer and S. T. Dumais, “A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge,” Psychological Rev., vol. 104, no. 2, pp. 211–240, Apr. 1997.”
MethodDetecting influenza epidemics using search engine query data
Ginsberg, J., Mohebbi, M. H., Patel, R. S., Brammer, L., Smolinski, M. S., & Brilliant, L. (2009) · Nature · doi:10.1038/nature07634 · Cited by 4,295 · cited 1×
“[47] estimate the magnitude of flu infection for a given geographic region.”
Word association norms, mutual information, and lexicography
Church, K., & Hanks, P. (1990) · doi:10.5555/89086.89095 · Cited by 3,590
The Google Similarity Distance
Cilibrasi, R. L., & Vitanyi, P. M. B. (2007) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2007.48 · Cited by 1,760
Domain names - implementation and specification
Mockapetris, P. V. (1987) · doi:10.17487/rfc1035 · Cited by 1,698
Mining the Web for Synonyms: PMI-IR versus LSA on TOEFL
Turney, P. D. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44795-4_42 · Cited by 1,285 · cited 2×
“[19] P. Turney, “Mining the web for synonyms: PMI-IR versus LSA on TOEFL,” in Proc.”
MethodWord association norms, mutual information, and lexicography
Church, K. W., & Hanks, P. (1989) · Proceedings of the 27th annual meeting on Association for Computational Linguistics - · doi:10.3115/981623.981633 · Cited by 1,094 · cited 1×
“…obfuscation techniques [15], with examples given in Table I for the domain paypal.com: • Type I: URL obfuscation with other domain: The mld.ps is a real domain name, usually registered by the phisher, while the original domain being phished is part of the path, the query or the upper level domain.”
BackgroundBeyond blacklists: learning to detect malicious web sites from suspicious URLs
Ma, J., Saul, L. K., Savage, S., & Voelker, G. M. (2009) · Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1557019.1557153 · Cited by 803
Cantina: a content-based approach to detecting phishing web sites
Zhang, Y., Hong, J. I., & Cranor, L. F. (2007) · Proceedings of the 16th international conference on World Wide Web · doi:10.1145/1242572.1242659 · Cited by 757
EXPOSURE : Finding malicious domains using passive DNS analysis
Bilge, L. (2011) · Graduate School and Research Center in Digital Science (EURECOM) · Cited by 661
Learning to detect phishing emails
Fette, I., Sadeh, N., & Tomasic, A. (2007) · Proceedings of the 16th international conference on World Wide Web · doi:10.1145/1242572.1242660 · Cited by 604
Identifying suspicious URLs: an application of large-scale online learning
Ma, J., Saul, L. K., Savage, S., & Voelker, G. M. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553462 · Cited by 498 · cited 2×
“A lexical analysis is performed to create binary features from each label observed in the URL in [15], [23], [24], [37].”
BackgroundMethodA framework for detection and measurement of phishing attacks
Garera, S., Provos, N., Chew, M., & Rubin, A. D. (2007) · Proceedings of the 2007 ACM workshop on Recurring malcode · doi:10.1145/1314389.1314391 · Cited by 489 · cited 1×
“In the following we present the most used URL obfuscation techniques [15], with examples given in Table I for the domain paypal.com:”
PhishNet: Predictive Blacklisting to Detect Phishing Attacks
Prakash, P., Kumar, M., Kompella, R. R., & Gupta, M. (2010) · 2010 Proceedings IEEE INFOCOM · doi:10.1109/infcom.2010.5462216 · Cited by 416 · cited 2×
“These sets are composed from different sources—as in several phishing detection work [15], [22], [23]—already used in [24] and [25].”
BackgroundBuilding a dynamic reputation system for DNS
Antonakakis, M., Perdisci, R., Dagon, D., Lee, W., & Feamster, N. (2010) · Cited by 396
Apples-to-apples in cross-validation studies: pitfalls in classifier performance measurement
Forman, G., & Scholz, M. (2010) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1882471.1882479 · Cited by 368 · cited 1×
“And as presented in [26], imbalanced dataset in cross-validation provides misleading results.”
BackgroundDetecting algorithmically generated malicious domain names
Yadav, S., Reddy, A. K. K., Reddy, A. L. N., & Ranjan, S. (2010) · Proceedings of the 10th ACM SIGCOMM conference on Internet measurement · doi:10.1145/1879141.1879148 · Cited by 355
Development of the Domain Name System
Mockapetris, P. V., & Dunlap, K. J. (1995) · ACM SIGCOMM Computer Communication Review · doi:10.1145/205447.205459 · Cited by 350
Exposure: A Passive DNS Analysis Service to Detect and Report Malicious Domains
Bilge, L., Sen, S., Balzarotti, D., Kirda, E., & Kruegel, C. (2014) · ACM Transactions on Information and System Security · doi:10.1145/2584679 · Cited by 317 · cited 1×
“Passively captured DNS traffic is used to recognise malicious domains in [35], [36].”
BackgroundSurvey on web spam detection: principles and algorithms
Spirin, N., & Han, J. (2012) · ACM SIGKDD Explorations Newsletter · doi:10.1145/2207243.2207252 · Cited by 252 · cited 1×
“A comprehensive overview on different data mining approaches for spamming email detection is given in [45].”
BackgroundDetecting Algorithmically Generated Domain-Flux Attacks With DNS Traffic Analysis
Yadav, S., Reddy, A. K. K., Reddy, A. L. N., & Ranjan, S. (2012) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2012.2184552 · Cited by 222 · cited 1×
“This kind of URLs and malicious domains are however widely used in botnet communication (C&C) or spamming activities [29], such activities does not rely on a social engineering process as phishing does.”
BackgroundPhishDef: URL names say it all
Le, A., Markopoulou, A., & Faloutsos, M. (2011) · 2011 Proceedings IEEE INFOCOM · doi:10.1109/infcom.2011.5934995 · Cited by 200 · cited 3×
“A lexical analysis is performed to create binary features from each label observed in the URL in [15], [23], [24], [37].”
BackgroundMethodImproved Phishing Detection using Model-Based Features.
Bergholz, A., Chang, J. H., Paaß, G., Reichartz, F., & Strobel, S. (2008) · Conference on Email and Anti-Spam · Cited by 181
Visual-similarity-based phishing detection
Medvet, E., Kirda, E., & Kruegel, C. (2008) · Proceedings of the 4th international conference on Security and privacy in communication netowrks · doi:10.1145/1460877.1460905 · Cited by 170 · cited 1×
“One approach is to compare the content of presumed phishing Web pages with the original Web page being phished as in [30], [31], [32], [33], [34].”
BackgroundBeautiful Data: The Stories Behind Elegant Data Solutions
Segaran, T., & Hammerbacher, J. (2009) · Cited by 170
Development of the domain name system
Mockapetris, P., & Dunlap, K. J. (1988) · Symposium proceedings on Communications architectures and protocols · doi:10.1145/52324.52338 · Cited by 161 · cited 1×
“The examples of obfuscated phishing URL from Type I to IV highlight a global characteristic in URL obfuscation, namely that there is no relation between the mld.ps and the rest of the URL.”
BackgroundDISCO: A Multilingual Database of Distributionally Similar Words
Kolb, P. (2008) · Cited by 143
Detecting visually similar Web pages: Application to phishing detection
Chen, T.-C., Dick, S., & Miller, J. T. (2010) · ACM Transactions on Internet Technology · doi:10.1145/1754393.1754394 · Cited by 124 · cited 1×
“One approach is to compare the content of presumed phishing Web pages with the original Web page being phished as in [30]–[34].”
BackgroundA hybrid phish detection approach by identity discovery and keywords retrieval
Xiang, G., & Hong, J. (2009) · International World Wide Web Conference (WWW) · doi:10.1145/1526709.1526786 · Cited by 117
Highly predictive blacklisting
Zhang, J., Porras, P. A., & Ullrich, J. (2008) · Cited by 106
Detecting Phishing Emails Using Hybrid Features
Ma, L., Ofoghi, B., Watters, P., & Brown, S. (2009) · Symposia and Workshops on Ubiquitous, Autonomic and Trusted Computing (UIC-ATC) · doi:10.1109/uic-atc.2009.103 · Cited by 94
Ramanathan, V., & Wechsler, H. (2012) · EURASIP Journal on Information Security · doi:10.1186/1687-417x-2012-1 · Cited by 84 · cited 1×
“Approximately the same technique is used in [41] to block phishing emails based on semantic content analysis.”
MethodPHONEY: Mimicking User Response to Detect Phishing Attacks
Chandrasekaran, M., Chinchani, R., & Upadhyaya, S. (2006) · IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM) · doi:10.1109/wowmom.2006.87 · Cited by 72
Discovering trends in software engineering with google trend
Rech, J. (2007) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/1234741.1234765 · Cited by 61 · cited 1×
“Search engine query data has already been used for the analysis of search interests over time [46].”
MethodProactive Discovery of Phishing Related Domain Names
Marchal, S., François, J., State, R., & Engel, T. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33338-5_10 · Cited by 49 · cited 4×
“However this tool, even if efficient in most cases and especially for phishing domain names analysis as shown in our previous work [17], is not necessarily suited to intra-URL relatedness computation.”
BackgroundMethodInfluentialSelf-citationAn Anti-Phishing System Employing Diffused Information
Chen, T.-C., Stepan, T., Dick, S., & Miller, J. T. (2014) · ACM Transactions on Information and System Security · doi:10.1145/2584680 · Cited by 44 · cited 1×
“One approach is to compare the content of presumed phishing Web pages with the original Web page being phished as in [30]–[34].”
BackgroundImproving distributed join efficiency with extended bloom filter operations
Michael, L., Nejdl, W., Papapetrou, O., & Siberski, W. (2007) · International Conference on Advanced Information Networking and Applications (AINA) · doi:10.1109/aina.2007.80 · Cited by 40 · cited 1×
“To perform set intersection and union with Bloom filter the requirement is that both Bloom filters have same size (m) and share the same hash functions [21].”
BackgroundPhishScore: Hacking phishers' minds
Marchal, S., François, J., State, R., & Engel, T. (2014) · International Conference on Network and Service Management (CNSM) · doi:10.1109/cnsm.2014.7014140 · Cited by 39 · cited 2×
“• Type IV: URL obfuscation with IP address: the URL’s hostname is replaced by an IP address and the brand being phished is part of the path or the query.”
BackgroundSelf-citationNero, P. J., Wardman, B., Copes, H., & Warner, G. (2011) · eCrime Researchers Summit (eCrime) · doi:10.1109/ecrime.2011.6151979 · Cited by 20 · cited 1×
“Assuming a phisher wants to trap PayPal users, he must use a domain.tld other than paypal.com, as this domain is already registered by PayPal Inc.”
MethodDetecting and Tracking Topics and Events from Web Search Logs
Liu, H., He, J., Gu, Y., Xiong, H., & Du, X. (2012) · ACM Transactions on Information Systems · doi:10.1145/2382438.2382440 · Cited by 17
Lexical URL analysis for discriminating phishing and legitimate e-mail messages
Khonji, M., Iraqi, Y., & Jones, A. R. (2011) · Australasian Journal of Paramedicine · Cited by 15
PhishSafe: leveraging modern JavaScript API's for transparent and robust protection
Braun, B., Johns, M., Köstler, J., & Posegga, J. (2014) · ACM Conference on Data and Application Security and Privacy (CODASPY) · doi:10.1145/2557547.2557553 · Cited by 7
Intelligent phishing detection parameter framework for E-banking transactions based on Neuro-fuzzy
Barraclough, P., Hossain, M. A., Sexton, G., & Aslam, N. (2014) · Science and Information Conference · doi:10.1109/sai.2014.6918240 · Cited by 7 · cited 1×
“In [39], the authors show how parameters can be”
BackgroundMarchal, S., François, J., State, R., & Engel, T. (2012) · IEEE International Workshop on Information Forensics and Security · doi:10.1109/wifs.2012.6412631 · Cited by 6 · cited 1×
“It is also different from [42], where we determine URL set maliciousness based on semantic analysis for forensic purpose.”
BackgroundSelf-citationPhishLive: A View of Phishing and Malware Attacks from an Edge Router
Cao, L., Probst, T., & Kompella, R. R. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-36516-4_24 · Cited by 2 · cited 2×
“[43] L. Cao, T. Probst, and R. Kompella, “PhishLive: A view of phishing and malware attacks from an edge router,” in Proc.”
Method
Research works printed but not linked: 8
- zhang, 2012, Parsing and detecting phishing pages based on semantic understanding of text, J Inf & Comput Sci, 9, 1521
- mockapetris, 1987, RFC 1034 Domain Names—Concepts and Facilities
- 2014, Global Phishing Survey Trends and Domain Name Use
- 2014, Phishing Activity Trends Report
- 2008, SSAC advisory on fast flux hosting and DNS
- 2010, 2010 Identity Fraud Survey Report
- 2007, Gartner survey shows phishing attacks escalated in 2007
- rech, 2007, ACM SIGSOFT Softw Eng Notes, 32, 1
Cited by
Malicious URL Detection using Machine Learning: A Survey
Sahoo, D., Liu, C., & Hoi, S. C. H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.07179 · Cited by 274
“[41], [42] define a new concept of intraURL relatedness which is a measure to quantify the relations”
BackgroundMethodDetecting Internet of Things attacks using distributed deep learning
De La Torre Parra, G., Rad, P., Choo, K.-K. R., & Beebe, N. (2020) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2020.102662 · Cited by 328
Classifying phishing URLs using recurrent neural networks
Bahnsen, A. C., Bohorquez, E. C., Villegas, S., Vargas, J., & Gonzalez, F. A. (2017) · 2017 APWG Symposium on Electronic Crime Research (eCrime) · doi:10.1109/ecrime.2017.7945048 · Cited by 228
“Some of the machine learning methods that have been used to detect phishing include: support vector machines [17], streaming analytics [18], gradient boosting [6], [19], random forests [20], latent Dirichlet allocation [21], online incremental learning [22], and neural networks [23].”
MethodPhishing or Not Phishing? A Survey on the Detection of Phishing Websites
Zieni, R., Massari, L., & Calzarossa, M. C. (2023) · IEEE Access · doi:10.1109/access.2023.3247135 · Cited by 158
“, [49], [67], [76], [82], [91], [106], [128]).”
BackgroundInfluentialA survey of phishing attack techniques, defence mechanisms and open research challenges
Jain, A. K., & Gupta, B. B. (2022) · Enterprise Information Systems · doi:10.1080/17517575.2021.1896786 · Cited by 192
Metadata
- Authors
- Marchal, S., Francois, J., State, R., & Engel, T.
- Affiliations
- University of Luxembourg (LU)
- Venue
- IEEE Transactions on Network and Service Management
- Published
- 2014-12
- Volume
- 11
- Issue
- 4
- Pages
- 458-471
- ISSN
- 1932-4537 (ISSN-L)
- OpenAlex
- W2040750298
- Cited by
- 271
- References
- 50
- Citations by year
- 2026: 47 · 2025: 41 · 2024: 34 · 2023: 23 · 2022: 23 · 2021: 26 · 2020: 24 · 2019: 15 · 2018: 12 · 2017: 16 · 2016: 6 · 2015: 4
- FWCI
- 24.04
- Type
- journal article
- Language
- en
- Open access
- green
- orbilu.uni.lu
- Locations
- IEEE Transactions on Network and Service Management (publishedVersion)
- Open Repository and Bibliography (University of Luxembourg) (submittedVersion, other-oa)
- HAL (submittedVersion, other-oa)
- Funders
- Fonds National de la Recherche, Luxembourg (3967419)
- Topics
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Security and Intrusion Detection
- Keywords
- phishing detection, phishing URL classification, URL features, streaming analytics, storms, Bloom filter, email security, HTTP proxy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.