The DOI matches its Crossref record; 96 of the 96 references it cites were found in a registry or an index.
Survey on web spam detection: principles and algorithms
Abstract
Search engines became a de facto place to start information acquisition on the Web. Though due to web spam phenomenon, search results are not always as good as desired. Moreover, spam evolves that makes the problem of providing high quality search even more challenging. Over the last decade research on adversarial information retrieval has gained a lot of interest both from academia and industry. In this paper we present a systematic review of web spam detection techniques with the focus on algorithms and underlying principles. We categorize all existing algorithms into three categories based on the type of information they use: content-based methods, link-based methods, and methods based on non-traditional data such as user behaviour, clicks, HTTP sessions. In turn, we perform a subcategorization of link-based category into five groups based on ideas and principles used: labels propagation, link pruning and reweighting, labels refinement, graph regularization, and featurebased. We also define the concept of web spam numerically and provide a brief survey on various spam forms. Finally, we summarize the observations and underlying principles applied for web spam detection.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1999) · Cited by 12,854
Introduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278
Authoritative sources in a hyperlinked environment
Kleinberg, J. M. (1999) · Journal of the ACM · doi:10.1145/324133.324140 · Cited by 8,959
Solutions of Ill-Posed Problems.
Bell, J. B., Tikhonov, A. N., & Arsenin, V. Y. (1978) · Mathematics of Computation · doi:10.2307/2006360 · Cited by 8,101
A vector space model for automatic indexing
Salton, G., Wong, A., & Yang, C. S. (1975) · Communications of the ACM · doi:10.1145/361219.361220 · Cited by 7,521 · cited 1×
“They also show that 70% of pages in the *.biz domain and 20% in the *.com domain are spam.”
BackgroundNonnegative Matrices in the Mathematical Sciences
(1979) · Elsevier eBooks · doi:10.1016/c2013-0-10361-3 · Cited by 5,451
Spline Models for Observational Data.
Muller, H.-G., & Wahba, G. (1991) · Journal of the American Statistical Association · doi:10.2307/2290434 · Cited by 4,928
Spline Models for Observational Data
Wahba, G. (1990) · doi:10.1137/1.9781611970128 · Cited by 4,184
Learning with Local and Global Consistency
Zhou, D., Bousquet, O., Lal, T. N., Weston, J., & Schölkopf, B. (2003) · MPG.PuRe (Max Planck Society) · Cited by 3,864
SimRank: a measure of structural-context similarity
Jeh, G., & Widom, J. (2002) · Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/775047.775126 · Cited by 1,925
Encyclopedia of Database Systems
(2009) · Springer eBooks · doi:10.1007/978-0-387-39940-9 · Cited by 1,739
Multilevelk-way Partitioning Scheme for Irregular Graphs
Karypis, G., & Kumar, V. (1998) · Journal of Parallel and Distributed Computing · doi:10.1006/jpdc.1997.1404 · Cited by 1,675
Haveliwala, T. H. (2002) · Proceedings of the 11th international conference on World Wide Web · doi:10.1145/511446.511513 · Cited by 1,567
Propagation of trust and distrust
Guha, R., Kumar, R., Raghavan, P., & Tomkins, A. (2004) · Proceedings of the 13th international conference on World Wide Web · doi:10.1145/988672.988727 · Cited by 1,464
Accurately interpreting clickthrough data as implicit feedback
Joachims, T., Granka, L., Pan, B., Hembrooke, H., & Gay, G. (2005) · Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1076034.1076063 · Cited by 1,408
Syntactic clustering of the Web
Broder, A. Z., Glassman, S. C., Manasse, M. S., & Zweig, G. (1997) · Computer Networks and ISDN Systems · doi:10.1016/s0169-7552(97)00031-7 · Cited by 1,343
A Bayesian Approach to Filtering Junk E-Mail
Sahami, M., Dumais, S., Heckerman, D. E., & Horvitz, E. (1998) · Cited by 1,206
Scaling personalized web search
Jeh, G., & Widom, J. (2003) · Proceedings of the twelfth international conference on World Wide Web - WWW '03 · doi:10.1145/775152.775191 · Cited by 1,195
Analysis of a very large web search engine query log
Silverstein, C., Marais, H., Henzinger, M., & Moricz, M. (1999) · ACM SIGIR Forum · doi:10.1145/331403.331405 · Cited by 1,153
Combating Web Spam with TrustRank
Gyöngyi, Z., Garcia-Molina, H., & Pedersen, J. (2004) · Proceedings 2004 VLDB Conference · doi:10.1016/b978-012088469-8.50052-8 · Cited by 1,085
CBMS-NSF REGIONAL CONFERENCE SERIES IN APPLIED MATHEMATICS
Gottlieb, D., Orszag, S. A., Huber, P. J., & Roberts, F. S. (2011) · Cited by 999
Langville, A., & Meyer, C. (2004) · Internet Mathematics · doi:10.1080/15427951.2004.10129091 · Cited by 894
Dalvi, N., Domingos, P., Mausam, Sanghai, S., & Verma, D. (2004) · Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1014052.1014066 · Cited by 887
Simple BM25 extension to multiple weighted fields
Robertson, S., Zaragoza, H., & Taylor, M. (2004) · Proceedings of the thirteenth ACM international conference on Information and knowledge management · doi:10.1145/1031171.1031181 · Cited by 701
Mining the Web: Discovering Knowledge from Hypertext Data
Chakrabarti, S. (2002) · Thammasat University Digital Collections · Cited by 696
Improved Algorithms for Topic Distillation in a Hyperlinked Environment
Bharat, K., & Henzinger, M. R. (2017) · ACM SIGIR Forum · doi:10.1145/3130348.3130367 · Cited by 676
Detecting spam web pages through content analysis
Ntoulas, A., Najork, M., Manasse, M., & Fetterly, D. (2006) · Proceedings of the 15th international conference on World Wide Web · doi:10.1145/1135777.1135794 · Cited by 608
Solutions of ill-posed problems
Tikhonov, A. N., & Arsenin, V. Y. (1977) · International Linear Collider · Cited by 574
Bianchini, M., Gori, M., & Scarselli, F. (2005) · ACM Transactions on Internet Technology · doi:10.1145/1052934.1052938 · Cited by 539
A Survey on PageRank Computing
Berkhin, P. (2005) · Internet Mathematics · doi:10.1080/15427951.2005.10129098 · Cited by 474
Improved algorithms for topic distillation in a hyperlinked environment
Bharat, K., & Henzinger, M. R. (1998) · Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/290941.290972 · Cited by 444
A survey of learning-based techniques of email spam filtering
Blanzieri, E., & Bryl, A. (2008) · Artificial Intelligence Review · doi:10.1007/s10462-009-9109-6 · Cited by 407
SALSA: the stochastic approach for link-structure analysis
Lempel, R., & Moran, S. (2001) · ACM Transactions on Information Systems · doi:10.1145/382979.383041 · Cited by 362
Know your neighbors: web spam detection using the web topology
Castillo, C., Donato, D., Gionis, A., Murdock, V., & Silvestri, F. (2007) · Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1277741.1277814 · Cited by 310
Challenges in web search engines
Henzinger, M. R., Motwani, R., & Silverstein, C. (2002) · ACM SIGIR Forum · doi:10.1145/792550.792553 · Cited by 307
Spam, damn spam, and statistics: using statistical analysis to locate spam web pages
Fetterly, D., Manasse, M., & Najork, M. (2004) · Proceedings of the 7th International Workshop on the Web and Databases: colocated with ACM SIGMOD/PODS 2004 · doi:10.1145/1017074.1017077 · Cited by 291
Davison, B. D. (2000) · Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/345508.345597 · Cited by 264
Fighting Spam on Social Web Sites: A Survey of Approaches and Future Challenges
Heymann, P., Koutrika, G., & Garcia-Molina, H. (2007) · IEEE Internet Computing · doi:10.1109/mic.2007.125 · Cited by 260
GENVL and WWWW: Tools for taming the Web
McBryan, O. A. (1994) · Computer Networks and ISDN Systems · doi:10.1016/s0169-7552(94)90149-x · Cited by 242
Identifying link farm spam pages
Wu, B., & Davison, B. D. (2005) · Special interest tracks and posters of the 14th international conference on World Wide Web - WWW '05 · doi:10.1145/1062745.1062762 · Cited by 241
Finding authorities and hubs from link structures on the World Wide Web
Borodin, A., Roberts, G. O., Rosenthal, J. S., & Tsaparas, P. (2001) · Proceedings of the 10th international conference on World Wide Web · doi:10.1145/371920.372096 · Cited by 230
Blocking Blog Spam with Language Model Disagreement
Mishne, G., Carmel, D., & Lempel, R. (2005) · UvA-DARE (University of Amsterdam) · Cited by 216
Statistical language models for information retrieval
Zhai, C. (2007) · Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Tutorial Abstracts on XX - NAACL '07 · doi:10.3115/1614181.1614183 · Cited by 213
Eiron, N., McCurley, K. S., & Tomlin, J. A. (2004) · Proceedings of the 13th international conference on World Wide Web · doi:10.1145/988672.988714 · Cited by 209
Link Analysis, Eigenvectors and Stability
Ng, A. Y., Zheng, A. X., & Jordan, M. I. (2001) · Cited by 206
Some applications of Rabin’s fingerprinting method
Broder, A. Z. (1993) · Sequences II · doi:10.1007/978-1-4613-9323-8_11 · Cited by 204
A reference collection for web spam
Castillo, C., Donato, D., Becchetti, L., Boldi, P., Leonardi, S., Santini, M., & Vigna, S. (2006) · ACM SIGIR Forum · doi:10.1145/1189702.1189703 · Cited by 202
BrowseRank: letting web users vote for page importance
Liu, Y., Gao, B., Liu, T.-Y., Zhang, Y., Ma, Z., He, S., & Li, H. (2008) · Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1390334.1390412 · Cited by 172
Gyöngyi, Z., & García-Molina, H. (2005) · Cited by 165
Using PageRank to Characterize Web Structure
Pandurangan, G., Raghavan, P., & Upfal, E. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-45655-4_36 · Cited by 164
Daswani, N., & Stoppelman, M. (2007) · Cited by 156
On the evolution of clusters of near-duplicate Web pages
Fetterly, D., Manasse, M. S., & Najork, M. (2004) · doi:10.1109/laweb.2003.1250280 · Cited by 153
Recognizing Nepotistic Links on the Web
Davison, B. D. (2000) · Cited by 152
Web Spam Detection with Anti-Trust Rank
Krishnan, V., & Raj, R. (2006) · Cited by 142
Link spam detection based on mass estimation
Gyöngyi, Z., Berkhin, P., & Yahoo Inc (2006) · Cited by 140
Castillo, C., & Davison, B. D. (2010) · now publishers, Inc. eBooks · doi:10.1561/1500000021 · Cited by 136
Detecting phrase-level duplication on the world wide web
Fetterly, D., Manasse, M. S., & Najork, M. (2005) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1076034.1076066 · Cited by 131
Detecting Spam Blogs: A Machine Learning Approach
Kolari, P., Java, A., Finin, T., Oates, T., & Joshi, A. (2006) · Maryland Shared Open Access Repository (USMAI Consortium) · doi:10.13016/m27m0444d · Cited by 127
Link-Based Characterization and Detection of Web Spam
Becchetti, L., Castillo, C., Donato, D., Leonardi, S., & Baeza-Yates, R. A. (2006) · Cited by 127
Improvement of HITS-based algorithms on web documents
Li, L., Shang, Y., & Zhang, W. (2002) · International World Wide Web Conference (WWW) · doi:10.1145/511446.511514 · Cited by 115
The connectivity sonar: detecting site functionality by structural patterns
Amitay, E., Carmel, D., Darlow, A., Lempel, R., & Soffer, A. (2003) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/900051.900060 · Cited by 115
Making Eigenvector-Based Reputation Systems Robust to Collusion
Zhang, H., Goel, A., Govindan, R., Mason, K., & Roy, B. V. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-30216-2_8 · Cited by 112
Propagating Trust and Distrust to Demote Web Spam.
Wu, B., Goel, V., & Davison, B. D. (2006) · Cited by 105
Baeza-Yates, R. A., Boldi, P., & Castillo, C. (2006) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1148170.1148225 · Cited by 101
Cloaking and Redirection: A Preliminary Study.
Wu, B., & Davison, B. D. (2005) · Cited by 99
Proceedings of the 2024 SIAM International Conference on Data Mining (SDM)
(2024) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611978032 · Cited by 98
Wang, Y., Ma, M., Niu, Y., & Chen, H. (2007) · International World Wide Web Conference (WWW) · doi:10.1145/1242572.1242612 · Cited by 89
Towards Scaling Fully Personalized PageRank
Fogaras, D., & Rácz, B. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-30216-2_9 · Cited by 84 · cited 9×
“The key part is to initialize the personalization vector Tr in PPR by normalizing the predictions of the base algorithm: rp = .s(p) , where s(p) is a predic- p.V s(p) tion of the base algorithm and rp is the component of the vector Tr corresponding to page p. Finally, the third strategy is to use stacked graphical le…”
MethodInfluentialAre click-through data adequate for learning web search rankings?
Dou, Z., Song, R., Yuan, X., & Wen, J.-R. (2008) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/1458082.1458095 · Cited by 82
Detecting semantic cloaking on the web
Wu, B., & Davison, B. D. (2006) · International World Wide Web Conference (WWW) · doi:10.1145/1135777.1135901 · Cited by 77
Exploring linguistic features for web spam detection: a preliminary study
Jakub Piskorski, B., Marcin Sydow, A., & Dawid Weiss, C. (2008) · Adversarial Information Retrieval on the Web · doi:10.1145/1451983.1451990 · Cited by 68
Stacked Graphical Models for Efficient Inference in Markov Random Fields
Kou, Z., & Cohen, W. W. (2007) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972771.57 · Cited by 63
A taxonomy of JavaScript redirection spam
Chellapilla, K., & Maykov, A. (2007) · Adversarial Information Retrieval on the Web · doi:10.1145/1244408.1244423 · Cited by 62
Analysis and improvement of HITS algorithm for detecting Web communities
Nomura, S., Oyama, S., Hayamizu, T., & Ishida, T. (2003) · Systems and Computers in Japan · doi:10.1002/scj.10425 · Cited by 61
Yoshida, K., Adachi, F., Washio, T., Motoda, H., Homma, T., Nakashima, A., Fujikawa, H., & Yamazaki, K. (2004) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1014052.1014107 · Cited by 60
Statistical Language Models for Information Retrieval
Zhai, C. (2009) · Synthesis lectures on human language technologies · doi:10.1007/978-3-031-02130-5 · Cited by 60
Graph regularization methods for Web spam detection
Abernethy, J., Chapelle, O., & Castillo, C. (2010) · Machine Learning · doi:10.1007/s10994-010-5171-1 · Cited by 59
Splog detection using self-similarity analysis on blog temporal dynamics
Lin, Y., Sundaram, H., Chi, Y., Tatemura, J., & Tseng, B. L. (2007) · Adversarial Information Retrieval on the Web · doi:10.1145/1244408.1244410 · Cited by 59
Link-Based Similarity Search to Fight Web Spam
Benczúr, A. A., Csalogány, K., & Sarlós, T. (2006) · Cited by 57
Pagerank Increase under Different Collusion Topologies.
Baeza-Yates, R. A., Castillo, C., & López, V. (2005) · Cited by 56
Where to Start Browsing the Web?
Fogaras, D. (2003) · Lecture notes in computer science · doi:10.1007/978-3-540-39884-4_6 · Cited by 49
Improving web spam classification using rank-time features
Svore, K. M., Wu, Q., Burges, C. J. C., & Raman, A. P. (2007) · Adversarial Information Retrieval on the Web · doi:10.1145/1244408.1244411 · Cited by 45
Site level noise removal for search engines
Carvalho, A. L. D. C., Chirita, P. A., de Moura, E. S., Calado, P. P., & Nejdl, W. (2006) · International World Wide Web Conference (WWW) · doi:10.1145/1135777.1135793 · Cited by 44
Tracking Web Spam with Hidden Style Similarity
Urvoy, T., Lavergne, T., & Filoche, P. (2006) · Adversarial Information Retrieval on the Web · Cited by 44
Transductive link spam detection
Zhou, D., Burges, C. J. C., & Tao, T. (2007) · Adversarial Information Retrieval on the Web · doi:10.1145/1244408.1244413 · Cited by 43
Link spam target detection using page farms
Zhou, B., & Pei, J. (2009) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/1552303.1552306 · Cited by 41
Countering web spam with credibility-based link analysis
Caverlee, J., & Liu, L. (2007) · ACM Symposium on Principles of Distributed Computing (PODC) · doi:10.1145/1281100.1281124 · Cited by 41
Predicting web spam with HTTP session information
Webb, S., Caverlee, J., & Pu, C. (2008) · doi:10.1145/1458082.1458129 · Cited by 39
Optimal Link Bombs are Uncoordinated.
Adalı, S., Liu, T., & Magdon‐Ismail, M. (2005) · Adversarial Information Retrieval on the Web · Cited by 37
SpamRank - fully automatic link spam detection. Work in progress
Benczúr, A. A., Csalogány, K., Sarlós, K., & Uher, M. (2005) · SZTAKI Publication Repository (Hungarian Academy of Sciences) · Cited by 32
Algorithms and incentives for robust ranking
Bhattacharjee, R., & Goel, A. (2007) · Symposium on Discrete Algorithms · doi:10.5555/1283383.1283428 · Cited by 26
Wu, B., & Davison, B. D. (2006) · ACM Symposium on Applied Computing · doi:10.1145/1141277.1141535 · Cited by 19
Let web spammers expose themselves
Cheng, Z., Gao, B., Sun, C., Jiang, Y., & Liu, T. (2011) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/1935826.1935902 · Cited by 18
Sketching Landscapes of Page Farms
Zhou, B., & Pei, J. (2007) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972771.67 · Cited by 9
OSD: An Online Web Spam Detection System
Zhou, B., & Pei, J. (2009) · Cited by 4
(2005)
Cited by
A Survey of Fake News: Fundamental Theories, Detection Methods, and Opportunities
Zhou, X., & Zafarani, R. (2021) · ACM Computing Surveys · doi:10.1145/3395046 · Cited by 1,184
Ex Machina: Personal Attacks Seen at Scale
Wulczyn, E., Thain, N., & Dixon, L. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052591 · Cited by 636
What Yelp Fake Review Filter Might Be Doing?
Mukherjee, A., Venkataraman, V., Liu, B., & Glance, N. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v7i1.14389 · Cited by 557
A comprehensive survey of anomaly detection techniques for high dimensional big data
Thudumu, S., Branch, P., Jin, J., & Singh, J. (2020) · Journal of Big Data · doi:10.1186/s40537-020-00320-x · Cited by 425
Spotting opinion spammers using behavioral footprints
Mukherjee, A., Kumar, A., Liu, B., Wang, J., Hsu, M., Castellanos, M., & Ghosh, R. (2013) · Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/2487575.2487580 · Cited by 426
Metadata
- Authors
- Spirin, N., & Han, J.
- Venue
- ACM SIGKDD Explorations Newsletter
- Published
- 2012-05
- Volume
- 13
- Issue
- 2
- Pages
- 50-64
- ISSN
- 1931-0145 (ISSN-L)
- 1931-0153
- OpenAlex
- W2100738695
- Cited by
- 252
- References
- 96
- FWCI
- 51.55
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.