The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
What's new on the web?: the evolution of the web from a search engine perspective
Abstract
We seek to gain improved insight into how Web search engines shouldcope with the evolving Web, in an attempt to provide users with themost up-to-date results possible. For this purpose we collectedweekly snapshots of some 150 Web sites over the course of one year,and measured the evolution of content and link structure. Our measurements focus on aspects of potential interest to search engine designers: the evolution of link structure over time, the rate ofcreation of new pages and new distinct content on the Web, and the rate of change of the content of existing pages under search-centric measures of degree of change.Our findings indicate a rapid turnover rate of Web pages, i.e.,high rates of birth and death, coupled with an even higher rate ofturnover in the hyperlinks that connect them. For pages that persistover time we found that, perhaps surprisingly, the degree of contentshift as measured using TF.IDF cosine distance does not appear to beconsistently correlated with the frequency of contentupdating. Despite this apparent non-correlation, the rate of content shift of a given page is likely to remain consistent over time. That is, pages that change a great deal in one week will likely change by a similarly large degree in the following week. Conversely, pages that experience little change will continue to experience little change. We conclude the paper with a discussion of the potential implications ofour results for the design of effective Web search engines.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The anatomy of a large-scale hypertextual Web search engine
Brin, S., & Page, L. (1998) · Computer Networks and ISDN Systems · doi:10.1016/s0169-7552(98)00110-x · Cited by 16,004
Introduction to Modern Information Retrieval
Salton, G., & McGill, M. J. (1983) · Virtual Defense Library (Ministerio de Defensa) · Cited by 11,392
Broder, A., Kumar, R., Maghoul, F., Raghavan, P., Rajagopalan, S., Stata, R., Tomkins, A., & Wiener, J. (2000) · Computer Networks · doi:10.1016/s1389-1286(00)00083-9 · Cited by 2,810
Syntactic clustering of the Web
Broder, A. Z., Glassman, S. C., Manasse, M. S., & Zweig, G. (1997) · Computer Networks and ISDN Systems · doi:10.1016/s0169-7552(97)00031-7 · Cited by 1,343
Introduction to Modern Information Retrieval
Chowdhury, G. G. (2004) · Cited by 922
Stochastic models for the Web graph
Kumar, R., Raghavan, P., Rajagopalan, S., Sivakumar, D., Tomkins, A., & Upfal, E. (2002) · Proceedings 41st Annual Symposium on Foundations of Computer Science · doi:10.1109/sfcs.2000.892065 · Cited by 676
Mining the Web's link structure
Chakrabarti, S., Dom, B. E., Kumar, S. R., Raghavan, P., Rajagopalan, S., Tomkins, A., Gibson, D., & Kleinberg, J. (1999) · Computer · doi:10.1109/2.781636 · Cited by 501
The Evolution of the Web and Implications for an Incremental Crawler
Cho, J. J., & García-Molina, H. (2000) · Cited by 499
Self-citationCopy detection mechanisms for digital documents
Brin, S., Davis, J., & García-Molina, H. (1995) · Proceedings of the 1995 ACM SIGMOD international conference on Management of data - SIGMOD '95 · doi:10.1145/223784.223855 · Cited by 479
Synchronizing a database to improve freshness
Cho, J., & Garcia-Molina, H. (2000) · ACM SIGMOD Record · doi:10.1145/335191.335391 · Cited by 346
Self-citation
Cited by
Graph evolution: Densification and shrinking diameters
Leskovec, J., Kleinberg, J., & Faloutsos, C. (2007) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/1217299.1217301 · Cited by 2,461
Graphs over time: densification laws, shrinking diameters and possible explanations
Leskovec, J., Kleinberg, J., & Faloutsos, C. (2005) · Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining · doi:10.1145/1081870.1081893 · Cited by 2,281
Structure and evolution of online social networks
Kumar, R., Novak, J., & Tomkins, A. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150476 · Cited by 1,084
Microscopic evolution of social networks
Leskovec, J., Backstrom, L., Kumar, R., & Tomkins, A. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401948 · Cited by 728
A multifaceted approach to understanding the botnet phenomenon
Abu Rajab, M., Zarfoss, J., Monrose, F., & Terzis, A. (2006) · Proceedings of the 6th ACM SIGCOMM conference on Internet measurement · doi:10.1145/1177080.1177086 · Cited by 595
Structure and Evolution of Online Social Networks
Kumar, R., Novak, J., & Tomkins, A. (2010) · Link Mining: Models, Algorithms, and Applications · doi:10.1007/978-1-4419-6515-8_13 · Cited by 431
Herrmann, D., Wendolsky, R., & Federrath, H. (2009) · Proceedings of the 2009 ACM workshop on Cloud computing security · doi:10.1145/1655008.1655013 · Cited by 424
Olston, C., & Najork, M. (2010) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000017 · Cited by 370
Self-citationGraph Evolution: Densification and Shrinking Diameters
Leskovec, J., Kleinberg, J. M., & Faloutsos, C. (2006) · arXiv (Cornell University) · doi:10.48550/arxiv.physics/0603229 · Cited by 300
Repeatable evaluation of search services in dynamic environments
Jensen, E. C., Beitzel, S. M., Chowdhury, A., & Frieder, O. (2007) · ACM Transactions on Information Systems · doi:10.1145/1292591.1292592 · Cited by 269
Survey of Temporal Information Retrieval and Related Applications
Campos, R., Dias, G., Jorge, A. M., & Jatowt, A. (2015) · ACM Computing Surveys · doi:10.1145/2619088 · Cited by 262
Understanding temporal query dynamics
Kulkarni, A., Teevan, J., Svore, K. M., & Dumais, S. T. (2011) · Proceedings of the fourth ACM international conference on Web search and data mining · doi:10.1145/1935826.1935862 · Cited by 94
Web accessibility guidelines: A lesson from the evolving Web
Harper, S., & Chen, A. Q. (2012) · World Wide Web · doi:10.1007/s11280-011-0130-8 · Cited by 72
Leveraging temporal dynamics of document content in relevance ranking
Elsas, J. L., & Dumais, S. T. (2010) · Proceedings of the third ACM international conference on Web search and data mining · doi:10.1145/1718487.1718489 · Cited by 54
Traffic in Social Media II: Modeling Bursty Popularity
Ratkiewicz, J., Menczer, F., Fortunato, S., Flammini, A., & Vespignani, A. (2010) · 2010 IEEE Second International Conference on Social Computing · doi:10.1109/socialcom.2010.63 · Cited by 38
Berberich, K., Vazirgiannis, M., & Weikum, G. (2005) · Internet Mathematics · doi:10.1080/15427951.2005.10129110 · Cited by 32
What's really new on the web?: identifying new pages from a series of unstable web snapshots
Toyoda, M., & Kitsuregawa, M. (2006) · Proceedings of the 15th international conference on World Wide Web · doi:10.1145/1135777.1135815 · Cited by 21
Using neighbors to date web documents
Nunes, S., Ribeiro, C., & David, G. (2007) · Proceedings of the 9th annual ACM international workshop on Web information and data management · doi:10.1145/1316902.1316924 · Cited by 17
Metadata
- Authors
- Ntoulas, A., Cho, J., & Olston, C.
- Venue
- Proceedings of the 13th international conference on World Wide Web
- Publisher
- ACM
- Published
- 2004-05-17
- Pages
- 1-12
- OpenAlex
- W2157748587
- Cited by
- 452
- References
- 18
- FWCI
- 31.22
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.