The DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
What's new on the web?: the evolution of the web from a search engine perspective
Abstract
We seek to gain improved insight into how Web search engines shouldcope with the evolving Web, in an attempt to provide users with themost up-to-date results possible. For this purpose we collectedweekly snapshots of some 150 Web sites over the course of one year,and measured the evolution of content and link structure. Our measurements focus on aspects of potential interest to search engine designers: the evolution of link structure over time, the rate ofcreation of new pages and new distinct content on the Web, and the rate of change of the content of existing pages under search-centric measures of degree of change.Our findings indicate a rapid turnover rate of Web pages, i.e.,high rates of birth and death, coupled with an even higher rate ofturnover in the hyperlinks that connect them. For pages that persistover time we found that, perhaps surprisingly, the degree of contentshift as measured using TF.IDF cosine distance does not appear to beconsistently correlated with the frequency of contentupdating. Despite this apparent non-correlation, the rate of content shift of a given page is likely to remain consistent over time. That is, pages that change a great deal in one week will likely change by a similarly large degree in the following week. Conversely, pages that experience little change will continue to experience little change. We conclude the paper with a discussion of the potential implications ofour results for the design of effective Web search engines.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The anatomy of a large-scale hypertextual Web search engine
Brin, S., & Page, L. (1998) · Computer Networks and ISDN Systems · doi:10.1016/s0169-7552(98)00110-x · Cited by 16,004
Introduction to Modern Information Retrieval
Salton, G., & McGill, M. J. (1983) · Virtual Defense Library (Ministerio de Defensa) · Cited by 11,390
Broder, A., Kumar, R., Maghoul, F., Raghavan, P., Rajagopalan, S., Stata, R., Tomkins, A., & Wiener, J. (2000) · Computer Networks · doi:10.1016/s1389-1286(00)00083-9 · Cited by 2,810
Syntactic clustering of the Web
Broder, A. Z., Glassman, S. C., Manasse, M. S., & Zweig, G. (1997) · Computer Networks and ISDN Systems · doi:10.1016/s0169-7552(97)00031-7 · Cited by 1,343
Introduction to Modern Information Retrieval
Chowdhury, G. G. (2004) · Cited by 922
Stochastic models for the Web graph
Kumar, R., Raghavan, P., Rajagopalan, S., Sivakumar, D., Tomkins, A., & Upfal, E. (2002) · Proceedings 41st Annual Symposium on Foundations of Computer Science · doi:10.1109/sfcs.2000.892065 · Cited by 676
Mining the Web's link structure
Chakrabarti, S., Dom, B. E., Kumar, S. R., Raghavan, P., Rajagopalan, S., Tomkins, A., Gibson, D., & Kleinberg, J. (1999) · Computer · doi:10.1109/2.781636 · Cited by 501
The Evolution of the Web and Implications for an Incremental Crawler
Cho, J. J., & García-Molina, H. (2000) · Cited by 499
Self-citationCopy detection mechanisms for digital documents
Brin, S., Davis, J., & García-Molina, H. (1995) · Proceedings of the 1995 ACM SIGMOD international conference on Management of data - SIGMOD '95 · doi:10.1145/223784.223855 · Cited by 479
Synchronizing a database to improve freshness
Cho, J., & Garcia-Molina, H. (2000) · ACM SIGMOD Record · doi:10.1145/335191.335391 · Cited by 346
Self-citationRate of change and other metrics: a live study of the world wide web
Douglis, F., Feldmann, A., Krishnamurthy, B., & Mogul, J. C. (1997) · Cited by 319
Brewington, B. E., & Cybenko, G. (2000) · Computer Networks · doi:10.1016/s1389-1286(00)00045-1 · Cited by 292
A large‐scale study of the evolution of Web pages
Fetterly, D., Manasse, M., Najork, M., & Wiener, J. L. (2004) · Software: Practice and Experience · doi:10.1002/spe.577 · Cited by 202
A large-scale study of the evolution of web pages
Fetterly, D., Manasse, M., Najork, M., & Wiener, J. (2003) · Proceedings of the twelfth international conference on World Wide Web - WWW '03 · doi:10.1145/775152.775246 · Cited by 189
Synchronizing a database to improve freshness
Cho, J. J., & García-Molina, H. (2000) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/342009.335391 · Cited by 170
Self-citationLife, death, and lawfulness on the electronic frontier
Pitkow, J. E., & Pirolli, P. L. T. (1997) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/258549.258805 · Cited by 158
Approximating Aggregate Queries about Web Pages via Random Walks
Bar-Yossef, Z., Berg, A., Chien, S., Fakcharoenphol, J., & Weitz, D. (2000) · Cited by 125
Optimal robot scheduling for Web search engines
Coffman, E. G., Liu, Z., & Weber, R. (1998) · Journal of Scheduling · Cited by 117
Cited by
Graph evolution: Densification and shrinking diameters
Leskovec, J., Kleinberg, J., & Faloutsos, C. (2007) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/1217299.1217301 · Cited by 2,461
Graphs over time: densification laws, shrinking diameters and possible explanations
Leskovec, J., Kleinberg, J., & Faloutsos, C. (2005) · Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining · doi:10.1145/1081870.1081893 · Cited by 2,281
Structure and evolution of online social networks
Kumar, R., Novak, J., & Tomkins, A. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150476 · Cited by 1,084
Microscopic evolution of social networks
Leskovec, J., Backstrom, L., Kumar, R., & Tomkins, A. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401948 · Cited by 728
A multifaceted approach to understanding the botnet phenomenon
Abu Rajab, M., Zarfoss, J., Monrose, F., & Terzis, A. (2006) · Proceedings of the 6th ACM SIGCOMM conference on Internet measurement · doi:10.1145/1177080.1177086 · Cited by 595
Metadata
- Authors
- Ntoulas, A., Cho, J., & Olston, C.
- Venue
- Proceedings of the 13th international conference on World Wide Web
- Publisher
- ACM
- Published
- 2004-05-17
- Pages
- 1-12
- OpenAlex
- W2157748587
- Cited by
- 452
- References
- 18
- FWCI
- 31.22
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.