CoChat
Sign Up
Cited by 21

The DOI matches its Crossref record.

What's really new on the web?: identifying new pages from a series of unstable web snapshots

Toyoda, Masashi, Kitsuregawa, Masaru·2006·Proceedings of the 15th international conference on World Wide Web
DOI 10.1145/1135777.1135815
Open this paper in CoChat — free account

Abstract

Identifying and tracking new information on the Web is important in sociology, marketing, and survey research, since new trends might be apparent in the new information. Such changes can be observed by crawling the Web periodically. In practice, however, it is impossible to crawl the entire expanding Web repeatedly. This means that the novelty of a page remains unknown, even if that page did not exist in previous snapshots. In this paper, we propose a novelty measure for estimating the certainty that a newly crawled page appeared between the previous and current crawls. Using this novelty measure, new pages can be extracted from a series of unstable snapshots for further analysis and mining to identify new trends on the Web. We evaluated the precision, recall, and miss rate of the novelty measure using our Japanese web archive, and applied it to a Web archive search engine.

Overview

Integrity signals

Registry

Crossref · 10.1145/1135777.1135815

Cites

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 22

  1. 10.1002/asi.20082 · DOI
  2. 10.1145/988672.988716 · DOI
  3. M. K. Bergman . The Deep Web : Surfacing Hidden Value. Journal of Electronic Publishing , 7 ( 1 ), 2001 . M. K. Bergman. The Deep Web: Surfacing Hidden Value. Journal of Electronic Publishing, 7(1), 2001.
  4. 10.5555/297805.297941 · DOI
  5. 10.5555/645496.757722 · DOI
  6. 10.5555/347319.346287 · DOI
  7. 10.5555/297805.297827 · DOI
  8. 10.5555/283554.283370 · DOI
  9. 10.5555/347319.346290 · DOI
  10. J. Cho and H. Garcia-Molina . The Evolution of the Web and Implications for an Incremental Crawler . In Proceedings of the 26th International Conference on Very Large Databases (VLDB) , pages 200 -- 209 , 2000 . J. Cho and H. Garcia-Molina. The Evolution of the Web and Implications for an Incremental Crawler. In Proceedings of the 26th International Conference on Very Large Databases (VLDB), pages 200--209, 2000.
  11. 10.1145/988672.988714 · DOI
  12. 10.1145/775152.775246 · DOI · Paper page
  13. 10.1145/988672.988739 · DOI
  14. J. M. Kleinberg . Authoritative Sources in a Hyperlinked Environment . In Proceedings of the ACM-SIAM Symposium on Discrete Algorithms , pages 668 -- 677 , 1998 . J. M. Kleinberg. Authoritative Sources in a Hyperlinked Environment. In Proceedings of the ACM-SIAM Symposium on Discrete Algorithms, pages 668--677, 1998.
  15. 10.1145/775152.775233 · DOI
  16. 10.5555/795666.796570 · DOI
  17. 10.5555/645925.671512 · DOI
  18. 10.1145/371920.371965 · DOI
  19. 10.1145/988672.988674 · DOI
  20. S. R. Ravi Kumar , Prabhakar Raghavan and A. Tomkins. Trawling the Web for Emerging Cyber-Communities. In Proceedings of the 8th International World Wide Web Conference , pages 403 -- 415 , 1999 . S. R. Ravi Kumar, Prabhakar Raghavan and A. Tomkins. Trawling the Web for Emerging Cyber-Communities. In Proceedings of the 8th International World Wide Web Conference, pages 403--415, 1999.
  21. 10.1145/504216.504244 · DOI
  22. 10.1145/900051.900059 · DOI

Cited by

Sorted by citations

See all 21

Metadata

Authors
Toyoda, M., & Kitsuregawa, M.
Affiliations
University of Tokyo, Tokyo, JAPAN
Venue
Proceedings of the 15th international conference on World Wide Web
Publisher
ACM
Published
2006-05-23
Pages
233-241
DOI
10.1145/1135777.1135815
Cited by
21
References
0
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review