The DOI matches its Crossref record; 87 of the 87 references it cites were found in a registry or an index.
Web mining research: a survey
Open accessAbstract
With the huge amount of information available online, the World Wide Web is a fertile area for data mining research.The Web mining research is at the cross road of research from several research communities, such as database, information retrieval, and within AI, especially the sub-areas of machine learning and natural language processing.However, there is a lot of confusions when comparing research eorts from dierent point of views.In this paper, we survey the research in the area of Web mining, point out some confusions regarded the usage of the term Web mining and suggest three Web mining categories.Then we situate some of the research with respect to these three categories.We also explore the connection between the Web mining categories and the related agent paradigm.For the survey, we focus on representation issues, on the process, on the learning algorithm, and on the application of the recent works as the criteria.We conclude the paper with some research issues.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Lecture Notes in Computer Science 1205
(1999) · Industrial Robot: An International Journal · doi:10.1108/ir.1999.04926fae.001 · Cited by 38,662
The anatomy of a large-scale hypertextual Web search engine
Brin, S., & Page, L. (1998) · Computer Networks and ISDN Systems · doi:10.1016/s0169-7552(98)00110-x · Cited by 16,004
Indexing by latent semantic analysis
Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199009)41:6<391::aid-asi1>3.0.co;2-9 · Cited by 12,751
Introduction to Modern Information Retrieval
Salton, G., & McGill, M. J. (1983) · Virtual Defense Library (Ministerio de Defensa) · Cited by 9,680
Authoritative sources in a hyperlinked environment
Kleinberg, J. M. (1999) · Journal of the ACM · doi:10.1145/324133.324140 · Cited by 8,959
Proceedings of the 25th international conference on Machine learning - ICML '08
(2008) · doi:10.1145/1390156 · Cited by 6,680
Introduction to modern information retrieval
Dillon, M. (1983) · Information Processing & Management · doi:10.1016/0306-4573(83)90062-6 · Cited by 5,909
Advances in knowledge discovery and data mining
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90007-3 · Cited by 2,971
Fab: content-based, collaborative recommendation
Balabanović, M., & Shoham, Y. (1997) · Communications of the ACM · doi:10.1145/245108.245124 · Cited by 2,934
Proceedings of the 16th international joint conference on Artificial Intelligence - IJCAI '99
Colton, S., Bundy, A., & Walsh, T. (1999) · Cited by 2,500
From data mining to knowledge discovery: an overview
Fayyad, U. M., Piatetsky-Shapiro, G., & Smyth, P. (1996) · Knowledge Discovery and Data Mining · Cited by 2,395
Agents that reduce work and information overload
Maes, P. (1994) · Communications of the ACM · doi:10.1145/176789.176792 · Cited by 2,321
Web usage mining: discovery and applications of usage patterns from Web data
Srivastava, J., Cooley, R., Deshpande, M., & Tan, P.-N. (2000) · ACM SIGKDD Explorations Newsletter · doi:10.1145/846183.846188 · Cited by 2,105
Authoritative sources in a hyperlinked environment
Kleinberg, J. M. (1998) · Symposium on Discrete Algorithms · doi:10.5555/314613.315045 · Cited by 1,814
Inductive learning algorithms and representations for text categorization
Dumais, S., Platt, J., Heckerman, D., & Sahami, M. (1998) · Proceedings of the seventh international conference on Information and knowledge management · doi:10.1145/288627.288651 · Cited by 1,461
Data Preparation for Mining World Wide Web Browsing Patterns
Cooley, R., Mobasher, B., & Srivastava, J. (1999) · Knowledge and Information Systems · doi:10.1007/bf03325089 · Cited by 1,455
Accessibility of information on the web
Lawrence, S., & Giles, C. L. (1999) · Nature · doi:10.1038/21987 · Cited by 1,349
Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI 2017)
Amendola, G., Ricca, F., & Truszczyński, M. (2017) · International Joint Conference on Artificial Intelligence · Cited by 1,179
Web mining: information and pattern discovery on the World Wide Web
Cooley, R., Mobasher, B., & Srivastava, J. (2002) · Proceedings Ninth IEEE International Conference on Tools with Artificial Intelligence · doi:10.1109/tai.1997.632303 · Cited by 1,162
DataGuides: Enabling Query Formulation and Optimization in Semistructured Databases
Goldman, R., & Widom, J. (1997) · Cited by 1,136
The Lorel query language for semistructured data
Abiteboul, S., Quass, D., McHugh, J., Widom, J., & Wiener, J. L. (1997) · International Journal on Digital Libraries · doi:10.1007/s007990050005 · Cited by 1,058
Trawling the Web for emerging cyber-communities
Kumar, R., Raghavan, P., Rajagopalan, S., & Tomkins, A. (1999) · Computer Networks · doi:10.1016/s1389-1286(99)00040-7 · Cited by 1,000
Topic Detection and Tracking Pilot Study Final Report
Allan, J., Carbonell, J., Doddington, G. R., Yamron, J., & Yang, Y. (2018) · doi:10.1184/r1/6626252.v1 · Cited by 939
Learning Information Extraction Rules for Semi-Structured and Free Text
Soderland, S. (1999) · Machine Learning · doi:10.1023/a:1007562322031 · Cited by 934
The TSIMMIS project: Integration of heterogeneous information sources
Chawathe, S. S., García-Molina, H., Hammer, J., Ireland, K., Yannis Papakonstantinou, D., Ullman, J. D., & Widom, J. (1994) · Cited by 924
Hearst, M. A. (1999) · Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics - · doi:10.3115/1034678.1034679 · Cited by 867
Knowledge discovery and data mining: towards a unifying framework
Fayyad, U. M., Piatetsky-Shapiro, G., & Smyth, P. (1996) · Cited by 846
Using Maximum Entropy for Text Classification
Nigam, K., Lafferty, J., & McCallum, A. (1999) · Cited by 781
Enhanced hypertext categorization using hyperlinks
Chakrabarti, S., Dom, B., & Indyk, P. (1998) · Proceedings of the 1998 ACM SIGMOD international conference on Management of data · doi:10.1145/276304.276332 · Cited by 769
Abiteboul, S. (1997) · Lecture Notes in Computer Science · doi:10.1007/3-540-62222-5_33 · Cited by 746
Cowie, J., & Lehnert, W. (1996) · Communications of the ACM · doi:10.1145/234173.234209 · Cited by 731
Machine learning and data mining
Mitchell, T. M. (1999) · Communications of the ACM · doi:10.1145/319382.319388 · Cited by 691
Web Watcher: A Tour Guide for the World Wide Web.
Joachims, T., Freitag, D., & Mitchell, T. M. (1997) · Cited by 685
Learning to extract symbolic knowledge from the World Wide Web
Craven, M. W., DiPasquo, D., Freitag, D., McCallum, A., Mitchell, T. M., Nigam, K., & Slattery, S. (1998) · Cited by 677
Improved Algorithms for Topic Distillation in a Hyperlinked Environment
Bharat, K., & Henzinger, M. R. (2017) · ACM SIGIR Forum · doi:10.1145/3130348.3130367 · Cited by 676
Domain-specific keyphrase extraction
Frank, E., Paynter, G. W., Witten, I. H., Gutwin, C. A., & Nevill-Manning, C. G. (1999) · Research Commons (University of Waikato) · Cited by 647
On-Line New Event Detection and Tracking
Allan, J., Papka, R., & Lavrenko, V. (2017) · ACM SIGIR Forum · doi:10.1145/3130348.3130366 · Cited by 646
Database techniques for the World-Wide Web: a survey
Florescu, D., Levy, A., & Mendelzon, A. (1998) · ACM SIGMOD Record · doi:10.1145/290593.290605 · Cited by 561
A query language and optimization techniques for unstructured data
Buneman, P., Davidson, S. B., Hillebrand, G. G., & Suciu, D. M. (1996) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/233269.233368 · Cited by 535
On-line new event detection and tracking
Allan, J., Papka, R., & Lavrenko, V. (1998) · Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/290941.290954 · Cited by 530
The World-Wide Web: quagmire or gold mine?
Etzioni, O. (1996) · Communications of the ACM · doi:10.1145/240455.240473 · Cited by 527
Mining the Web's link structure
Chakrabarti, S., Dom, B. E., Kumar, S. R., Raghavan, P., Rajagopalan, S., Tomkins, A., Gibson, D., & Kleinberg, J. (1999) · Computer · doi:10.1109/2.781636 · Cited by 501
WEBSOM – Self-organizing maps of document collections
Kaski, S., Honkela, T., Lagus, K., & Kohonen, T. (1998) · Neurocomputing · doi:10.1016/s0925-2312(98)00039-3 · Cited by 495
Buneman, P. (1997) · Proceedings of the sixteenth ACM SIGACT-SIGMOD-SIGART symposium on Principles of database systems · doi:10.1145/263661.263675 · Cited by 448
Improved algorithms for topic distillation in a hyperlinked environment
Bharat, K., & Henzinger, M. R. (1998) · Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/290941.290972 · Cited by 444
Feature Selection for Unbalanced Class Distribution and Naive Bayes
Mladenić, D., & Grobelnik, M. (1999) · Cited by 427
Generating finite-state transducers for semi-structured data extraction from the Web
Hsu, C.-N., & Dung, M.-T. (1998) · Information Systems · doi:10.1016/s0306-4379(98)00027-1 · Cited by 411
A Hybrid User Model for News Story Classification
Billsus, D., & Pazzani, M. J. (1999) · CISM International Centre for Mechanical Sciences · doi:10.1007/978-3-7091-2490-1_10 · Cited by 357
Learning approaches for detecting and tracking news events
Yang, Y., Carbonell, J. G., Brown, R. D., Pierce, T., Archibald, B. T., & Liu, X. (1999) · IEEE Intelligent Systems · doi:10.1109/5254.784083 · Cited by 339
Advances in Inductive Logic Programming
Raedt, L. D. (1996) · Cited by 318
Extracting Semistructured Information from the Web.
Hammer, J., García-Molina, H., Cho, J. J., Aranha, R. F. M., & García-Crespo, Á. (1997) · Cited by 309
Feature Engineering for Text Classification
Scott, S., & Matwin, S. (1999) · Cited by 307
Data mining for hypertext: a tutorial survey
Chakrabarti, S. (2000) · ACM SIGKDD Explorations Newsletter · doi:10.1145/846183.846187 · Cited by 291
Information Extraction with HMM Structures Learned by Stochastic Optimization
Freitag, D., & McCallum, A. (2000) · Cited by 282
Text-learning and related intelligent agents: a survey
Mladenic, D. (1999) · IEEE Intelligent Systems · doi:10.1109/5254.784084 · Cited by 282
Information Extraction with HMMs and Shrinkage
Freitag, D., & McCallum, A. K. (1999) · Cited by 237
Information extraction from HTML: application of a general machine learning approach
Freitag, D. (1998) · Cited by 236
Empirical Methods in Information Extraction
Cardie, C. (1997) · Cited by 227
Extraction Patterns for Information Extraction Tasks: A Survey
Muslea, I. (1999) · Cited by 226
Extracting schema from semistructured data
Nestorov, S., Abiteboul, S., & Motwani, R. (1998) · Proceedings of the 1998 ACM SIGMOD international conference on Management of data · doi:10.1145/276304.276331 · Cited by 224
Web usage mining: discovery and application of interesting patterns from web data
Cooley, R., & Srivastava, J. (2000) · University of Minnesota eBooks · Cited by 219
Maximizing text-mining performance
Weiss, S. M., Apte, C., Damerau, F. J., Johnson, D. E., Oles, F. J., Goetz, T., & Hampp, T. (1999) · IEEE Intelligent Systems · doi:10.1109/5254.784086 · Cited by 213
Using Reinforcement Learning to Spider the Web Efficiently
Rennie, J. D. M., & McCallum, A. (1999) · Cited by 212
Green, S., Hurst, L., Nangle, B., & Cunningham, P. (1997) · Arrow@dit (Dublin Institute of Technology) · Cited by 206
Research Issues in Web Data Mining
Madria, S. K., Bhowmick, S. S., Ng, W. K., & Lim, E. P. (1999) · Lecture Notes in Computer Science · doi:10.1007/3-540-48298-9_32 · Cited by 205
Mendelzon, A. O., Mihaila, G. A., & Milo, T. (2002) · Fourth International Conference on Parallel and Distributed Information Systems · doi:10.1109/pdis.1996.568671 · Cited by 198
Mining association rules in multiple relations
Dehaspe, L., & Raedt, L. (1997) · Lecture Notes in Computer Science · doi:10.1007/3540635149_40 · Cited by 188
A machine learning approach to building domain-specific search engines
McCallum, A., Nigam, K., Rennie, J. D. M., & Seymore, K. (1999) · Cited by 183
Atzeni, P., & Mecca, G. (1997) · Proceedings of the sixteenth ACM SIGACT-SIGMOD-SIGART symposium on Principles of database systems · doi:10.1145/263661.263678 · Cited by 182
User Modeling in Adaptive Interface
Langley, P. (1999) · CISM International Centre for Mechanical Sciences · doi:10.1007/978-3-7091-2490-1_48 · Cited by 171
Automating the analysis and cataloging of sky surveys
Fayyad, U. M., Djorgovski, S. G., & Weir, N. (1996) · Knowledge Discovery and Data Mining · Cited by 142
Information Extraction A Multidisciplinary Approach to an Emerging Information Technology
Pazienza, M. T. (1997) · Lecture notes in computer science · doi:10.1007/3-540-63438-x · Cited by 138
Discovering structural association of semistructured data
Wang, K., & Liu, H. (2000) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/69.846290 · Cited by 137
Inferring structure in semistructured data
Nestorov, S., Abiteboul, S., & Motwani, R. (1997) · ACM SIGMOD Record · doi:10.1145/271074.271084 · Cited by 120
Data mining and the Web: past, present and future
Garofalakis, M., Rastogi, R. R., Seshadri, S., & Shim, K. (1999) · ACM International Workshop on Web Information and Data Management (WIDM) · doi:10.1145/319759.319781 · Cited by 101
Proceedings of the Sixteenth International Conference on Machine Learning (ICML99)
Wu, D., Bennett, K., Cristianini, N., & Shawe-Taylor, J. S. (1999) · Cited by 89
A Mutually Beneficial Integration of Data Mining and Information Extraction
Nahm, U. Y., & Mooney, R. J. (2000) · Cited by 85
Applying data mining techniques for descriptive phrase extraction in digital document collections
Ahonen-Myka, H., Heinonen, O., Klemettinen, M., & Verkamo, A. I. (2002) · IEEE International Forum on Research and Technology Advances in Digital Libraries (ADL) · doi:10.1109/adl.1998.670374 · Cited by 82
Levy, A. Y., & Weld, D. S. (2000) · Artificial Intelligence · doi:10.1016/s0004-3702(00)00013-8 · Cited by 80
An Overview of Issues in Developing Industrial Data Mining and Knowledge Discovery Applications
Piatetsky-Shapiro, G., Brachman, R., Khabaza, T., Klösgen, W., & Simoudis, E. (1996) · Cited by 73
Medical Data Mining on the Internet: Research on a Cancer Information System
Houston, A. L., Chen, H., HUBBARD, S. M., Schatz, B. R., Ng, T. D., Sewell, R. R., & Tolle, K. M. (1999) · Artificial Intelligence Review · doi:10.1023/a:1006548623067 · Cited by 47
From data mining to knowledge discovery: current challenges and future directions
Uthurusamy, R. (1996) · Knowledge Discovery and Data Mining · Cited by 35
Intelligent Agents for Web-based Tasks: An Advice-Taking Approach
SHAVLIK, J. W., & Eliassi‐Rad, T. (1998) · Cited by 23
Large-Scale Mining of Usage Data on Web Sites
Παλιούρας, Γ., Papatheodorou, C., Karkaletsis, V., Tzitziras, P., & Spyropoulos, C. D. (2000) · Cited by 19
Modeling subjective uncertainty in image annotation
Smyth, P., Fayyad, U. M., Burl, M. C., & Perona, P. (1996) · Knowledge Discovery and Data Mining · Cited by 14
Afrati, F., & Kolaitis, P. (1997) · Lecture notes in computer science · doi:10.1007/3-540-62222-5 · Cited by 13
Cited by
Frequent pattern mining: current status and future directions
Han, J., Cheng, H., Xin, D., & Yan, X. (2007) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-006-0059-1 · Cited by 1,380
Social Media Mining: An Introduction
Zafarani, R., Abbasi, M. A., & Liu, H. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9781139088510 · Cited by 592
Xing, W., & Ghorbani, A. (2004) · Proceedings. Second Annual Conference on Communication Networks and Services Research, 2004. · doi:10.1109/dnsr.2004.1344743 · Cited by 572
#Earthquake: Twitter as a Distributed Sensor System
Crooks, A., Croitoru, A., Stefanidis, A., & Radzikowski, J. (2013) · Transactions in GIS · doi:10.1111/j.1467-9671.2012.01359.x · Cited by 469
Exploring entrepreneurial legitimacy in reward-based crowdfunding
Frydrych, D., Bock, A. J., Kinder, T., & Koeck, B. (2014) · Venture Capital · doi:10.1080/13691066.2014.916512 · Cited by 448
Metadata
- Authors
- Kosala, R., & Blockeel, H.
- Venue
- ACM SIGKDD Explorations Newsletter
- Published
- 2000-06
- Volume
- 2
- Issue
- 1
- Pages
- 1-15
- ISSN
- 1931-0145 (ISSN-L)
- 1931-0153
- OpenAlex
- W1989993068
- Cited by
- 1,438
- References
- 87
- FWCI
- 107.02
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.