The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Hierarchical Document Clustering Using Frequent Itemsets
Open accessAbstract
A major challenge in document clustering is the extremely high dimensionality. For example, the vocabulary for a document set can easily be thousands of words. On the other hand, each document often contains a small fraction of words in the vocabulary. These features require special handlings. Another requirement is hierarchical clustering where clustered documents can be browsed according to the increasing specificity of topics. In this paper, we propose to use the notion of frequent itemsets, which comes from association rule mining, for document clustering. The intuition of our clustering criterion is that each cluster is identified by some common words, called frequent itemsets, for the documents in the cluster. Frequent itemsets are also used to produce a hierarchical topic tree for clusters. By focusing on frequent items, the dimensionality of the document set is drastically reduced. We show that this method outperforms best existing methods in terms of both clustering accuracy and scalability.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Data mining: concepts and techniques
(2012) · Choice Reviews Online · doi:10.5860/choice.49-3305 · Cited by 28,531
(1992) · Cited by 23,431
(1996) · Cited by 20,179
Mining association rules between sets of items in large databases
Agrawal, R., Imieliński, T., & Swami, A. (1993) · Proceedings of the 1993 ACM SIGMOD international conference on Management of data · doi:10.1145/170035.170072 · Cited by 14,907 · cited 1×
“Many algorithms [25] were proposed for computing frequent itemsets, and the most well-known methods are the Apriori [ 5 , 6] and the FP-growth [24] algorithms.”
MethodFinding groups in data: an introduction to cluster analysis
(1991) · Choice Reviews Online · doi:10.5860/choice.28-4558 · Cited by 10,474
An algorithm for suffix stripping
Porter, M. F. (1980) · Program · doi:10.1108/eb046814 · Cited by 8,096
Algorithms for Clustering Data
Sarle, W. S. (1990) · Technometrics · doi:10.1080/00401706.1990.10484648 · Cited by 7,865
Mining frequent patterns without candidate generation
Han, J., Pei, J., & Yin, Y. (2000) · ACM SIGMOD Record · doi:10.1145/335191.335372 · Cited by 6,402
Agrawal, R., & Srikant, R. (2002) · Proceedings of the Eleventh International Conference on Data Engineering · doi:10.1109/icde.1995.380415 · Cited by 5,155
A Comparative Study on Feature Selection in Text Categorization
Yang, Y., & Pedersen, J. (1997) · Cited by 4,790
Data Mining: Concepts and Techniques
Thamaraiselvi, G., & Kaliammal, A. (2014) · Journal of Information and Knowledge · Cited by 4,088
Ankerst, M. (1999) · doi:10.1145/304181.304187 · Cited by 3,982
Mathematical Theory of Communication
(2013) · Encyclopedia of Systems Biology · doi:10.1007/978-1-4419-9863-7_100015 · Cited by 3,290
Mining frequent patterns without candidate generation
Han, J., Pei, J., & Yin, Y. (2000) · Proceedings of the 2000 ACM SIGMOD international conference on Management of data · doi:10.1145/342009.335372 · Cited by 3,216
A Comparison of Document Clustering Techniques
Steinbach, M. S., Karypis, G., & Kumar, V. (2000) · Cited by 2,532
Automatic subspace clustering of high dimensional data for data mining applications
Agrawal, R., Gehrke, J., Gunopulos, D., & Raghavan, P. (1998) · Proceedings of the 1998 ACM SIGMOD international conference on Management of data · doi:10.1145/276304.276314 · Cited by 2,412
Integrating classification and association rule mining
Liu, B., Hsu, W., & Ma, Y. (1998) · Cited by 2,247
Mining high-speed data streams
Domingos, P., & Hulten, G. (2000) · Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/347090.347107 · Cited by 2,185
Mining time-changing data streams
Hulten, G., Spencer, L., & Domingos, P. (2001) · Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/502512.502529 · Cited by 1,748
Rock: A robust clustering algorithm for categorical attributes
Guha, S., Rastogi, R., & Shim, K. (2000) · Information Systems · doi:10.1016/s0306-4379(00)00022-3 · Cited by 1,472
Kosala, R., & Blockeel, H. (2000) · ACM SIGKDD Explorations Newsletter · doi:10.1145/360402.360406 · Cited by 1,438
Scatter/Gather: A Cluster-based Approach to Browsing Large Document Collections
Cutting, D. R., Karger, D. R., Pedersen, J. O., & Tukey, J. W. (2017) · ACM SIGIR Forum · doi:10.1145/3130348.3130362 · Cited by 1,419
Algorithms for association rule mining — a general survey and comparison
Hipp, J., Güntzer, U., & Nakhaeizadeh, G. (2000) · ACM SIGKDD Explorations Newsletter · doi:10.1145/360402.360421 · Cited by 962
Fast algorithm for mining association rules
Agrawal, R. K. (1994) · Very Large Data Bases · Cited by 904
Fast and effective text mining using linear-time document clustering
Larsen, B., & Aone, C. (1999) · Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/312129.312186 · Cited by 857
Hierarchically Classifying Documents Using Very Few Words
Koller, D., & Sahami, M. (1997) · Cited by 854
Guha, S., Mishra, N., Motwani, R., & O'Callaghan, L. (2002) · Proceedings 41st Annual Symposium on Foundations of Computer Science · doi:10.1109/sfcs.2000.892124 · Cited by 623
ROCK: a robust clustering algorithm for categorical attributes
Guha, S., Rastogi, R., & Shim, K. (1999) · Proceedings 15th International Conference on Data Engineering (Cat. No.99CB36337) · doi:10.1109/icde.1999.754967 · Cited by 609
Karypis, G. (2002) · doi:10.21236/ada439508 · Cited by 546
Frequent term-based text clustering
Beil, F. W., Ester, M., & Xu, X. (2002) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/775047.775110 · Cited by 531
Self-citationA Tree Projection Algorithm for Generation of Frequent Item Sets
Agarwal, R. C., Aggarwal, C. C., & Prasad, V. V. V. (2001) · Journal of Parallel and Distributed Computing · doi:10.1006/jpdc.2000.1693 · Cited by 510
Feature Selection for Unbalanced Class Distribution and Naive Bayes
Mladenić, D., & Grobelnik, M. (1999) · Cited by 427
Incremental clustering and dynamic information retrieval
Charikar, M., Chekuri, C., Feder, T., & Motwani, R. (1997) · Proceedings of the twenty-ninth annual ACM symposium on Theory of computing - STOC '97 · doi:10.1145/258533.258657 · Cited by 354
Data mining for hypertext: a tutorial survey
Chakrabarti, S. (2000) · ACM SIGKDD Explorations Newsletter · doi:10.1145/846183.846187 · Cited by 291
Information retrieval: the role of equipment and an assessment of the technology
Berul, L. H. (2024) · doi:10.17918/00005341 · Cited by 289
Incremental Clustering and Dynamic Information Retrieval
Charikar, M., Chekuri, C., Feder, T., & Motwani, R. (2004) · SIAM Journal on Computing · doi:10.1137/s0097539702418498 · Cited by 237
Clustering transactions using large items
Wang, K., Xu, C., & Liu, B. (1999) · Proceedings of the eighth international conference on Information and knowledge management · doi:10.1145/319950.320054 · Cited by 219
Self-citationA fast algorithm for mining association rules
Huang, L., Chen, H., Wang, X., & Chen, G. (2000) · Journal of Computer Science and Technology · doi:10.1007/bf02948845 · Cited by 191
Automatic Document Classification
Borko, H., & Bernick, M. (1963) · Journal of the ACM · doi:10.1145/321160.321165 · Cited by 187
Web Mining Research: A SurveyPreprint
Kosala, R., & Blockeel, H. (2000) · arXiv (Cornell University) · doi:10.48550/arxiv.cs/0011033 · Cited by 173
(1997)
Cited by
Feature Selection for Clustering: A Review
Alelyani, S., Tang, J., & Liu, H. (2018) · Data Clustering · doi:10.1201/9781315373515-2 · Cited by 371
A coarse-to-fine framework to efficiently thwart plagiarism
Zhang, H., & Chow, T. W. S. (2011) · Pattern Recognition · doi:10.1016/j.patcog.2010.08.023 · Cited by 31
Metadata
- Authors
- Fung, B. C. M., Wang, K., & Ester, M.
- Venue
- Proceedings of the 2003 SIAM International Conference on Data Mining
- Published
- 2003-05
- Pages
- 59-70
- OpenAlex
- W1546703457
- Cited by
- 471
- References
- 42
- FWCI
- 13.26
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.