YALE: rapid prototyping for complex data mining tasks
The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
KDD is a complex and demanding task. While a large number of methods has been established for numerous problems, many challenges remain to be solved. New tasks emerge requiring the development of new methods or processing schemes. Like in software development, the development of such solutions demands for careful analysis, specification, implementation, and testing. Rapid prototyping is an approach which allows crucial design decisions as early as possible. A rapid prototyping system should support maximal re-use and innovative combinations of existing methods, as well as simple and quick integration of new ones.This paper describes Yale, a free open-source environment forKDD and machine learning. Yale provides a rich variety of methods whichallows rapid prototyping for new applications and makes costlyre-implementations unnecessary. Additionally, Yale offers extensive functionality for process evaluation and optimization which is a crucial property for any KDD rapid prototyping tool. Following the paradigm of visual programming eases the design of processing schemes. While the graphical user interface supports interactive design, the underlying XML representation enables automated applications after the prototyping phase.After a discussion of the key concepts of Yale, we illustrate the advantages of rapid prototyping for KDD on case studies ranging from data pre-processing to result visualization. These case studies cover tasks like feature engineering, text mining, data stream mining and tracking drifting concepts, ensemble methods and distributed data mining. This variety of applications is also reflected in a broad user base, we counted more than 40,000 downloads during the last twelve months.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Term-weighting approaches in automatic text retrieval
Salton, G., & Buckley, C. (1988) · Information Processing & Management · doi:10.1016/0306-4573(88)90021-0 · Cited by 9,740
A vector space model for automatic indexing
Salton, G., Wong, A., & Yang, C. S. (1975) · Communications of the ACM · doi:10.1145/361219.361220 · Cited by 7,521 · cited 1×
“by transforming the free text in documents into document vectors in the vector space model [17].”
MethodData mining: practical machine learning tools and techniques with Java implementations
Witten, I. H., & Frank, E. (2002) · ACM SIGMOD Record · doi:10.1145/507338.507355 · Cited by 5,137
Learning drifting concepts: Example selection vs. example weighting
Klinkenberg, R. (2004) · Intelligent Data Analysis · doi:10.3233/ida-2004-8305 · Cited by 438
Self-citationDetecting Concept Drift with Support Vector Machines
Klinkenberg, R., & Joachims, T. (2000) · Cited by 422
Self-citationData Mining - Practical Machine Learning Tools and Techniques with JAVA Implementations
Witten, I. H., & Frank, E. (2000) · Cumulative Index of Computer Aided Architectural Design · Cited by 394
Automatic Feature Extraction for Classifying Audio Data
Mierswa, I., & Morik, K. (2005) · Machine Learning · doi:10.1007/s10994-005-5824-7 · Cited by 158
Self-citationBoosting classifiers for drifting concepts
Klinkenberg, R., & Scholz, M. (2006) · Technische Universität Dortmund Eldorado (Technische Universität Dortmund) · doi:10.17877/de290r-14320 · Cited by 103
Self-citationThe MiningMart Approach to Knowledge Discovery in Databases
Morik, K. J., & Scholz, M. (2004) · Springer eBooks · doi:10.1007/978-3-662-07952-2_3 · Cited by 81
Self-citationSoftware infrastructure for natural language processing
Cunningham, H., Humphreys, K., Gaizauskas, R., & Wilks, Y. A. (1997) · Conference on Applied Natural Language Processing · doi:10.3115/974557.974592 · Cited by 71
The Design of Discovery Net: Towards Open Grid Services for Knowledge Discovery
AlSairafi, S., Emmanouil, F.-S., Ghanem, M. M., Giannadakis, N., Guo, Y., Kalaitzopoulos, D. R., Osmond, M., Rowe, A., Syed, J., & Wendel, P. (2003) · The International Journal of High Performance Computing Applications · doi:10.1177/1094342003173003 · Cited by 67
Sampling-based sequential subgroup mining
Scholz, M. (2005) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1081870.1081902 · Cited by 28
Self-citation
Cited by
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750
“The Konstanz Information Miner (KNIME) and RapidMiner [20] are two such systems.”
KEEL: a software tool to assess evolutionary algorithms for data mining problems
Alcalá-Fdez, J., Sánchez, L., García, S., del Jesus, M. J., Ventura, S., Garrell, J. M., Otero, J., Romero, C., Bacardit, J., Rivas, V. M., et al. (2009) · Soft Computing · doi:10.1007/s00500-008-0323-y · Cited by 1,505
Detecting large-scale system problems by mining console logs
Xu, W., Huang, L., Fox, A., Patterson, D., & Jordan, M. I. (2009) · Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles · doi:10.1145/1629575.1629587 · Cited by 1,332
Text Mining Infrastructure in R
Feinerer, I., Hornik, K., & Meyer, D. (2008) · Journal of Statistical Software · doi:10.18637/jss.v025.i05 · Cited by 1,252
Lessmann, S., Baesens, B., Mues, C., & Pietsch, S. (2008) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2008.35 · Cited by 1,226
Metadata
- Authors
- Mierswa, I., Wurst, M., Klinkenberg, R., Scholz, M., & Euler, T.
- Venue
- Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining
- Publisher
- ACM
- Published
- 2006-08-20
- Pages
- 935-940
- OpenAlex
- W2135514714
- Cited by
- 1,010
- References
- 12
- FWCI
- 40.52
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.