The DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
On Prediction Using Variable Order Markov Models
Open accessAbstract
This paper is concerned with algorithms for prediction of discrete sequences over a finite alphabet, using variable order Markov models. The class of such algorithms is large and in principle includes any lossless compression algorithm. We focus on six prominent prediction algorithms, including Context Tree Weighting (CTW), Prediction by Partial Match (PPM) and Probabilistic Suffix Trees (PSTs). We discuss the properties of these algorithms and compare their performance using real life sequences from three domains: proteins, English text and music pieces. The comparison is made with respect to prediction quality as measured by the average log-loss. We also compare classification algorithms based on these predictors with respect to a number of large protein classification tasks. Our results indicate that a ``decomposed'' CTW (a variant of the CTW algorithm) and PPM outperform all other algorithms in sequence prediction tasks. Somewhat surprisingly, a different algorithm, which is a modification of the Lempel-Ziv compression algorithm, significantly outperforms all algorithms on the protein classification problems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743
Murzin, A. G., Brenner, S. E., Hubbard, T., & Chothia, C. (1995) · Journal of Molecular Biology · doi:10.1016/s0022-2836(05)80134-2 · Cited by 6,445
Compression of individual sequences via variable-rate coding
Ziv, J., & Lempel, A. (1978) · IEEE Transactions on Information Theory · doi:10.1109/tit.1978.1055934 · Cited by 3,432
A Block-sorting Lossless Data Compression Algorithm
Burrows, M. T., & Wheeler, D. J. (1994) · Cited by 2,333
Maximum Entropy Markov Models for Information Extraction and Segmentation
McCallum, A., Freitag, D., & Pereira, F. C. N. (2000) · Cited by 1,315
Data Compression Using Adaptive Coding and Partial String Matching
Cleary, J., & Witten, I. (1984) · IEEE Transactions on Communications · doi:10.1109/tcom.1984.1096090 · Cited by 1,241
Pfam: A comprehensive database of protein domain families based on seed alignments
Sonnhammer, E. L. L., Eddy, S. R., & Durbin, R. (1997) · Proteins: Structure, Function, and Genetics · doi:10.1002/(sici)1097-0134(199707)28:3<405::aid-prot10>3.0.co;2-l · Cited by 1,221
The context-tree weighting method: basic properties
Willems, F. M. J., Shtarkov, Y. M., & Tjalkens, T. J. (1995) · IEEE Transactions on Information Theory · doi:10.1109/18.382012 · Cited by 919
Universal coding, information, prediction, and estimation
Rissanen, J. (1984) · IEEE Transactions on Information Theory · doi:10.1109/tit.1984.1056936 · Cited by 898
Multi-class protein fold recognition using support vector machines and neural networks
Ding, C. H. Q., & Dubchak, I. (2001) · Bioinformatics · doi:10.1093/bioinformatics/17.4.349 · Cited by 834
Witten, I. H., & Bell, T. C. (1991) · IEEE Transactions on Information Theory · doi:10.1109/18.87000 · Cited by 729
Mismatch string kernels for discriminative protein classification
Leslie, C. S., Eskin, E., Cohen, A., Weston, J., & Noble, W. S. (2004) · Bioinformatics · doi:10.1093/bioinformatics/btg431 · Cited by 561
Park, J., Karplus, K., Barrett, C., Hughey, R., Haussler, D., Hubbard, T., & Chothia, C. (1998) · Journal of Molecular Biology · doi:10.1006/jmbi.1998.2221 · Cited by 559
The power of amnesia: Learning probabilistic automata with variable memory length
Ron, D., Singer, Y., & Tishby, N. (1997) · Machine Learning · doi:10.1007/bf00114008 · Cited by 447
Using the Fisher kernel method to detect remote protein homologies.
Jaakkola, T., Diekhans, M., & Haussler, D. H. (1999) · PubMed · Cited by 408
Within the twilight zone: a sensitive profile-profile comparison tool based on information theory
Yona, G., & Levitt, M. (2002) · Journal of Molecular Biology · doi:10.1006/jmbi.2001.5293 · Cited by 287
Self-citationHierarchic organization of domains in globular proteins
Rose, G. D. (1979) · Journal of Molecular Biology · doi:10.1016/0022-2836(79)90363-2 · Cited by 228
A corpus for the evaluation of lossless compression algorithms
Arnold, R., & Bell, T. (2002) · Proceedings DCC '97. Data Compression Conference · doi:10.1109/dcc.1997.582019 · Cited by 203
Variations on probabilistic suffix trees: statistical modeling and prediction of protein families
Bejerano, G., & Yona, G. (2001) · Bioinformatics · doi:10.1093/bioinformatics/17.1.23 · Cited by 167
Self-citationUsing Compression-Based Language Models for Text Categorization
Teahan, W. J., & Harper, D. J. (2003) · Springer eBooks · doi:10.1007/978-94-017-0171-6_7 · Cited by 139
The origin and evolution of protein superfamilies.
Mo, D. (1976) · PubMed · Cited by 130
A strong version of the redundancy-capacity theorem of universal coding
Merhav, N., & Feder, M. (1995) · IEEE Transactions on Information Theory · doi:10.1109/18.382017 · Cited by 101
On loss functions which minimize to conditional expected values and posterior probabilities
Miller, J. W. V., Goodman, R. M., & Smyth, P. (1993) · IEEE Transactions on Information Theory · doi:10.1109/18.243457 · Cited by 68
Concentration inequalities for the missing mass and for histogram rule error
McAllester, D., & Ortiz, L. E. (2003) · Cited by 57
Semantically Motivated Improvements for PPM Variants
Bunton, S. (1997) · The Computer Journal · doi:10.1093/comjnl/40.2_and_3.76 · Cited by 55
Towards Behaviometric Security Systems: Learning to Identify a Typist
Nisenson, M., Yariv, I., El‐Yaniv, R., et al. (2003) · Lecture notes in computer science · doi:10.1007/978-3-540-39804-2_33 · Cited by 54
Self-citationPart-of-speech tagging using a Variable Memory Markov model
Schütze, H., & Singer, Y. (1994) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/981732.981757 · Cited by 53
Using compression for source-based classification of text
Thaper, N. (2001) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 9
Cited by
COMET: adaptive context-based modeling for ultrafast HIV-1 subtype identification
Struck, D., Lawyer, G., Ternes, A.-M., Schmit, J.-C., & Bercoff, D. P. (2014) · Nucleic Acids Research · doi:10.1093/nar/gku739 · Cited by 381
Forecasting occurrences of activities
Minor, B., & Cook, D. J. (2017) · Pervasive and Mobile Computing · doi:10.1016/j.pmcj.2016.09.010 · Cited by 45
“Other variable-order sequence prediction models have been examined by Begleiter et al. [8].”
Method
Metadata
- Authors
- Begleiter, R., El-Yaniv, R., & Yona, G.
- Venue
- Journal of Artificial Intelligence Research
- Publisher
- AI Access Foundation
- Published
- 2004-12-01
- Volume
- 22
- Pages
- 385-421
- ISSN
- 1076-9757 (ISSN-L)
- OpenAlex
- W2103960658
- Cited by
- 258
- References
- 28
- FWCI
- 3.73
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.