The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Learning and Extracting Finite State Automata with Second-Order Recurrent Neural Networks
Abstract
We show that a recurrent, second-order neural network using a real-time, forward training algorithm readily learns to infer small regular grammars from positive and negative string training samples. We present simulations that show the effect of initial conditions, training set size and order, and neural network architecture. All simulations were performed with random initial weight strengths and usually converge after approximately a hundred epochs of training. We discuss a quantization algorithm for dynamically extracting finite state automata during and after training. For a well-trained neural net, the extracted automata constitute an equivalence class of state machines that are reducible to the minimal machine of the inferred grammar. We then show through simulations that many of the neural net state machines are dynamically stable, that is, they correctly classify many long unseen strings. In addition, some of these extracted automata actually outperform the trained neural network for classification of unseen strings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2012) · Encyclopedia of the Sciences of Learning · doi:10.1007/978-1-4419-1428-6_5195 · Cited by 19,370
Elman, J. L. (1990) · Cognitive Science · doi:10.1207/s15516709cog1402_1 · Cited by 11,212
Introduction to automata theory, languages and computation
(1981) · Mathematics and Computers in Simulation · doi:10.1016/0378-4754(81)90068-9 · Cited by 10,699
Introduction to the theory of neural computation
(1994) · Neural Networks · doi:10.1016/0893-6080(94)90014-0 · Cited by 6,401
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369
Introduction to formal language theory
(1980) · Computer Languages · doi:10.1016/0096-0551(80)90068-5 · Cited by 1,254
Progress in supervised neural networks
Hush, D. R., & Horne, B. G. (1993) · IEEE Signal Processing Magazine · doi:10.1109/79.180705 · Cited by 1,247
Syntactic pattern recognition and applications
Fu, K. (1968) · Medical Entomology and Zoology · Cited by 1,015
Inductive Inference: Theory and Methods
Angluin, D., & Smith, C. H. (1983) · ACM Computing Surveys · doi:10.1145/356914.356918 · Cited by 887
Complexity of automaton identification from given data
Gold, E. M. (1978) · Information and Control · doi:10.1016/s0019-9958(78)90562-4 · Cited by 771
Finite State Automata and Simple Recurrent Networks
Cleeremans, A., Servan-Schreiber, D., & McClelland, J. L. (1989) · Neural Computation · doi:10.1162/neco.1989.1.3.372 · Cited by 458
The induction of dynamical recognizers
Pollack, J. B. (1991) · Machine Learning · doi:10.1007/bf00114845 · Cited by 294
Learning Finite State Machines With Self-Clustering Recurrent Networks
Zeng, Z., Goodman, R. M., & Smyth, P. (1993) · Neural Computation · doi:10.1162/neco.1993.5.6.976 · Cited by 167
Induction of Finite-State Languages Using Second-Order Recurrent Networks
Watrous, R. L., & Kuhn, G. M. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.406 · Cited by 151
Higher Order Recurrent Networks and Grammatical Inference
Giles, C. L., Sun, G.-Z., Chen, H.-H., Lee, Y.-C., & Chen, D. (1989) · neural information processing systems · Cited by 101
Self-citationGiles, C. L., & Omlin, C. W. P. (1993) · Connection Science · doi:10.1080/09540099308915703 · Cited by 99
Self-citationDiscrete recurrent neural networks for grammatical inference
Zeng, Z., Goodman, R. M., & Smyth, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279194 · Cited by 92
Discovering the Structure of a Reactive Environment by Exploration
Mozer, M. C., & Bachrach, J. R. (1990) · Neural Computation · doi:10.1162/neco.1990.2.4.447 · Cited by 37
Heuristics for the extraction of rules from discrete-time recurrent neural networks
Omlin, C. W. P., Giles, C. L., & MILLER, C. B. (2003) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.1992.287212 · Cited by 24
Self-citation
Cited by
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025
Deep learning and its applications to machine health monitoring
Zhao, R., Yan, R., Chen, Z., Mao, K., Wang, P., & Gao, R. X. (2019) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2018.05.050 · Cited by 2,894
Adaptive Computation and Machine Learning
(2012) · Boosting · doi:10.7551/mitpress/8291.003.0026 · Cited by 1,402
Progress in supervised neural networks
Hush, D. R., & Horne, B. G. (1993) · IEEE Signal Processing Magazine · doi:10.1109/79.180705 · Cited by 1,247
Survey and critique of techniques for extracting rules from trained artificial neural networks
Andrews, R., Diederich, J., & Tickle, A. B. (1995) · Knowledge-Based Systems · doi:10.1016/0950-7051(96)81920-4 · Cited by 1,225
Metadata
- Authors
- Giles, C. L., Miller, C. B., Chen, D., Chen, H. H., Sun, G. Z., & Lee, Y. C.
- Venue
- Neural Computation
- Publisher
- MIT Press - Journals
- Published
- 1992-05
- Volume
- 4
- Issue
- 3
- Pages
- 393-405
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W1982370770
- Cited by
- 392
- References
- 19
- FWCI
- 25.04
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.