The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Learning and Extracting Finite State Automata with Second-Order Recurrent Neural Networks
Abstract
We show that a recurrent, second-order neural network using a real-time, forward training algorithm readily learns to infer small regular grammars from positive and negative string training samples. We present simulations that show the effect of initial conditions, training set size and order, and neural network architecture. All simulations were performed with random initial weight strengths and usually converge after approximately a hundred epochs of training. We discuss a quantization algorithm for dynamically extracting finite state automata during and after training. For a well-trained neural net, the extracted automata constitute an equivalence class of state machines that are reducible to the minimal machine of the inferred grammar. We then show through simulations that many of the neural net state machines are dynamically stable, that is, they correctly classify many long unseen strings. In addition, some of these extracted automata actually outperform the trained neural network for classification of unseen strings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2012) · Encyclopedia of the Sciences of Learning · doi:10.1007/978-1-4419-1428-6_5195 · Cited by 19,370
Elman, J. L. (1990) · Cognitive Science · doi:10.1207/s15516709cog1402_1 · Cited by 11,212
Introduction to automata theory, languages and computation
(1981) · Mathematics and Computers in Simulation · doi:10.1016/0378-4754(81)90068-9 · Cited by 10,699
Introduction to the theory of neural computation
(1994) · Neural Networks · doi:10.1016/0893-6080(94)90014-0 · Cited by 6,401
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369
Introduction to formal language theory
(1980) · Computer Languages · doi:10.1016/0096-0551(80)90068-5 · Cited by 1,254
Progress in supervised neural networks
Hush, D. R., & Horne, B. G. (1993) · IEEE Signal Processing Magazine · doi:10.1109/79.180705 · Cited by 1,247
Syntactic pattern recognition and applications
Fu, K. (1968) · Medical Entomology and Zoology · Cited by 1,015
Inductive Inference: Theory and Methods
Angluin, D., & Smith, C. H. (1983) · ACM Computing Surveys · doi:10.1145/356914.356918 · Cited by 887
Complexity of automaton identification from given data
Gold, E. M. (1978) · Information and Control · doi:10.1016/s0019-9958(78)90562-4 · Cited by 771
Cited by
Maass, W., Natschläger, T., & Markram, H. (2002) · Neural Computation · doi:10.1162/089976602760407955 · Cited by 4,025
Deep learning and its applications to machine health monitoring
Zhao, R., Yan, R., Chen, Z., Mao, K., Wang, P., & Gao, R. X. (2019) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2018.05.050 · Cited by 2,894
Adaptive Computation and Machine Learning
(2012) · Boosting · doi:10.7551/mitpress/8291.003.0026 · Cited by 1,402
Progress in supervised neural networks
Hush, D. R., & Horne, B. G. (1993) · IEEE Signal Processing Magazine · doi:10.1109/79.180705 · Cited by 1,247
Survey and critique of techniques for extracting rules from trained artificial neural networks
Andrews, R., Diederich, J., & Tickle, A. B. (1995) · Knowledge-Based Systems · doi:10.1016/0950-7051(96)81920-4 · Cited by 1,225
Deep Neural Networks and Tabular Data: A Survey
Borisov, V., Leemann, T., Seßler, K., Haug, J., Pawelczyk, M., & Kasneci, G. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3229161 · Cited by 1,025
On the Computational Power of Neural Nets
Siegelmann, H. T., & Sontag, E. D. (1995) · Journal of Computer and System Sciences · doi:10.1006/jcss.1995.1013 · Cited by 692
Exploiting the past and the future in protein secondary structure prediction
Baldi, P., Brunak, S., Frasconi, P., Soda, G., & Pollastri, G. (1999) · Bioinformatics · doi:10.1093/bioinformatics/15.11.937 · Cited by 497
A general framework for adaptive processing of data structures
Frasconi, P., Gori, M., & Sperduti, A. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/72.712151 · Cited by 471
Juang, C.-F. (2002) · IEEE Transactions on Fuzzy Systems · doi:10.1109/91.995118 · Cited by 439
Tickle, A. B., Andrews, R., Golea, M., & Diederich, J. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/72.728352 · Cited by 438
Toward a connectionist model of recursion in human linguistic performance
Christiansen, M. (1999) · Cognitive Science · doi:10.1016/s0364-0213(99)00003-8 · Cited by 406
Noisy Time Series Prediction using Recurrent Neural Networks and Grammatical Inference
Giles, C. L., Lawrence, S., & Tsoi, A. C. (2001) · Machine Learning · doi:10.1023/a:1010884214864 · Cited by 374
Self-citationAnalog computation via neural networks
Siegelmann, H. T., & Sontag, E. D. (1994) · Theoretical Computer Science · doi:10.1016/0304-3975(94)90178-3 · Cited by 355
A Time Series Transformer based method for the rotating machinery fault diagnosis
Jin, Y., Hou, L., & Chen, Y. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.04.111 · Cited by 336
Input-output HMMs for sequence processing
Bengio, Y., & Frasconi, P. (1996) · IEEE Transactions on Neural Networks · doi:10.1109/72.536317 · Cited by 300
Kuntoğlu, M., Aslan, A., Pimenov, D. Y., Usca, Ü. A., Salur, E., Gupta, M. K., Mikolajczyk, T., Giasin, K., Kapłonek, W., & Sharma, S. (2020) · Sensors · doi:10.3390/s21010108 · Cited by 297
Connectionist modelling in psychology: A localist manifesto
Page, M. (2000) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00003356 · Cited by 267
Using neural networks for data mining
Craven, M. W., & Shavlik, J. W. (1997) · Future Generation Computer Systems · doi:10.1016/s0167-739x(97)00022-8 · Cited by 257
Explainable goal-driven agents and robots-a comprehensive review
Sado, F., Loo, C. K., Liew, W. S., Kerzel, M., & Wermter, S. (2023) · ACM Computing Surveys · doi:10.1145/3564240 · Cited by 79
“For example, using quantization algorithms [75], clustering outputs to infer FSAs [162, 229], and recursively testing inputs and outputs [215].”
MethodOne model for the learning of language
Yang, Y., & Piantadosi, S. T. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2021865119 · Cited by 50
Irie, K., Csordás, R., & Schmidhuber, J. (2023) · arXiv · doi:10.48550/arxiv.2310.16076 · Cited by 3
Metadata
- Authors
- Giles, C. L., Miller, C. B., Chen, D., Chen, H. H., Sun, G. Z., & Lee, Y. C.
- Venue
- Neural Computation
- Publisher
- MIT Press - Journals
- Published
- 1992-05
- Volume
- 4
- Issue
- 3
- Pages
- 393-405
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W1982370770
- Cited by
- 392
- References
- 19
- FWCI
- 25.04
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.