Recognizing Sequences of Sequences
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
The brain's decoding of fast sensory streams is currently impossible to emulate, even approximately, with artificial agents. For example, robust speech recognition is relatively easy for humans but exceptionally difficult for artificial speech recognition systems. In this paper, we propose that recognition can be simplified with an internal model of how sensory input is generated, when formulated in a Bayesian framework. We show that a plausible candidate for an internal or generative model is a hierarchy of 'stable heteroclinic channels'. This model describes continuous dynamics in the environment as a hierarchy of sequences, where slower sequences cause faster sequences. Under this model, online recognition corresponds to the dynamic decoding of causal sequences, giving a representation of the environment with predictive power on several timescales. We illustrate the ensuing decoding or recognition scheme using synthetic sequences of syllables, where syllables are sequences of phonemes and phonemes are sequences of sound-wave modulations. By presenting anomalous stimuli, we find that the resulting recognition dynamics disclose inference at multiple time scales and are reminiscent of neuronal dynamics seen in the real brain.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
A theory of cortical responses.
Friston, K. (2005) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2005.1622 · Cited by 4,578 · cited 1×
“These prediction errors are the difference between the causal states (Equation 2); at any level j , and their prediction from the level above, evaluated at the conditional expectations [18,35].”
BackgroundSelf-citationTowards a neural basis of auditory sentence processing
Friederici, A. D. (2002) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01839-8 · Cited by 2,231 · cited 1×
“may be that these prediction errors are related to electrophysiological responses to violations of phonotactic rules, [44,45].”
BackgroundHierarchical Bayesian inference in the visual cortex
Lee, T. S., & Mumford, D. (2003) · Journal of the Optical Society of America A · doi:10.1364/josaa.20.001434 · Cited by 1,473 · cited 2×
“This is important because it connects a dynamic systems perspective on neuronal dynamics [19,21,22] with the large body of work on the brain as an inference machine [13–18].”
BackgroundA free energy principle for the brain
Friston, K., Kilner, J., & Harrison, L. (2006) · Journal of Physiology-Paris · doi:10.1016/j.jphysparis.2006.10.001 · Cited by 1,369
Self-citationThe novelty P3: an event-related brain potential (ERP) sign of the brain's evaluation of novelty
Friedman, D., Cycowicz, Y. M., & Gaeta, H. (2001) · Neuroscience & Biobehavioral Reviews · doi:10.1016/s0149-7634(01)00019-7 · Cited by 1,282
Object perception as Bayesian inference
Kersten, D., Mamassian, P., & Yuille, A. (2004) · Annual Review of Psychology · doi:10.1146/annurev.psych.55.090902.142005 · Cited by 1,269 · cited 2×
“Here we ask the following question: How does the brain recognize the dynamic and ambiguous causes of noisy sensory input? Based on experimental and theoretical evidence [13–18] we assume the brain is a recognition system that uses an internal model of its environment.”
BackgroundAn information theoretical approach to prefrontal executive function
Koechlin, E., & Summerfield, C. (2007) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2007.04.005 · Cited by 1,183
Cognitive control, hierarchy, and the rostro–caudal organization of the frontal lobes
Badre, D. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.02.004 · Cited by 980
Hierarchical Models in the Brain
Friston, K. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000211 · Cited by 952 · cited 8×
“In biological implementations of the recognition scheme used here [20], these cells encode prediction error.”
BackgroundMethodInfluentialSelf-citationReplay and Time Compression of Recurring Spike Sequences in the Hippocampus
Nádasdy, Z., Hirase, H., Czurkó, A., Csicsvari, J., & Buzsáki, G. (1999) · The Journal of Neuroscience · doi:10.1523/jneurosci.19-21-09497.1999 · Cited by 932 · cited 1×
“Links to computational models Learning, storing, inferring and executing sequences is a key topic in experimental [71–79], and theoretical neurosciences [80– 82]; and robotics [83–86].”
MethodVision as Bayesian inference: analysis by synthesis?
Yuille, A., & Kersten, D. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.05.002 · Cited by 849
A Hierarchy of Time-Scales and the Brain
Kiebel, S. J., Daunizeau, J., & Friston, K. J. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000209 · Cited by 694 · cited 5×
“This was no problem in our previous model, where we modelled a series of simple chirps, with varying amplitude and frequency [6].”
BackgroundMethodInfluentialSelf-citationHierarchical models of behavior and prefrontal function
Botvinick, M. M. (2008) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2008.02.009 · Cited by 532 · cited 1×
“The three areas where this is most evident are auditory processing [12,37,54–56], cognitive control [57–59], and motor control [60].”
BackgroundThe functional anatomy of the MMN: a DCM study of the roving paradigm
Garrido, M. I., Friston, K. J., Kiebel, S. J., Stephan, K. E., Baldeweg, T., & Kilner, J. M. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.05.018 · Cited by 458
Self-citationRepresentation of the temporal envelope of sounds in the human brain
Giraud, A.-L., Lorenzi, C., Ashburner, J., Wable, J., Johnsrude, I., Frackowiak, R., & Kleinschmidt, A. (2000) · Journal of Neurophysiology · doi:10.1152/jn.2000.84.3.1588 · Cited by 353
Redcay, E. (2008) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2007.06.004 · Cited by 349
Estimating the sources of motor errors for adaptation and generalization
Berniker, M., & Kording, K. (2008) · Nature Neuroscience · doi:10.1038/nn.2229 · Cited by 316 · cited 1×
“This requires the ‘dynamic fusion’ of bottom-up sensory input and top-down predictions, Several authors e.g., [83,89–92] use recurrent networks to implement this fusion.”
BackgroundNunez, P. L., & Silberstein, R. B. (2000) · Brain Topography · doi:10.1023/a:1026683200895 · Cited by 307
DEM: A variational treatment of dynamic systems
Friston, K. J., Trujillo-Barreto, N., & Daunizeau, J. (2008) · NeuroImage · doi:10.1016/j.neuroimage.2008.02.054 · Cited by 278 · cited 2×
“These dynamics effectively suppress or explain away prediction error; see [34] for details.”
BackgroundMethodA model of the ventral visual system based on temporal stability and local memory
Wyss, R., König, P., & Verschure, P. F. M. J. (2006) · PLoS Biology · doi:10.1371/journal.pbio.0040120 · Cited by 145
Research works printed but not linked: 69
- D Poeppel, 2008, Speech perception at the interface of neurobiology and linguistics., PhilosTransRSocLond B BiolSci, 363, 1071
- RJ Zatorre, 2002, Structure and function of auditory cortex: music and speech., Trends Cogn Sci, 6, 37, 10.1016/S1364-6613(00)01816-7 · DOI · Paper page
- D Simon, 2006, Brain responses to dynamic facial expressions of pain., Pain, 126, 309, 10.1016/j.pain.2006.08.033 · DOI · Paper page
- JC Thompson, 2007, Common and distinct brain activation to viewing dynamic sequences of face and hand movements., Neuroimage, 37, 966, 10.1016/j.neuroimage.2007.05.058 · DOI
- L Deng, 2006, Structured speech modeling., Ieee Transactions on Audio Speech and Language Processing, 14, 1492, 10.1109/TASL.2006.878265 · DOI
- MA Long, 2008, Using temperature to analyse temporal dynamics in the songbird motor pathway., Nature, 456, 189, 10.1038/nature07448 · DOI · Paper page
- K Sen, 2001, Feature analysis of natural sounds in the songbird auditory forebrain., JNeurophysiol, 86, 1445, 10.1152/jn.2001.86.3.1445 · DOI · Paper page
- MH Davis, 2003, Hierarchical processing in spoken language comprehension., JNeurosci, 23, 3423, 10.1523/JNEUROSCI.23-08-03423.2003 · DOI · Paper page
- T Overath, 2008, Encoding of spectral correlation over time in auditory cortex., JNeurosci, 28, 13268, 10.1523/JNEUROSCI.4596-08.2008 · DOI
- K von Kriegstein, 2008, Task-dependent modulation of medial geniculate body is behaviorally relevant for speech recognition., CurrBiol, 18, 1855, 10.1016/j.cub.2008.10.052 · DOI
- DM Wolpert, 1995, An internal model for sensorimotor integration., Science, 269, 1880, 10.1126/science.7569931 · DOI · Paper page
- KP Kording, 2004, Bayesian integration in sensorimotor learning., Nature, 427, 244, 10.1038/nature02169 · DOI · Paper page
- M Rabinovich, 2008, Neuroscience - Transient dynamics for neural processing., Science, 321, 48, 10.1126/science.1155564 · DOI · Paper page
- T Fukai, 1997, A simple neural network exhibiting selective activation of neuronal ensembles: from winner-take-all to winners-share-all., Neural Comput, 9, 77, 10.1162/neco.1997.9.1.77 · DOI · Paper page
- MI Rabinovich, 2006, Dynamical principles in neuroscience., Reviews of Modern Physics, 78, 1213, 10.1103/RevModPhys.78.1213 · DOI · Paper page
- MI Rabinovich, 2008, Transient cognitive dynamics, metastability, and decision making., Plos Computational Biology, 4, 10.1371/journal.pcbi.1000072 · DOI · Paper page
- VS Afraimovich, 2004, On the origin of reproducible sequential activity in neural circuits., Chaos, 14, 1123, 10.1063/1.1819625 · DOI
- M Rabinovich, 2001, Dynamical encoding by networks of competing neuron groups: Winnerless competition., Physical Review Letters, 8706
- KJ Friston, 1997, Transients, metastability, and neuronal dynamics., Neuroimage, 5, 164, 10.1006/nimg.1997.0259 · DOI · Paper page
- P Varona, 2002, Winnerless competition between sensory neurons generates chaos: A possible mechanism for molluscan hunting behavior., Chaos, 12, 672, 10.1063/1.1498155 · DOI
- VS Afraimovich, 2004, Heteroclinic contours in neural ensembles and the winnerless competition principle., International Journal of Bifurcation and Chaos, 14, 1195, 10.1142/S0218127404009806 · DOI
- M Breakspear, 2003, Modulation of excitatory synaptic coupling facilitates synchronization and complex dynamics in a nonlinear model of neuronal dynamics., Neurocomputing, 52-4, 151, 10.1016/S0925-2312(02)00740-3 · DOI
- D Durstewitz, 2008, Computational significance of transient dynamics in cortical networks., European Journal of Neuroscience, 27, 217, 10.1111/j.1460-9568.2007.05976.x · DOI · Paper page
- DV Buonomano, 2009, State-dependent computations: spatiotemporal processing in cortical networks., NatRevNeurosci, 10, 113
- MV Ivanchenko, 2008, Pacemaker and network mechanisms of rhythm generation: Cooperation and competition., Journal of Theoretical Biology, 253, 452, 10.1016/j.jtbi.2008.04.016 · DOI
- O Creutzfeldt, 1980, Thalamocortical Transformation of Responses to Complex Auditory-Stimuli., Experimental Brain Research, 39, 87, 10.1007/BF00237072 · DOI · Paper page
- X Wang, 2008, Neural coding of temporal information in auditory thalamus and cortex., Neuroscience, 154, 294, 10.1016/j.neuroscience.2008.03.065 · DOI · Paper page
- M Nahum, 2008, Low-level information and high-level perception: The case of speech in noise., Plos Biology, 6, 978, 10.1371/journal.pbio.0060126 · DOI
- M Holmberg, 2007, Speech encoding in a model of peripheral auditory processing: Quantitative assessment by means of automatic speech recognition., Speech Communication, 49, 917, 10.1016/j.specom.2007.05.009 · doi:10.1016/j.specom.2007.05.009
- CJ Sumner, 2002, A revised model of the inner-hair cell and auditory-nerve complex., Journal of the Acoustical Society of America, 111, 2178, 10.1121/1.1453451 · DOI · Paper page
- G Dehaene-Lambertz, 2000, Electrophysiological correlates of phonological processing: A cross-linguistic study., Journal of Cognitive Neuroscience, 12, 635, 10.1162/089892900562390 · DOI · Paper page
- C Eulitz, 2004, Neurobiological evidence for abstract phonological representations in the mental lexicon during speech recognition., Journal of Cognitive Neuroscience, 16, 577, 10.1162/089892904323057308 · DOI · Paper page
- M Friedrich, 2005, Phonotactic knowledge and lexical-semantic processing in one-year-olds: Brain responses to words and nonsense words in picture contexts., Journal of Cognitive Neuroscience, 17, 1785, 10.1162/089892905774589172 · DOI · Paper page
- EF Lau, 2008, A cortical network for semantics: (de)constructing the N400., Nature Reviews Neuroscience, 9, 920, 10.1038/nrn2532 · DOI · Paper page
- E Foulke, 1969, Review of Research on Intelligibility and Comprehension of Accelerated Speech., Psychological Bulletin, 72, 50, 10.1037/h0027575 · DOI · Paper page
- NJ Versfeld, 2002, The relationship between the intelligibility of time-compressed speech and speech in noise in young and elderly listeners., Journal of the Acoustical Society of America, 111, 401, 10.1121/1.1426376 · DOI
- DB Pisoni, 1993, Long-Term-Memory in Speech-Perception - Some New Findings on Talker Variability, Speaking Rate and Perceptual-Learning., Speech Communication, 13, 109, 10.1016/0167-6393(93)90063-Q · DOI · Paper page
- N Vaughan, 2006, Sequencing versus nonsequencing working memory in understanding of rapid speech by older listeners., Journal of the American Academy of Audiology, 17, 506, 10.3766/jaaa.17.7.6 · DOI
- A Budhiraja, 2007, A survey of numerical methods for nonlinear filtering problems., Physica D-Nonlinear Phenomena, 230, 27, 10.1016/j.physd.2006.08.015 · DOI
- K Judd, 2004, Indistinguishable states II - The imperfect model scenario., Physica D-Nonlinear Phenomena, 196, 224
- MA Nowak, 1999, An error limit for the evolution of language., ProcBiolSci, 266, 2131
- S Kumar, 2007, Hierarchical processing of auditory objects in humans., PLoSComputBiol, 3, e100
- A Boemio, 2005, Hierarchical and asymmetric temporal sensitivity in human auditory cortices., NatNeurosci, 8, 389
- SL Denham, 2006, The role of predictive models in the formation of auditory streams., Journal of Physiology-Paris, 100, 154, 10.1016/j.jphysparis.2006.09.012 · DOI
- E Todorov, 2005, From task parameters to motor synergies: A hierarchical framework for approximately-optimal control of redundant manipulators., JRobotSyst, 22, 691
- G Chechik, 2006, Reduction of information redundancy in the ascending auditory pathway., Neuron, 51, 359, 10.1016/j.neuron.2006.06.030 · DOI · Paper page
- I Nelken, 2008, Processing of complex sounds in the auditory system., Current Opinion in Neurobiology, 18, 413, 10.1016/j.conb.2008.08.014 · DOI
- B Maess, 2006, Localizing the distributed language network responsible for the N400 measured by MEG during auditory sentence processing., Brain Research, 1096, 163, 10.1016/j.brainres.2006.04.037 · DOI
- C Van Petten, 2006, Neural localization of semantic context effects in electromagnetic and hemodynamic studies., Brain and Language, 97, 279, 10.1016/j.bandl.2005.11.003 · DOI · Paper page
- JL McClelland, 1986, The Trace Model of Speech-Perception., Cognitive Psychology, 18, 1, 10.1016/0010-0285(86)90015-0 · DOI · Paper page
- D Norris, 1994, Shortlist - A Connectionist Model of Continuous Speech Recognition., Cognition, 52, 189, 10.1016/0010-0277(94)90043-4 · DOI · Paper page
- D Norris, 2008, Shortlist B: A Bayesian model of continuous speech recognition., Psychological Review, 115, 357, 10.1037/0033-295X.115.2.357 · DOI · Paper page
- MM Botvinick, 2006, Short-term memory for serial order: a recurrent neural network model., PsycholRev, 113, 201
- BM Broome, 2006, Encoding and decoding of overlapping odor sequences., Neuron, 51, 467, 10.1016/j.neuron.2006.07.018 · DOI · Paper page
- MH Davis, 2007, Hearing speech sounds: top-down influences on the interface between audition and speech perception., HearRes, 229, 132
- MS Fee, 2004, Neural mechanisms of vocal sequence generation in the songbird., AnnNYAcadSci, 1016, 153, 10.1196/annals.1298.022 · DOI · Paper page
- D Ji, 2008, Firing rate dynamics in the hippocampus induced by trajectory learning., JNeurosci, 28, 4679, 10.1523/JNEUROSCI.4597-07.2008 · DOI · Paper page
- E Koechlin, 2006, Broca's area and the hierarchical organization of human behavior., Neuron, 50, 963, 10.1016/j.neuron.2006.05.017 · DOI · Paper page
- D Kumaran, 2007, Match mismatch processes underlie human hippocampal responses to associative novelty., JNeurosci, 27, 8517, 10.1523/JNEUROSCI.1677-07.2007 · DOI · Paper page
- GS Berns, 1998, A computational model of how the basal ganglia produce sequences., Journal of Cognitive Neuroscience, 10, 108, 10.1162/089892998563815 · DOI · Paper page
- JL Elman, 1990, Finding Structure in Time., Cognitive Science, 14, 179, 10.1207/s15516709cog1402_1 · DOI · Paper page
- O Jensen, 1996, Theta/gamma networks with slow NMDA channels learn sequences and encode episodic memory: Role of NMDA channels in recall., Learning & Memory, 3, 264, 10.1101/lm.3.2-3.264 · DOI · Paper page
- T Kulvicius, 2007, Chained learning architectures in a simple closed-loop behavioural context., Biological Cybernetics, 97, 363, 10.1007/s00422-007-0176-y · DOI
- J Namikawa, 2008, A model for learning to segment temporal sequences, utilizing a mixture of RNN experts together with adaptive variance., Neural Netw, 21, 1466, 10.1016/j.neunet.2008.09.005 · DOI
- J Tani, 2003, Learning to generate articulated behavior through the bottom-up and the top-down interaction processes., Neural Netw, 16, 11, 10.1016/S0893-6080(02)00214-9 · DOI
- A Cleeremans, 1991, Learning the Structure of Event Sequences., Journal of Experimental Psychology-General, 120, 235, 10.1037/0096-3445.120.3.235 · DOI · Paper page
- JT Sakata, 2008, Online Contributions of Auditory Feedback to Neural Activity in Avian Song Control Circuitry., Journal of Neuroscience, 28, 11378, 10.1523/JNEUROSCI.3254-08.2008 · DOI
- J Tani, 2007, On the interactions between top-down anticipation and bottom-up regression., Front Neurorobotics, 1, 2, 10.3389/neuro.12.002.2007 · DOI
- Y Yamashita, 2008, Emergence of Functional Hierarchy in a Multiple Timescale Neural Network Model: A Humanoid Robot Experiment., Plos Computational Biology, 4, 10.1371/journal.pcbi.1000220 · DOI · Paper page
Cited by
The anatomy of language: a review of 100 fMRI studies published in 2009
Price, C. J. (2010) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2010.05444.x · Cited by 1,387
Attention, uncertainty, and free-energy
Feldman, H., & Friston, K. J. (2010) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2010.00215 · Cited by 1,365
Self-citationNeural syntax: cell assemblies, synapsembles, and readers
Buzsáki, G. (2010) · Neuron · doi:10.1016/j.neuron.2010.09.023 · Cited by 1,289
Action understanding and active inference
Friston, K., Mattout, J., & Kilner, J. (2011) · Biological Cybernetics · doi:10.1007/s00422-011-0424-z · Cited by 665
Self-citationEvidence for a hierarchy of predictions and prediction errors in human cortex
Wacongne, C., Labyt, E., van Wassenhove, V., Bekinschtein, T., Naccache, L., & Dehaene, S. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1117807108 · Cited by 560
Metadata
- Authors
- Kiebel, S. J., von Kriegstein, K., Daunizeau, J., & Friston, K. J.
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2009-08-14
- Volume
- 5
- Issue
- 8
- Pages
- e1000464
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- OpenAlex
- W2105714874
- Cited by
- 115
- References
- 21
- FWCI
- 4.57
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.