Goodness-of-Fit Tests and Nonparametric Adaptive Estimation for Spike Train Analysis
Open accessAbstract
When dealing with classical spike train analysis, the practitioner often performs goodness-of-fit tests to test whether the observed process is a Poisson process, for instance, or if it obeys another type of probabilistic model (Yana et al. in Biophys. J. 46(3):323-330, 1984; Brown et al. in Neural Comput. 14(2):325-346, 2002; Pouzat and Chaffiol in Technical report, http://arxiv.org/abs/arXiv:0909.2785, 2009). In doing so, there is a fundamental plug-in step, where the parameters of the supposed underlying model are estimated. The aim of this article is to show that plug-in has sometimes very undesirable effects. We propose a new method based on subsampling to deal with those plug-in issues in the case of the Kolmogorov-Smirnov test of uniformity. The method relies on the plug-in of good estimates of the underlying model that have to be consistent with a controlled rate of convergence. Some nonparametric estimates satisfying those constraints in the Poisson or in the Hawkes framework are highlighted. Moreover, they share adaptive properties that are useful from a practical point of view. We show the performance of those methods on simulated data. We also provide a complete analysis with these tools on single unit activity recorded on a monkey during a sensory-motor task.Electronic Supplementary MaterialThe online version of this article (doi:10.1186/2190-8567-4-3) contains supplementary material.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Using the same dataset for parameter estimation and goodness of fit leads to high bias.
“Reynaud-Bouret et al. [2014] show that the Hawkes process parameters when examined for goodness of fit on the same dataset which was used for parameter estimation leads to a high bias.”
From Keep-Alive Caching for the Hawkes process (2023) · cites this paper as Reynaud-Bouret et al. [2014]
Section 2.1 (page not provided) says that, in most parametric cases, a condition needed for the goodness-of-fit test fails when the estimate is based on the same data used by the test. Its illustration concerns a Kolmogorov–Smirnov test, however; it does not show that reusing data produces high bias in Hawkes process parameter estimates. (LLM, conf=0.84)
Experimental — accuracy not yet measuredThe goodness of fit test is known to have limitations when the same data is used both to estimate the parameters and to compute the KS-statistic.
“since the goodness of fit test is known to have some limitations when the same data is used both to estimate the parameters and to compute the KS- statistic.”
From Keep-Alive Caching for the Hawkes process (2023) · cites this paper as Reynaud-Bouret et al. [2014]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Rizzolatti, G., & Craighero, L. (2004) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.27.070203.144230 · Cited by 7,433 · see citing works
Cattaneo, L., & Rizzolatti, G. (2009) · Archives of Neurology · doi:10.1001/archneurol.2009.41 · Cited by 3,137 · cited 1× · see citing works
“Various studies have exhibited different kinds of correlations between some motor, sensory, or cognitive events in a behaving animal and a variation of the firing rate of specific neurons, before, during or after this event [11, 12].”
Background
Flagged first, then by citations
Research works printed but not linked: 48
- Yana K Takeuchi N Takikawa Y Shinomura M A method for testing an extended Poisson hypothesis of spontaneous quantal transmitter release at neuromuscular junctions Biophys J 1984 4 3 323 330 10.1016/S0006-3495(84)84028-X 6487732 PMC1434949 · DOI
- Brown E Barbieri R Ventura V Kass R Frank L The time rescaling theorem and its application to neural spike train analysis Neural Comput 2002 4 2 325 346 10.1162/08997660252741149 11802915 · DOI
- Pouzat C, Chaffiol A: On goodness of fit tests for models of neuronal spike trains considered as counting processes. Technical report; 2009. arXiv:0909.2785. 0909.2785 · arXiv
- Daley DJ Vere-Jones D An Introduction to the Theory of Point Processes. Volume I: Elementary Theory and Methods Probability and Its Applications (New York) 2003 2 Springer, New York
- Pouzat C Chaffiol A Automatic spike train analysis and report generation. An implementation with R, R2HTML and STAR J Neurosci Methods 2009 4 119 144 10.1016/j.jneumeth.2009.01.037 19473708 · DOI
- Ventura V Carta R Kass R Gettner S Olson C Statistical analysis of temporal evolution in single-neuron firing rates Biostatistics 2002 4 1 20 10.1093/biostatistics/3.1.1 12933620 · DOI
- Georgopoulos A Schwartz A Kettner R Neuronal population coding of movement direction Science 1986 4 1416 1419 10.1126/science.3749885 3749885 · DOI
- Shinomoto S Estimating the firing rate Analysis of Parallel Spike Trains Springer Series in Computational Neuroscience 2010
- Grün S Diesmann M Aertsen A Unitary events analysis Analysis of Parallel Spike Trains Springer Series in Computational Neuroscience 2010
- Riehle A Grammont F Diesmann M Grün S Dynamical changes and temporal precision of synchronised spiking activity in monkey motor cortex during movement preparation J Physiol 2000 4 569 582 11165921
- Krumin M Reutsky I Shoham S Correlation-based analysis and generation of multiple spike trains using Hawkes models with an exogenous input Front Comput Neurosci 2010 4 Article ID 147 PMC2995522 21151360
- Pernice V Staude B Cardanobile S Rotter S How structure determines correlations in neuronal networks PLoS Comput Biol 2012 4 Article ID 031916 PMC3098224 21625580
- Pernice V Staude B Cardanobile S Rotter S Recurrent interactions in spiking networks with arbitrary topology Phys Rev E, Stat Nonlinear Soft Matter Phys 2011 4 Article ID e1002059 22587132
- Chornoboy E Schramm L Karr A Maximum likelihood identification of neural point process systems Biol Cybern 1988 4 265 275 10.1007/BF00332915 3196770 · DOI
- Pillow J Shlens J Paninski L Sher A Litke A Chichilnisky E Simoncelli E Spatio-temporal correlations and visual signalling in a complete neuronal population Nature 2008 4 995 999 10.1038/nature07140 18650810 PMC2684455 · DOI
- Okatan M Wilson M Brown E Analyzing functional connectivity using a network likelihood model of ensemble neural spiking activity Neural Comput 2005 4 1927 1961 10.1162/0899766054322973 15992486 · DOI
- Tuleau-Malot C, Rouis A, Grammont F, Reynaud-Bouret P: Multiple tests based on a Gaussian approximation of the unitary events method. Technical report; 2012. http://hal.archives-ouvertes.fr/hal-00757323. 24708365
- Ogata Y Statistical models for earthquakes occurrences and residual analysis for point processes J Am Stat Assoc 1988 4 401 9 27 10.1080/01621459.1988.10478560 · DOI
- Tuckwell H Introduction to Theoretical Neurobiology. Volume 2: Nonlinear and Stochastic Theories Cambridge Studies in Mathematical Biology 1988
- Barnard G Time intervals between accidents—a note on Maguire, Pearson and Wynn’s paper Biometrika 1953 4 1/2 212 213 10.2307/2333109 · DOI
- Lawless J Statistical Models and Methods for Lifetime Data 1982 Wiley-Interscience, New York
- Stephens M EDF statistics for goodness of fit and some comparisons J Am Stat Assoc 1974 4 347 730 737 10.1080/01621459.1974.10480196 · DOI
- Andersen PK Borgan Ø Gill RD Keiding N Statistical Models Based on Counting Processes Springer Series in Statistics 1993 Springer, New York
- Shimazaki H Shinomoto S Kernel bandwidth optimization in spike rate estimation J Comput Neurosci 2010 4 171 182 10.1007/s10827-009-0180-4 19655238 PMC2940025 · DOI
- Brémaud P Point Processes and Queues: Martingale Dynamics Springer Series in Statistics 1981 Springer, New York
- Shorack GR Wellner JA Empirical Processes with Applications to Statistics Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics 1986 Wiley, New York
- Gerstein G Perkel D Simultaneous recorded trains of action potentials: analysis and functional interpretation Science 1969 4 828 830 10.1126/science.164.3881.828 5767782 · DOI
- Tsybakov AB Introduction to Nonparametric Estimation Springer Series in Statistics 2009 Springer, New York Revised and extended from the 2004 French original, translated by Vladimir Zaiats.
- Goldenshluger A Lepski O Bandwidth selection in kernel density estimation: oracle inequalities and adaptive minimax optimality Ann Stat 2011 4 3 1608 1632 10.1214/11-AOS883 · DOI
- Reynaud-Bouret P Adaptive estimation of the intensity of inhomogeneous Poisson processes via concentration inequalities Probab Theory Relat Fields 2003 4 103 153 10.1007/s00440-003-0259-1 · DOI
- Rydén T An EM algorithm for estimation in Markov-modulated Poisson processes Comput Stat Data Anal 1996 4 4 431 447 10.1016/0167-9473(95)00025-9 · DOI
- Reynaud-Bouret P Rivoirard V Near optimal thresholding estimation of a Poisson intensity on the real line Electron J Stat 2010 4 172 238 10.1214/08-EJS319 · DOI
- Reynaud-Bouret P Rivoirard V Tuleau-Malot C Adaptive density estimation: a curse of support? J Stat Plan Inference 2011 4 115 139 10.1016/j.jspi.2010.05.017 · DOI
- Willett RM Nowak RD Multiscale Poisson intensity and density estimation IEEE Trans Inf Theory 2007 4 9 3171 3187
- Perkel D Gerstein G Moore G Neuronal spike trains and stochastic processes. II. Simultaneous spike trains Biophys J 1967 4 419 440 10.1016/S0006-3495(67)86597-4 4292792 PMC1368069 · DOI
- Gusto G Schbath S FADO: a statistical method to detect favored or avoided distances between occurrences of motifs using the Hawkes’ model Stat Appl Genet Mol Biol 2005 4 Article ID 24 (electronic) 16646842
- Carstensen L Sandelin A Winther O Hansen N Multivariate Hawkes process models of the occurrence of regulatory elements BMC Bioinform 2010 4 Article ID 456 PMC2949889 20828413
- Brillinger D The identification of point process systems Ann Probab 1975 4 6 909 924 10.1214/aop/1176996218 · DOI
- Brillinger D Bryant H Segundo J Identification of synaptic interactions Biol Cybern 1976 4 213 228 10.1007/BF00365087 953079 · DOI
- Brémaud P Massoulié L Stability of nonlinear Hawkes processes Ann Probab 1996 4 3 1563 1588 10.1214/aop/1065725193 · DOI
- Reynaud-Bouret P Schbath S Adaptive estimation for Hawkes processes; application to genome analysis Ann Stat 2010 4 5 2781 2822 10.1214/10-AOS806 · DOI
- Blundell C Heller K Beck J Modelling reciprocating relationships with Hawkes processes Proceedings of NIPS 2012
- Hansen N, Reynaud-Bouret P, Rivoirard V: Lasso and probabilistic inequalities for multivariate point processes. Bernoulli (to appear). arXiv:1208.0570. 1208.0570 · arXiv
- Reynaud-Bouret P Rivoirard V Tuleau-Malot C Inference of functional connectivity in neurosciences via Hawkes processes 1st IEEE Global Conference on Signal and Information Processing 2013
- Aertsen A Gerstein G Habib M Palm G Dynamics of neuronal firing correlation: modulation of “effective connectivity” J Neurophysiol 1989 4 5 900 917 2723733
- Grammont F Riehle A Spike synchronisation and firing rate in a population of motor cortical neurons in relation to movement direction and reaction time Biol Cybern 2003 4 360 373 10.1007/s00422-002-0385-3 12750898 · DOI
- Riehle A Grammont F MacKay A Cancellation of a planned movement in monkey motor cortex NeuroReport 2006 4 3 281 285 10.1097/01.wnr.0000201510.91867.a0 16462598 · DOI
- Ogata Y On Lewis’ simulation method for point processes IEEE Trans Inf Theory 1981 4 23 31 10.1109/TIT.1981.1056305 · DOI
Cited by
Sorted by citations
Keep-Alive Caching for the Hawkes process
Narayana, S., & Kash, I. A. (2023) · arXiv · doi:10.48550/arxiv.2309.03521
- Authors
- Reynaud-Bouret, P., Rivoirard, V., Grammont, F., & Tuleau-Malot, C.
- Affiliations
- Centre National de la Recherche Scientifique (FR)
- Corresponding author
- Reynaud-Bouret, Patricia
- Venue
- The Journal of Mathematical Neuroscience
- Published
- 2014
- Volume
- 4
- Issue
- 1
- Pages
- 3
- ISSN
- 2190-8567 (ISSN-L)
- PMC
- PMC4021619
- OpenAlex
- W2113032265
- Cited by
- 103
- References
- 2
- Citations by year
- 2025: 6 · 2024: 9 · 2023: 2 · 2022: 6 · 2021: 10 · 2020: 33 · 2019: 5 · 2018: 6 · 2017: 10 · 2016: 11 · 2015: 4 · 2013: 1
- FWCI
- 15.20
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- The Journal of Mathematical Neuroscience (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- HAL (submittedVersion, cc-by)
- HAL (submittedVersion, other-oa)
- Funders
- Agence Nationale de la Recherche (ANR 2011 BS01 010 01, 2011 BS01 010 01, ANR-11-BS01-0010, ANR 2011 BS01 010 01 projet Calibration)
- Topics
- Physical Sciences › Mathematics › Applied Mathematics › Point processes and geometric inequalities
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Inference
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Keywords
- spike train analysis, goodness-of-fit tests, subsampling, plug-in estimation, Poisson process, Hawkes process, convergence rate, single-unit activity, sensory-motor task, monkeys
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
