The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.
Neural Controlled Differential Equations for Irregular Time Series
Abstract
Neural ordinary differential equations are an attractive option for modelling temporal dynamics. However, a fundamental issue is that the solution to an ordinary differential equation is determined by its initial condition, and there is no mechanism for adjusting the trajectory based on subsequent observations. Here, we demonstrate how this may be resolved through the well-understood mathematics of \emph{controlled differential equations}. The resulting \emph{neural controlled differential equation} model is directly applicable to the general setting of partially-observed irregularly-sampled multivariate time series, and (unlike previous work on this problem) it may utilise memory-efficient adjoint-based backpropagation even across observations. We demonstrate that our model achieves state-of-the-art performance against similar (ODE or RNN based) models in empirical studies on a range of datasets. Finally we provide theoretical results demonstrating universal approximation, and that our model subsumes alternative ODE models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Recent investigations have demonstrated that the continuous nature of neural ODEs facilitates more stable learning trajectories across sequential tasks, as the smooth manifold structure constrains the degree of interference between task-specific representations.
“Recent investigations have demonstrated that the continuous nature of neural ODEs facilitates more stable learning trajectories across sequential tasks, as the smooth manifold structure constrains the degree of interference between task-specific representations25.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Goldberger, A. L., Amaral, L. A. N., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G., Mietus, J. E., Moody, G. B., Peng, C.-K., & Stanley, H. E. (2000) · Circulation · doi:10.1161/01.cir.101.23.e215 · Cited by 14,809
Flagged first, then by citations
Research works printed but not linked: 71
- K.-i. Funahashi and Y. Nakamura, "Approximation of dynamical systems by continuous time recurrent neural networks," Neural Networks, vol. 6, no. 6, pp. 801 -806, 1993.
- C. Bailer-Jones, D. MacKay, and P. Withers, "A recurrent neural network for modelling dynamical systems," Network: Computation in Neural Systems, vol. 9, pp. 531-47, 1998.
- W. E, "A Proposal on Machine Learning via Dynamical Systems," Commun. Math. Stat., vol. 5, no. 1, pp. 1-11, 2017.
- M. Ciccone, M. Gallieri, J. Masci, C. Osendorfer, and F. Gomez, "NAIS-Net: Stable Deep Networks from Non-Autonomous Differential Equations," in Advances in Neural Information Processing Systems 31, pp. 3025-3035, Curran Associates, Inc., 2018.
- A. Gelman and J. Hill, Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge University Press, 2007.
- Z. Che, S. Purushotham, K. Cho, D. Sontag, and Y. Liu, "Recurrent Neural Networks for Multivariate Time Series with Missing Values," Scientific Reports, vol. 8, no. 1, p. 6085, 2018.
- W. Cao, D. Wang, J. Li, H. Zhou, L. Li, and Y. Li, "BRITS: Bidirectional Recurrent Imputation for Time Series," in Advances in Neural Information Processing Systems 31 (S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, eds.), pp. 6775-6785, Curran Associates, Inc., 2018.
- H. Mei and J. M. Eisner, "The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process," in Advances in Neural Information Processing Systems 30 (I.
- Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, eds.), pp. 6754-6764, Curran Associates, Inc., 2017.
- M. Mozer, D. Kazakov, and R. Lindsey, "Discrete Event, Continuous Time RNNs," arXiv:1710.04110, 2017. · arXiv
- S. C.-X. Li and B. M. Marlin, "A scalable end-to-end Gaussian process adapter for irregularly sampled time series classification," in Advances in Neural Information Processing Systems, pp. 1804-1812, 2016.
- J. Futoma, S. Hariharan, and K. Heller, "Learning to Detect Sepsis with a Multitask Gaussian Process RNN Classifier," in Proceedings of the 34th International Conference on Machine Learning, pp. 1174- 1182, 2017.
- S. N. Shukla and B. Marlin, "Interpolation-Prediction Networks for Irregularly Sampled Time Series," in International Conference on Learning Representations, 2019.
- Y. Rubanova, T. Q. Chen, and D. K. Duvenaud, "Latent Ordinary Differential Equations for Irregularly- Sampled Time Series," in Advances in Neural Information Processing Systems 32, pp. 5320-5330, Curran Associates, Inc., 2019.
- E. De Brouwer, J. Simm, A. Arany, and Y. Moreau, "GRU-ODE-Bayes: Continuous Modeling of Sporadically-Observed Time Series," in Advances in Neural Information Processing Systems 32, pp. 7379-7390, Curran Associates, Inc., 2019.
- R. T. Q. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud, "Neural Ordinary Differential Equations," in Advances in Neural Information Processing Systems 31, pp. 6571-6583, Curran Associates, Inc., 2018.
- T. J. Lyons, "Differential equations driven by rough signals," Revista Matemática Iberoamericana, vol. 14, no. 2, pp. 215-310, 1998.
- T. Lyons, "Rough paths, signatures and the modelling of functions on streams," arXiv:1405.4537, 2014. · arXiv
- B. M. Hambly and T. J. Lyons, "Uniqueness for the signature of a path of bounded variation and the reduced path group," Annals of Mathematics, vol. 171, no. 1, pp. 109-167, 2010.
- I. Chevyrev and H. Oberhauser, "Signature moments to characterize laws of stochastic processes," arXiv:1810.10971, 2018. · arXiv
- T. Lyons, M. Caruana, and T. Levy, Differential equations driven by rough paths. Springer, 2004. École d' Été de Probabilités de Saint-Flour XXXIV -2004.
- P. K. Friz and N. B. Victoir, "Multidimensional stochastic processes as rough paths: theory and applications," Cambridge University Press, 2010.
- I. Perez Arribas, "Derivatives pricing using signature payoffs," arXiv:1809.09466, 2018. · arXiv
- P. Bonnier, P. Kidger, I. Perez Arribas, C. Salvi, and T. Lyons, "Deep Signature Transforms," in Advances in Neural Information Processing Systems, pp. 3099-3109, 2019.
- I. Jordan, P. A. Sokol, and I. M. Park, "Gated recurrent units viewed through the lens of continuous time dynamical systems," arXiv:1906.01005, 2019. · arXiv
- M. B. Giles and N. A. Pierce, "An Introduction to the Adjoint Approach to Design," Flow, Turbulence and Combustion, vol. 65, pp. 393-415, Dec 2000.
- W. W. Hager, "Runge-Kutta methods in optimal control and the transformed adjoint system," Numerische Mathematik, vol. 87, pp. 247-282, Dec 2000.
- A. Gholami, K. Keutzer, and G. Biros, "ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEs," arXiv:1902.10298, 2019. · arXiv
- T. Zhang, Z. Yao, A. Gholami, J. E. Gonzalez, K. Keutzer, M. W. Mahoney, and G. Biros, "ANODEV2: A Coupled Neural ODE Framework," in Advances in Neural Information Processing Systems 32, pp. 5151- 5161, Curran Associates, Inc., 2019.
- A. Bagnall, H. A. Dau, J. Lines, M. Flynn, J. Large, A. Bostrom, P. Southam, and E. Keogh, "The uea multivariate time series classification archive," arXiv:1811.00075, 2018. · arXiv
- M. Reyna, C. Josef, R. Jeter, S. Shashikumar, B. Moody, M. B. Westover, A. Sharma, S. Nemati, and G. Clifford, "Early Prediction of Sepsis from Clinical Data: The PhysioNet/Computing in Cardiology Challenge," Critical Care Medicine, vol. 48, no. 2, pp. 210-217, 2019.
- P. Warden, "Speech commands: A dataset for limited-vocabulary speech recognition," arXiv:1804.03209, 2020. · arXiv
- J. Jia and A. R. Benson, "Neural Jump Stochastic Differential Equations," in Advances in Neural Information Processing Systems 32, pp. 9847-9858, Curran Associates, Inc., 2019.
- C. Cuchiero, W. Khosrawi, and J. Tiechmann, "A generative adversarial network approach to calibration of local stochastic volatility models," arXiv:2005.02505, 2020. · arXiv
- B. Tzen and M. Raginsky, "Theoretical guarantees for sampling and inference in generative models with latent diffusions," COLT, 2019.
- R. Deng, B. Chang, M. Brubaker, G. Mori, and A. Lehrmann, "Modeling Continuous Stochastic Processes with Dynamic Normalizing Flows," arXiv:2002.10516, 2020. · arXiv
- X. Liu, T. Xiao, S. Si, Q. Cao, S. Kumar, and C.-J. Hsieh, "Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noise," arXiv:1906.02355, 2019. · arXiv
- V. Oganesyan, A. Volokhova, and D. Vetrov, "Stochasticity in Neural ODEs: An Empirical Study," arXiv:2002.09779, 2020. · arXiv
- N. Twomey, M. Kozłowski, and R. Santos-Rodríguez, "Neural ODEs with stochastic vector field mixtures," arXiv:1905.09905, 2019. · arXiv
- X. Li, T.-K. L. Wong, R. T. Q. Chen, and D. K. Duvenaud, "Scalable Gradients and Variational Inference for Stochastic Differential Equations," AISTATS, 2020.
- B. Tzen and M. Raginsky, "Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit," arXiv:1905.09883, 2019. · arXiv
- L. Hodgkinson, C. van der Heide, F. Roosta, and M. Mahoney, "Stochastic Normalizing Flows," arXiv:2002.09547, 2020. · arXiv
- I. Chevyrev and A. Kormilitzin, "A primer on the signature method in machine learning," arXiv:1603.03788, 2016. · arXiv
- I. Perez Arribas, G. M. Goodwin, J. R. Geddes, T. Lyons, and K. E. A. Saunders, "A signature- based machine learning model for distinguishing bipolar disorder and borderline personality disorder," Translational Psychiatry, vol. 8, no. 1, p. 274, 2018.
- A. Fermanian, "Embedding and learning with signatures," arXiv:1911.13211, 2019. · arXiv
- J. Morrill, A. Kormilitzin, A. Nevado-Holgado, S. Swaminathan, S. Howison, and T. Lyons, "The Signature-based Model for Early Detection of Sepsis from Electronic Health Records in the Intensive Care Unit," International Conference in Computing in Cardiology, 2019.
- J. Reizenstein, Iterated-integral signatures in machine learning. PhD thesis, University of Warwick, 2019. http://wrap.warwick.ac.uk/131162/.
- M. Moor, M. Horn, C. Bock, K. Borgwardt, and B. Rieck, "Path Imputation Strategies for Signature Models," arXiv:2005.12359, 2020. · arXiv
- J. Morrill, A. Fermanian, P. Kidger, and T. Lyons, "A Generalised Signature Method for Time Series," arXiv:2006.00873, 2020. · arXiv
- S. Liao, T. Lyons, W. Yang, and H. Ni, "Learning stochastic differential equations using RNN with log signature features," arXiv:1908.08286, 2019. · arXiv
- F. J. Király and H. Oberhauser, "Kernels for sequentially ordered data," Journal of Machine Learning Research, 2019.
- C. Toth and H. Oberhauser, "Variational Gaussian Processes with Signature Covariances," ICML 2020, 2020.
- P. Kidger and T. Lyons, "Signatory: differentiable computations of the signature and logsignature transforms, on both CPU and GPU," arXiv:2001.00706, 2020. · arXiv
- A. Quaglino, M. Gallieri, J. Masci, and J. Koutník, "Snode: Spectral discretization of neural odes for system identification," in International Conference on Learning Representations, 2020.
- C. Finlay, J.-H. Jacobsen, L. Nurbekyan, and A. Oberman, "How to train your neural ODE," arXiv:2002.02798, 2020. · arXiv
- S. Massaroli, M. Poli, M. Bin, J. Park, A. Yamashita, and H. Asama, "Stable Neural flows," arXiv:2003.08063, 2020. · arXiv
- S. Massaroli, M. Poli, J. Park, A. Yamashita, and H. Asama, "Dissecting Neural ODEs," arXiv:2002.08071, 2020. · arXiv
- H. Yan, J. Du, V. Y. F. Tan, and J. Feng, "On Robustness of Neural Ordinary Differential Equations," arXiv:1910.05513, 2019. · arXiv
- I. Perez Arribas, C. Salvi, and L. Szpruch, "Sig-SDEs model for quantitative finance," arXiv:2006.00218, 2020. · arXiv
- F. Castell and J. Gaines, "The ordinary differential equation approach to asymptotically efficient schemes for solution of stochastic differential equations," Annales de l'Institut Henri Poincaré. Probabilités et Statistiques, vol. 32, 1996.
- S. Malham and A. Wiese, "Stochastic Lie Group Integrators," SIAM J. Sci. Comput., vol. 30, no. 2, pp. 597-617, 2007.
- L. G. Gyurkó, Numerical methods for approximating solutions to rough differential equations. PhD thesis, University of Oxford, 2008.
- A. Janssen, Order book models, signatures and numerical approximations of rough differential equations. PhD thesis, University of Oxford, 2011.
- H. Boedihardjo, T. Lyons, and D. Yang, "Uniform factorial decay estimates for controlled differential equations," Electronic Communications in Probability, vol. 20, no. 94, 2015.
- J. Foster, Numerical approximations for stochastic differential equations. PhD thesis, University of Oxford, 2020.
- J. Foster, T. Lyons, and H. Oberhauser, "An optimal polynomial approximation of Brownian motion," SIAM J. Numer. Anal., vol. 58, no. 3, pp. 1393-1421, 2020.
- J. M. Varah, "A Lower Bound for the Smallest Singular Value of a Matrix," Linear Algebra and its Applications, vol. 11, no. 1, pp. 3-5, 1975.
- A. Pinkus, "Approximation theory of the MLP model in neural networks," Acta Numer., vol. 8, pp. 143- 195, 1999.
- P. Kidger and T. Lyons, "Universal Approximation with Deep Narrow Networks," arXiv:1905.08539, 2019. · arXiv
- D. Kingma and J. Ba, "Adam: A method for stochastic optimization," ICLR, 2015.
- A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, "PyTorch: An Imperative Style, High-Performance Deep Learning Library," in Advances in Neural Information Processing Systems 32, pp. 8024-8035, Curran Associates, Inc., 2019.
Cited by
Sorted by citations
Jin, M., Koh, H. Y., Wen, Q., Zambon, D., Alippi, C., Webb, G. I., King, I., & Pan, S. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3443141 · Cited by 567
CAPTURE: Disentangling Preference Drift from Memory Poisoning in Personalized LLM Agents
Hossain, S. M. A., Shayoni, R. K., & Morol, M. K. (2026) · arXiv · doi:10.48550/arxiv.2609.02265 · Cited by 3
“It holds a latent belief h t evolving in continuous time between events, in which neither the event embedding nor absolute time appears, so the between-event flow is autonomous and time enters only through the integration interval ∆ t (Chen et al., 2018; Kidger et al., 2020).”
Zhou, S., & Li, Q. (2025) · Scientific Reports · doi:10.1038/s41598-025-31685-9
Metadata
- Authors
- Kidger, P., Morrill, J., Foster, J., & Lyons, T.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2020
- arXiv
- 2005.08926
- References
- 1
- Type
- Preprint
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
