Improving Multi-Step Prediction of Learned Time Series Models
Open accessAbstract
Most typical statistical and machine learning approaches to time series modeling optimize a single-step prediction error. In multiple-step simulation, the learned model is iteratively applied, feeding through the previous output as its new input. Any such predictor however, inevitably introduces errors, and these compounding errors change the input distribution for future prediction steps, breaking the train-test i.i.d assumption common in supervised learning. We present an approach that reuses training data to make a no-regret learner robust to errors made during multi-step prediction. Our insight is to formulate the problem as imitation learning; the training data serves as a "demonstrator" by providing corrections for the errors made during multi-step prediction. By this reduction of multi-step time series prediction to imitation learning, we establish theoretically a strong performance guarantee on the relation between training error and the multi-step prediction error. We present experimental results of our method, DaD, and show significant improvement over the traditional approach in two notably different domains, dynamic system modeling and video texture prediction.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
DAgger by Coaching and Data as Demonstrator have been proposed to improve DAgger's learning performance, but they have not been applied in the cloth-like deformable object domain.
“DAgger by Coaching [102] and Data as Demonstrator (DaD) [103] have been proposed to improve learning performance of DAgger, but they have not been applied in the CDO domain.”
From Data-Driven Robotic Manipulation of Cloth-like Deformable Objects: The Present, Challenges and Future Prospects (2023) · cites this paper as [103]
The DATA AS DEMONSTRATOR Algorithm section (p. 2) introduces DaD, and the Conclusion (p. 6) reports improved multi-step prediction. But the reported comparison is with a single-step prediction baseline, not DAgger; the Reduction to imitation learning section (p. 3) instead says DaD draws on DAgger’s theoretical guarantees. The paper does not establish the broader claim that DaD had not been applied in the cloth-like deformable object domain. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · see citing works
Identification and control of dynamical systems using neural networks
Narendra, K. S., & Parthasarathy, K. (1990) · IEEE Transactions on Neural Networks · doi:10.1109/72.80202 · Cited by 8,031 · see citing works
Backpropagation through time: what it does and how to do it
Werbos, P. J. (1990) · Proceedings of the IEEE · doi:10.1109/5.58337 · Cited by 4,879 · see citing works
Random Features for Large-Scale Kernel Machines
Rahimi, A., & Recht, B. (2007) · Cited by 2,783 · see citing works
Predicting time series with support vector machines
Müller, K. R., Smola, A. J., Rätsch, G., Schölkopf, B., Kohlmorgen, J., & Vapnik, V. N. (1997) · Lecture notes in computer science · doi:10.1007/bfb0020283 · Cited by 983 · see citing works
Learning Stable Nonlinear Dynamical Systems With Gaussian Mixture Models
Khansari-Zadeh, S. M., & Billard, A. (2011) · IEEE Transactions on Robotics · doi:10.1109/tro.2011.2159412 · Cited by 771 · see citing works
Chaudhry, R., Ravichandran, A., Hager, G. D., & Vidal, R. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206821 · Cited by 639 · see citing works
Weighted Sums of Random Kitchen Sinks: Replacing minimization with randomization in learning
Rahimi, A., & Recht, B. (2008) · Cited by 520 · see citing works
Search-based structured prediction
Daumé, H., Langford, J. C., & Marcu, D. (2009) · Machine Learning · doi:10.1007/s10994-009-5106-x · Cited by 465 · see citing works
On the Generalization Ability of On-Line Learning Algorithms
Cesa‐Bianchi, N., Conconi, A., & Gentile, C. (2004) · IEEE Transactions on Information Theory · doi:10.1109/tit.2004.833339 · Cited by 426 · see citing works
Gaussian Process Dynamical Models
Wang, J., Hertzmann, A., & Blei, D. M. (2005) · Cited by 418 · see citing works
Efficient Reductions for Imitation Learning
Ross, S., & Bagnell, D. (2018) · doi:10.1184/r1/6554699 · Cited by 366 · see citing works
Sparse Spectrum Gaussian Process Regression
Lázaro-Gredilla, M., Quiñonero-Candela, J., Rasmussen, C. E., & Figueiras-Vidal, A. R. (2010) · doi:10.5555/1756006.1859914 · Cited by 353 · see citing works
Fastfood: approximating kernel expansions in loglinear time
Le, Q. V., Sarlós, T., & Smola, A. J. (2013) · Cited by 337 · see citing works
Learning for control from multiple demonstrations
Coates, A., Abbeel, P., & Ng, A. Y. (2008) · International Conference on Machine Learning · doi:10.1145/1390156.1390175 · Cited by 228 · see citing works
Learning Nonlinear Dynamical Systems Using an EM Algorithm
Ghahramani, Z., & Roweis, S. T. (1998) · Cited by 226 · see citing works
Exploration and apprenticeship learning in reinforcement learning
Abbeel, P., & Ng, A. Y. (2005) · International Conference on Machine Learning · doi:10.1145/1102351.1102352 · Cited by 224 · see citing works
Classifying Video with Kernel Dynamic Textures
Chan, A. B., & Vasconcelos, N. M. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.382996 · Cited by 167 · see citing works
GP-UKF: Unscented kalman filters with Gaussian process prediction and observation models
Ko, J., Klein, D. J., Fox, D., et al. (2007) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2007.4399284 · Cited by 119 · see citing works
A Constraint Generation Approach to Learning Stable Linear Dynamical Systems
Siddiqi, S. M., Boots, B., & Gordon, G. J. (2008) · doi:10.21236/ada480921 · Cited by 115 · see citing works
No-Regret Reductions for Imitation Learning and Structured PredictionPreprint
Ross, S., Gordon, G. J., & Bagnell, J. A. (2010) · arXiv (Cornell University) · Cited by 107 · see citing works
Self-citationChaudhry, R., Ravichandran, A., Hager, G., et al. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvprw.2009.5206821 · Cited by 94 · see citing works
Learning vehicular dynamics, with application to modeling helicopters
Abbeel, P., Ganapathi, V., & Ng, A. Y. (2005) · Cited by 72 · see citing works
Search-based Structured PredictionPreprint
Daumé, H., Langford, J. C., & Marcu, D. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0907.0786 · Cited by 71 · see citing works
Time series prediction by chaotic modeling of nonlinear dynamical systems
Basharat, A., & Shah, M. (2009) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459429 · Cited by 57 · see citing works
On the Generalization Ability of On-line Learning Algorithms
Cesa‐Bianchi, N., Conconi, A., & Gentile, C. (2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.003.0051 · Cited by 52 · see citing works
Dynamical Modeling with Kernels for Nonlinear Time Series Prediction
Ralaivola, L., & d’Alché–Buc, F. (2003) · Cited by 52 · see citing works
Learning nonlinear dynamic models
Langford, J. C., Salakhutdinov, R., & Zhang, T. (2009) · International Conference on Machine Learning · doi:10.1145/1553374.1553451 · Cited by 31 · see citing works
Kernel Predictive Linear Gaussian models for nonlinear stochastic dynamical systems
Wingate, D., & Singh, S. P. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143972 · Cited by 22 · see citing works
Learning first-order Markov models for control
Abbeel, P., & Ng, A. Y. (2004) · Cited by 21 · see citing works
A Kernel Subspace Method by Stochastic Realization for Learning Nonlinear Dynamical Systems
Kawahara, Y., Yairi, T., & Machida, K. (2007) · The MIT Press eBooks · doi:10.7551/mitpress/7503.003.0088 · Cited by 17 · see citing works
Stability Conditions for Online LearnabilityPreprint
Ross, S., & Bagnell, J. A. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1108.3154 · Cited by 10 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 3
- Daum´ e III, H.; Langford, J.; and Marcu, D. 2009. Searchbased structured prediction. Machine learning 75(3):297325.
- Le, Q.; Sarl´ os, T.; and Smola, A. 2013. FastfoodApproximating Kernel Expansions in Loglinear Time. ICML 244252.
- Rahimi, A., and Recht, B. 2007. Random features for largescale kernel machines. NIPS 1-8.
Cited by
Sorted by citations
Deep learning for molecular design—a review of the state of the art
Elton, D. C., Boukouvalas, Z., Fuge, M. D., & Chung, P. W. (2019) · Molecular Systems Design & Engineering · doi:10.1039/c9me00039a · Cited by 683 · see citing works
An algorithmic perspective on imitation learning
Osa, T., Pajarinen, J., Neumann, G., Bagnell, J. A., Abbeel, P., & Peters, J. (2018) · Foundations and Trends in Robotics · doi:10.1561/2300000053 · Cited by 405 · see citing works
Self-citationMachado, M. C., Bellemare, M. G., Talvitie, E., Veness, J., Hausknecht, M., & Bowling, M. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.5699 · Cited by 359 · see citing works
Towards learning-based planning: The nuPlan benchmark for real-world autonomous driving
Karnchanachari, N., Geromichalos, D., Tan, K. S., Li, N., Eriksen, C., Yaghoubi, S., Mehdipour, N., Bernasconi, G., Fong, W. K., Guo, Y., et al. (2024) · 2024 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra57147.2024.10610077 · Cited by 46 · see citing works
“We also implemented a multi-step prediction baseline variant as discussed in [84] and originally proposed in [85] to further address the distribution shift, for the experiments in this work but do not open-source this implementation.”
MethodSequence Multi-task Learning to Forecast Mental Wellbeing from Sparse Self-reported Data
Spathis, D., Servia-Rodriguez, S., Farrahi, K., Mascolo, C., & Rentfrow, J. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330730 · Cited by 30 · see citing works
Kadi, H. A., & Terzić, K. (2023) · Sensors · doi:10.3390/s23052389 · Cited by 19 · see citing works
“DAgger by Coaching [102] and Data as Demonstrator (DaD) [103] have been proposed to improve learning performance of DAgger , but they have not been applied in the CDO domain.”
Method
- Authors
- Venkatraman, A., Hebert, M., & Bagnell, J.
- Affiliations
- Carnegie Mellon University (US)
- ORCID
- Hebert, Martial
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2015-02-21
- Volume
- 29
- Issue
- 1
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2268617045
- Cited by
- 198
- References
- 32
- Citations by year
- 2026: 8 · 2025: 20 · 2024: 23 · 2023: 28 · 2022: 10 · 2021: 18 · 2020: 28 · 2019: 22 · 2018: 22 · 2017: 13 · 2016: 3 · 2015: 3
- FWCI
- 11.42
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
- Funders
- National Science Foundation (1252522)
- Defense Advanced Research Projects Agency
- Topics
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Forecasting Techniques and Applications
- Keywords
- multi-step forecasting, no-regret learning, imitation learning, dynamic system modeling, DAD
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
