Automated reverse engineering of nonlinear dynamical systems
Open accessThe DOI matches its Crossref record; 53 of the 55 references it cites were found in a registry or an index.
Abstract
Complex nonlinear dynamics arise in many fields of science and engineering, but uncovering the underlying differential equations directly from observations poses a challenging task. The ability to symbolically model complex networked systems is key to understanding them, an open problem in many disciplines. Here we introduce for the first time a method that can automatically generate symbolic equations for a nonlinear coupled dynamical system directly from time series data. This method is applicable to any system that can be described using sets of ordinary nonlinear differential equations, and assumes that the (possibly noisy) time series of all variables are observable. Previous automated symbolic modeling approaches of coupled physical systems produced linear models or required a nonlinear model to be provided manually. The advance presented here is made possible by allowing the method to model each (possibly coupled) variable separately, intelligently perturbing and destabilizing the system to extract its less observable characteristics, and automatically simplifying the equations during modeling. We demonstrate this method on four simulated and two real systems spanning mechanics, ecology, and systems biology. Unlike numerical models, symbolic models have explanatory value, suggesting that automated “reverse engineering” approaches for model-free symbolic nonlinear system identification may play an increasing role in our ability to understand progressively more complex systems in the future.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“An automated process for reverse engineering of nonlinear dynamical systems was presented by Bongard & Lipson [227], illustrating how the method could be used for automated modelling in systems biology, including the automatic generation of testable hypotheses.”
From Reverse engineering and identification in systems biology: strategies, perspectives and challenges (2014) · cites this paper as [227]
Citations
Cites
Genetic algorithms in search, optimization, and machine learning
(1989) · Choice Reviews Online · doi:10.5860/choice.27-0936 · Cited by 49,010
Optimization by Simulated Annealing
Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983) · Science · doi:10.1126/science.220.4598.671 · Cited by 45,134
(1990) · Choice Reviews Online · doi:10.5860/choice.27-3338 · Cited by 35,305
Artificial intelligence: a modern approach
(1995) · Choice Reviews Online · doi:10.5860/choice.33-1577 · Cited by 21,974
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728
Hosmer, D. W., & Lemeshow, S. (2000) · doi:10.1002/0471722146 · Cited by 20,302
Genetic Algorithms in Search, Optimization and Machine Learning
Goldberg, D. E., & Robson, D. (1988) · Cited by 18,391
(2005) · Technometrics · doi:10.1198/tech.2005.s303 · Cited by 17,923
System Identification: Theory for the User
Lennart Ljung, A. (1987) · Cited by 17,690
Genetic Programming: On the Programming of Computers by Means of Natural Selection
Koza, J. R. (1992) · Medical Entomology and Zoology · Cited by 13,082
Artificial intelligence: A modern approach
Nilsson, N. J. (1996) · Artificial Intelligence · doi:10.1016/0004-3702(96)00007-0 · Cited by 10,583
System identification—Theory for the user
Sinha, N. K. (1989) · Automatica · doi:10.1016/0005-1098(89)90019-8 · Cited by 9,184
Multilayer feedforward networks are universal approximators
HornikK., StinchcombeM., & WhiteH. (1989) · Neural Networks · doi:10.5555/70405.70408 · Cited by 9,081
Advances in Neural Information Processing Systems 14: Proceedings of the 2001 Conference
(2002) · The MIT Press eBooks · doi:10.7551/mitpress/1120.001.0001 · Cited by 8,890
Strogatz, S. H. (2001) · Nature · doi:10.1038/35065725 · Cited by 8,302
The Origins of Order: Self-Organization and Selection in Evolution
Kauffman, S. A. (1993) · doi:10.1093/oso/9780195079517.001.0001 · Cited by 7,855
Simple mathematical models with very complicated dynamics
May, R. M. (1976) · Nature · doi:10.1038/261459a0 · Cited by 7,001
Draper, N. R., & Smith, H. (1998) · Wiley Series in Probability and Statistics · doi:10.1002/9781118625590 · Cited by 5,024
Fundamentals of Ecology (1953)
Odum, E. P. (2017) · The Future of Nature · doi:10.12987/9780300188479-022 · Cited by 5,006
Universal Approximation Using Radial-Basis-Function Networks
Park, J., & Sandberg, I. W. (1991) · Neural Computation · doi:10.1162/neco.1991.3.2.246 · Cited by 4,112
Gunst, R. F. (1999) · Technometrics · doi:10.1080/00401706.1999.10485680 · Cited by 4,002
Using Bayesian Networks to Analyze Expression Data
Friedman, N., Linial, M., Nachman, I., & Pe'er, D. (2000) · Journal of Computational Biology · doi:10.1089/106652700750050961 · Cited by 3,266
An Introduction to Systems Biology: Design Principles of Biological Circuits
Alon, U. (2006) · doi:10.1201/9781420011432 · Cited by 2,825
Mathematical Physiology: I: Cellular Physiology
(2009) · Interdisciplinary Applied Mathematics · doi:10.1007/978-0-387-75847-3 · Cited by 2,175
Seung, H. S., Opper, M., & Sompolinsky, H. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130417 · Cited by 1,651
Keener, J., & Sneyd, J. (1998) · Interdisciplinary Applied Mathematics · doi:10.1007/b98841 · Cited by 1,529 · cited 1×
“The lac operon system describing lactose metabolism in E. coli was adapted from Keener and Sneyd (47).”
Variations and Fluctuations of the Number of Individuals in Animal Species living together
Volterra, V. (1928) · ICES Journal of Marine Science · doi:10.1093/icesjms/3.1.3 · Cited by 1,496
Superfamilies of Evolved and Designed Networks
Milo, R., Itzkovitz, S., Kashtan, N., Levitt, R., Shen-Orr, S., Ayzenshtat, I., Sheffer, M., & Alon, U. (2004) · Science · doi:10.1126/science.1089167 · Cited by 1,412
The Numerical Analysis of Ordinary Differential Equations; Runge-Kutta and General Linear Methods.
Jackson, K. R., & Butcher, J. C. (1988) · Mathematics of Computation · doi:10.2307/2008600 · Cited by 1,109 · cited 1×
“Without partitioning, we used the standard fourth-order Runge–Kutta method (45) to integrate the differential equations of both the hidden system and candidate models.”
MethodGenomic analysis of regulatory network dynamics reveals large topological changes
Luscombe, N. M., Madan Babu, M., Yu, H., Snyder, M., Teichmann, S. A., & Gerstein, M. (2004) · Nature · doi:10.1038/nature02782 · Cited by 973
Coupled two-way clustering analysis of gene microarray data
Getz, G., Levine, E., & Domany, E. (2000) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.210134797 · Cited by 813
Optimality and evolutionary tuning of the expression level of a protein
Dekel, E., & Alon, U. (2005) · Nature · doi:10.1038/nature03842 · Cited by 766
Genetic Programming III: Darwinian Invention and Problem Solving
Koza, J. R., André, D., Bennett, F. H., & Keane, M. A. (1999) · Medical Entomology and Zoology · Cited by 763
Resilient Machines Through Continuous Self-Modeling
Bongard, J., Zykov, V., & Lipson, H. (2006) · Science · doi:10.1126/science.1133687 · Cited by 709
Self-citationFunctional genomic hypothesis generation and experimentation by a robot scientist
King, R. D., Whelan, K. E., Jones, F. M., Reiser, P. G. K., Bryant, C. H., Muggleton, S. H., Kell, D. B., & Oliver, S. G. (2004) · Nature · doi:10.1038/nature02236 · Cited by 662
Husmeier, D. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg313 · Cited by 530
Genetic Programming III - Darwinian Invention and Problem Solving
Nordin, P. (1999) · Evolutionary Computation · doi:10.1162/evco.1999.7.4.451 · Cited by 501
Schaffer, C. (1993) · Machine Learning · doi:10.1023/a:1022653209073 · Cited by 310
Local properties of Kauffman’s N-k model: A tunably rugged energy landscape
Weinberger, E. D. (1991) · Physical Review A · doi:10.1103/physreva.44.6399 · Cited by 219
DE HOON, M. J. L., IMOTO, S., KOBAYASHI, K., OGASAWARA, N., & MIYANO, S. (2002) · Biocomputing 2003 · doi:10.1142/9789812776303_0003 · Cited by 204
Nonlinear model structure identification using genetic programming
Gray, G. J., Murray-Smith, D. J., Li, Y., Sharman, K. C., & Weinbrenner, T. (1998) · Control Engineering Practice · doi:10.1016/s0967-0661(98)00087-2 · Cited by 160
Foster, J. A. (2001) · Nature Reviews Genetics · doi:10.1038/35076523 · Cited by 133
Nonlinear System Identification Using Coevolution of Models and Tests
Bongard, J. C., & Lipson, H. (2005) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2005.850293 · Cited by 130
Self-citationSome Considerations on the Reason for Bloat
Banzhaf, W., & Langdon, W. B. (2002) · Genetic Programming and Evolvable Machines · doi:10.1023/a:1014548204452 · Cited by 106
Active Coevolutionary Learning of Deterministic Finite Automata
Bongard, J., & Lipson, H. (2005) · Journal of Machine Learning Research · doi:10.5555/1046920.1194900 · Cited by 67
Self-citationGrey-box model identification via evolutionary computing
Tan, K. C., & Li, Y. (2002) · Control Engineering Practice · doi:10.1016/s0967-0661(02)00031-x · Cited by 63
Wikswo, J. P., Prokop, A., Baudenbacher, F., Cliffel, D., Csukas, B., & Velkovsky, M. (2006) · IEE Proceedings - Nanobiotechnology · doi:10.1049/ip-nbt:20050045 · Cited by 51
Optimal adaptive performance and delocalization in NK fitness landscapes
Campos, P. R. A., Adami, C., & Wilke, C. O. (2002) · Physica A Statistical Mechanics and its Applications · doi:10.1016/s0378-4371(01)00572-6 · Cited by 37
Making forecasts for chaotic physical processes.
Danforth, C., & Yorke, J. (2006) · Physical Review Letters · doi:10.1103/physrevlett.96.144102 · Cited by 35
Evolution of mathematical models of chaotic systems based on multiobjective genetic programming
Rodríguez‐Vázquez, K., & Fleming, P. J. (2004) · Knowledge and Information Systems · doi:10.1007/s10115-004-0184-3 · Cited by 26
Evolutionary program for the identification of dynamical systems
Angeline, P. J., & Fogel, D. B. (1997) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.271503 · Cited by 24
McKinney, B. A., Crowe, J. E., Voss, H. U., Crooke, P. S., Barney, N., & Moore, J. H. (2006) · Physical Review E · doi:10.1103/physreve.73.021912 · Cited by 18
Issues in nonlinear model structure identification using genetic programming
Gray, G. J. (1997) · doi:10.1049/cp:19971198 · Cited by 7
Research works printed but not linked: 11
- BA McKinney, JE Crowe, HU Voss, PS Crooke, N Barney, JH Moore Phys Rev E 73, 021912(1–7). (2006).
- Y Baram, RE Yaniv, K Luz J Machine Learning Res 5, 255–291 (2004).
- HW Andrew (University of Plymouth Plymouth UK) pp. 57–62 (1996).
- S Winkler, M Affenzeller, S Wagner Systems Sci 31, 5–14 (2005).
- Y Chen, J Yang, Y Zhang, J Dong Int J Comput Cognit 3, 19–26 (2005).
- J Bongard, H Lipson J Machine Learning Res 6, 1651–1678 (2005).
- M de Hoon S Imoto K Kobayashi N Ogasawara S Miyano (World Scientific Teaneck NJ) pp. 17–28 (2003).
- EO Wilson Consilience (Knopf, New York, 1998).
- CA Schaffer Machine Learning 10, 153–178 (1993).
- R Caruana, S Lawrence, L Giles Advances in Neural Information Processing Systems (MIT Press, Cambridge, MA), pp. 402–408 (2001).
- SA Kauffman The Origins of Order (Oxford Univ Press, New York, 1994).
Cited by
Discovering governing equations from data by sparse identification of nonlinear dynamical systems
Brunton, S. L., Proctor, J. L., & Kutz, J. N. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1517384113 · Cited by 4,796
“A seminal breakthrough by Bongard and Lipson (3) and Schmidt and Lipson (4) has resulted in a new approach to determine the underlying structure of a nonlinear dynamical system from data.”
BackgroundDistilling Free-Form Natural Laws from Experimental Data
Schmidt, M., & Lipson, H. (2009) · Science · doi:10.1126/science.1165893 · Cited by 2,890
Self-citationData-driven discovery of partial differential equations
Rudy, S. H., Brunton, S. L., Proctor, J. L., & Kutz, J. N. (2017) · Science Advances · doi:10.1126/sciadv.1602614 · Cited by 1,557
“In this series of developments, seminal contributions leveraging symbolic regression and an evolutionary algorithm (13, 14) were capable of directly determining nonlinear dynamical system from data.”
BackgroundDeep learning for universal linear embeddings of nonlinear dynamics
Lusch, B., Kutz, J. N., & Brunton, S. L. (2018) · Nature Communications · doi:10.1038/s41467-018-07210-0 · Cited by 1,424
Hidden physics models: Machine learning of nonlinear partial differential equations
Raissi, M., & Karniadakis, G. E. (2018) · Journal of Computational Physics · doi:10.1016/j.jcp.2017.11.039 · Cited by 1,349
“The literature on data-driven discovery of dynamical systems [3] is vast and encompasses equation-free modeling [4], artificial neural networks [5], nonlinear regression [6], empirical dynamic modeling [7,8], modeling emergent behavior [9], automated inference of dynamics [10–12], normal form identification in climate…”
Background
Metadata
- Authors
- Bongard, J., & Lipson, H.
- Affiliations
- Cornell University (US)
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2007-06-12
- Volume
- 104
- Issue
- 24
- Pages
- 9943-9948
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC1891254
- OpenAlex
- W2093828424
- Cited by
- 792
- References
- 55
- Citations by year
- 2026: 53 · 2025: 76 · 2024: 80 · 2023: 96 · 2022: 74 · 2021: 84 · 2020: 64 · 2019: 64 · 2018: 28 · 2017: 20 · 2016: 19 · 2015: 18 · 2014: 16 · 2013: 20 · 2012: 16
- FWCI
- 32.56
- Type
- journal article
- Language
- en
- Open access
- green
- europepmc.org
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Engineering › Control and Systems Engineering › Control Systems and Identification
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Model Reduction and Neural Networks
- Physical Sciences › Mathematics › Modeling and Simulation › Mathematical and Computational Methods
- Keywords
- reverse engineering, nonlinear dynamical systems, symbolic modeling, system identification, time series, ordinary differential equations, coupled dynamical systems, equation simplification, networked systems, systems biology, mechanics
- MeSH terms
- Algorithms, Engineering, Models, Biological, Nonlinear Dynamics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.