The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
The ODD protocol: A review and first update
Abstract
The ‘ODD’ (Overview, Design concepts, and Details) protocol was published in 2006 to standardize the published descriptions of individual-based and agent-based models (ABMs). The primary objectives of ODD are to make model descriptions more understandable and complete, thereby making ABMs less subject to criticism for being irreproducible. We have systematically evaluated existing uses of the ODD protocol and identified, as expected, parts of ODD needing improvement and clarification. Accordingly, we revise the definition of ODD to clarify aspects of the original version and thereby facilitate future standardization of ABM descriptions. We discuss frequently raised critiques in ODD but also two emerging, and unanticipated, benefits: ODD improves the rigorous formulation of models and helps make the theoretical foundations of large models more visible. Although the protocol was designed for ABMs, it can help with documenting any large, complex model, alleviating some general objections against such models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
On Optimal Use of a Patchy Environment
MacArthur, R. H., & Pianka, E. R. (1966) · The American Naturalist · doi:10.1086/282454 · Cited by 4,593
Natural Selection, the Costs of Reproduction, and a Refinement of Lack's Principle
Williams, G. C. (1966) · The American Naturalist · doi:10.1086/282461 · Cited by 3,276
A standard protocol for describing individual-based and agent-based models
Grimm, V., Berger, U., Bastiansen, F., Eliassen, S., Ginot, V., Giske, J., Goss-Custard, J., Grand, T., Heinz, S. K., Huse, G., et al. (2006) · Ecological Modelling · doi:10.1016/j.ecolmodel.2006.04.023 · Cited by 2,981
Self-citationPattern-Oriented Modeling of Agent-Based Complex Systems: Lessons from Ecology
Grimm, V., Revilla, E., Berger, U., Jeltsch, F., Mooij, W. M., Railsback, S. F., Thulke, H.-H., Weiner, J., Wiegand, T., & DeAngelis, D. L. (2005) · Science · doi:10.1126/science.1116681 · Cited by 2,175
Self-citationIndividual-based Modeling and Ecology
Grimm, V., & Railsback, S. F. (2005) · Princeton University Press eBooks · doi:10.1515/9781400850624 · Cited by 1,940
Self-citationBehavioural reaction norms: animal personality meets individual plasticity
Dingemanse, N. J., Kazem, A. J. N., Réale, D., & Wright, J. (2010) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2009.07.013 · Cited by 1,496
The Role of Time and Energy in Food Preference
Emlen, J. M. (1966) · The American Naturalist · doi:10.1086/282455 · Cited by 1,470
Grimm, V. (1999) · Ecological Modelling · doi:10.1016/s0304-3800(98)00188-4 · Cited by 939
Self-citationThe Swarm Simulation System: A Toolkit for Building Multi-Agent Simulations
Minar, N., Burkhart, R., Langton, C. G., & Askenazi, M. (1996) · Cited by 739
Individual-Based Modeling of Ecological and Evolutionary Processes
DeAngelis, D. L., & Mooij, W. M. (2005) · Annual Review of Ecology, Evolution, and Systematics · doi:10.1146/annurev.ecolsys.36.102003.152644 · Cited by 634
Self-citation
Cited by
Characterising performance of environmental models
Bennett, N. D., Croke, B. F. W., Guariso, G., Guillaume, J. H. A., Hamilton, S. H., Jakeman, A. J., Marsili-Libelli, S., Newham, L. T. H., Norton, J. P., Perrin, C., et al. (2013) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2012.09.011 · Cited by 1,536
A standard protocol for reporting species distribution models
Zurell, D., Franklin, J., König, C., Bouchet, P. J., Dormann, C. F., Elith, J., Fandos, G., Feng, X., Guillera‐Arroita, G., Guisan, A., et al. (2020) · Ecography · doi:10.1111/ecog.04960 · Cited by 945
Grimm, V., Railsback, S. F., Vincenot, C. E., Berger, U., Gallagher, C., DeAngelis, D. L., Edmonds, B., Ge, J., Giske, J., Groeneveld, J., et al. (2020) · Journal of Artificial Societies and Social Simulation · doi:10.18564/jasss.4259 · Cited by 892
Self-citationDescribing human decisions in agent-based models – ODD + D, an extension of the ODD protocol
Müller, B., Bohn, F., Dreßler, G., Groeneveld, J., Klassert, C., Martin, R., Schlüter, M., Schulze, J., Weise, H., & Schwarz, N. (2013) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2013.06.003 · Cited by 542
Modeling Metal Stocks and Flows: A Review of Dynamic Material Flow Analysis Methods
Müller, E., Hilty, L. M., Widmer, R., Schluep, M., & Faulstich, M. (2014) · Environmental Science & Technology · doi:10.1021/es403506a · Cited by 519
Everything you need to know about agent-based modelling and simulation
Macal, C. M. (2016) · Journal of Simulation · doi:10.1057/jos.2016.7 · Cited by 469
What's your move? Movement as a link between personality and spatial dynamics in animal populations
Spiegel, O., Leu, S. T., Bull, C. M., & Sih, A. (2017) · Ecology Letters · doi:10.1111/ele.12708 · Cited by 427
Pattern-oriented modelling: a ‘multi-scope’ for predictive systems ecology
Grimm, V., & Railsback, S. F. (2012) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2011.0180 · Cited by 426
Self-citationAgent-Based Modeling in Public Health: Current Applications and Future Directions
Tracy, M., Cerdá, M., & Keyes, K. M. (2018) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040617-014317 · Cited by 376
NEW HORIZONS FOR MANAGING THE ENVIRONMENT: A REVIEW OF COUPLED SOCIAL‐ECOLOGICAL SYSTEMS MODELING
SCHLÜTER, M., MCALLISTER, R. R. J., ARLINGHAUS, R., BUNNEFELD, N., EISENACK, K., HÖLKER, F., MILNER‐GULLAND, E. J., MÜLLER, B., NICHOLSON, E., QUAAS, M., et al. (2012) · Natural Resource Modeling · doi:10.1111/j.1939-7445.2011.00108.x · Cited by 373
Individual-based models in ecology after four decades
DeAngelis, D. L., & Grimm, V. (2014) · F1000Prime Reports · doi:10.12703/p6-39 · Cited by 360
Self-citationRai, V., & Robinson, S. A. (2015) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2015.04.014 · Cited by 335
The Complexities of Agent-Based Modeling Output Analysis
Lee, J.-S., Filatova, T., Ligmann-Zielinska, A., Hassani-Mahmooei, B., Stonedahl, F., Lorscheid, I., Voinov, A., Polhill, G., Sun, Z., & Parker, D. C. (2015) · Journal of Artificial Societies and Social Simulation · doi:10.18564/jasss.2897 · Cited by 324
Agent‐based models in translational systems biology
An, G., Mi, Q., Dutta‐Moscato, J., & Vodovotz, Y. (2009) · WIREs Systems Biology and Medicine · doi:10.1002/wsbm.45 · Cited by 321
Grimm, V., Augusiak, J., Focks, A., Frank, B. M., Gabsi, F., Johnston, A. S. A., Liu, C., Martin, B. T., Meli, M., Radchuk, V., et al. (2014) · Ecological Modelling · doi:10.1016/j.ecolmodel.2014.01.018 · Cited by 312
Self-citationBecher, M. A., Grimm, V., Thorbek, P., Horn, J., Kennedy, P. J., & Osborne, J. L. (2014) · Journal of Applied Ecology · doi:10.1111/1365-2664.12222 · Cited by 296
Self-citationA Third Wave in the Economics of Climate Change
Farmer, J. D., Hepburn, C., Mealy, P., & Teytelboym, A. (2015) · Environmental and Resource Economics · doi:10.1007/s10640-015-9965-2 · Cited by 283
Strengthening the reporting of empirical simulation studies: Introducing the STRESS guidelines
Monks, T., Currie, C. S. M., Onggo, B. S., Robinson, S., Kunc, M., & Taylor, S. J. E. (2019) · Journal of Simulation · doi:10.1080/17477778.2018.1442155 · Cited by 248
Advancing microbial sciences by individual-based modelling
Hellweger, F. L., Clegg, R. J., Clark, J. R., Plugge, C. M., & Kreft, J.-U. (2016) · Nature Reviews Microbiology · doi:10.1038/nrmicro.2016.62 · Cited by 202
Brotons, L., Aquilué, N., de Cáceres, M., Fortin, M.-J., & Fall, A. (2013) · PLoS ONE · doi:10.1371/journal.pone.0062392 · Cited by 212
The effect of occupancy on perceived control, neutral temperature, and behavioral patterns
Schweiker, M., & Wagner, A. (2016) · Energy and Buildings · doi:10.1016/j.enbuild.2015.10.051 · Cited by 146
An open-data-driven agent-based model to simulate infectious disease outbreaks
Hunter, E., Mac Namee, B., & Kelleher, J. (2018) · PLOS ONE · doi:10.1371/journal.pone.0208775 · Cited by 126
A modular computational framework for medical digital twins
Masison, J., Beezley, J., Mei, Y., Ribeiro, H., Knapp, A. C., Sordo Vieira, L., Adhikari, B., Scindia, Y., Grauer, M., Helba, B., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2024287118 · Cited by 98
An overview of agent-based models in plant biology and ecology
Zhang, B., & DeAngelis, D. L. (2020) · Annals of Botany · doi:10.1093/aob/mcaa043 · Cited by 93
Self-citationThe impact of plug-in behavior on the spatial–temporal flexibility of electric vehicle charging load
Gschwendtner, C., Knoeri, C., & Stephan, A. (2023) · Sustainable Cities and Society · doi:10.1016/j.scs.2022.104263 · Cited by 91
Addressing population health and health inequalities: the role of fundamental causes
Cerdá, M., Tracy, M., Ahern, J., & Galea, S. (2014) · American Journal of Public Health · doi:10.2105/ajph.2014.302055 · Cited by 73
Modelling Tourism in the Galapagos Islands: An Agent-Based Model Approach
Pizzitutti, F., Mena, C. F., & Walsh, S. J. (2014) · Journal of Artificial Societies and Social Simulation · doi:10.18564/jasss.2389 · Cited by 49
“The Agent Based Model 3.1 This ABM is described following the "'Overview – Design concepts - Details" (ODD) protocol proposed by Grimm et al. (2006; 2010).”
InfluentialReducing Urban Violence: A Contrast of Public Health and Criminal Justice Approaches
Cerdá, M., Tracy, M., & Keyes, K. M. (2018) · Epidemiology · doi:10.1097/ede.0000000000000756 · Cited by 45
Daloğlu, I., Nassauer, J. I., Riolo, R., & Scavia, D. (2014) · Ecology and Society · doi:10.5751/es-06597-190312 · Cited by 40
Provenance in modeling and simulation studies — Bridging gaps
Ruscheinski, A., & Uhrmacher, A. (2017) · 2017 Winter Simulation Conference (WSC) · doi:10.1109/wsc.2017.8247839 · Cited by 31
Exploring future scenarios for the global supply chain of tuna
Mullon, C., Guillotreau, P., Galbraith, E. D., Fortilus, J., Chaboud, C., Bopp, L., Aumont, O., & Kaplan, D. (2017) · Deep Sea Research Part II: Topical Studies in Oceanography · doi:10.1016/j.dsr2.2016.08.004 · Cited by 31
Simulating emerging coastal tourism vulnerabilities: an agent-based modelling approach
Student, J., Kramer, M. R., & Steinmann, P. (2020) · Annals of Tourism Research · doi:10.1016/j.annals.2020.103034 · Cited by 31
Simple movement rules result in ideal free distribution of mobile pastoralists
Moritz, M., Hamilton, I. M., Yoak, A. J., Scholte, P., Cronley, J., Maddock, P., & Pi, H. (2015) · Ecological Modelling · doi:10.1016/j.ecolmodel.2015.03.010 · Cited by 27
Describing economic agent-based models—Dahlem ABM documentation guidelines
Wolf, S., Bouchaud, J.-P., Cecconi, F., Cincotti, S., Dawid, H., Gintis, H., van der Hoog, S., Jaegera, C. C., Kovalevsky, D. V., Mandel, A., et al. (2013) · Complexity Economics · doi:10.7564/13-coec12 · Cited by 26
Relating simulation studies by provenance—Developing a family of Wnt signaling models
Budde, K., Smith, J., Wilsdorf, P., Haack, F., & Uhrmacher, A. M. (2021) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1009227 · Cited by 23
“Depending on the application domain, different modeling approaches have their own documentation guidelines [4–6].”
MethodBakhshianlamouki, E., Augustijn, E.-W., Brugnach, M., Voinov, A., & Wijnberg, K. (2025) · Environmental Research Communications · doi:10.1088/2515-7620/adc353 · Cited by 1
Metadata
- Authors
- Grimm, V., Berger, U., DeAngelis, D. L., Polhill, J. G., Giske, J., & Railsback, S. F.
- Venue
- Ecological Modelling
- Publisher
- Elsevier BV
- Published
- 2010-11
- Volume
- 221
- Issue
- 23
- Pages
- 2760-2768
- ISSN
- 0304-3800 (ISSN-L)
- OpenAlex
- W2140862057
- Cited by
- 2,440
- References
- 57
- FWCI
- 124.71
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.