The DOI matches its Crossref record; 102 of the 102 references it cites were found in a registry or an index.
The Complexities of Agent-Based Modeling Output Analysis
Open accessAbstract
The proliferation of agent-based models (ABMs) in recent decades has motivated model practitioners to improve the transparency, replicability, and trust in results derived from ABMs. The complexity of ABMs has risen in stride with advances in computing power and resources, resulting in larger models with complex interactions and learning and whose outputs are often high-dimensional and require sophisticated analytical approaches. Similarly, the increasing use of data and dynamics in ABMs has further enhanced the complexity of their outputs. In this article, we offer an overview of the state-of-the-art approaches in analyzing and reporting ABM outputs highlighting challenges and outstanding issues. In particular, we examine issues surrounding variance stability (in connection with determination of appropriate number of runs and hypothesis testing), sensitivity analysis, spatio-temporal analysis, visualization, and effective communication of all these to non-technical audiences, such as various stakeholders.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Holland, J. H. (1992) · The MIT Press eBooks · doi:10.7551/mitpress/1090.001.0001 · Cited by 47,002
Design and Analysis of Computer Experiments
Sacks, J., Welch, W. J., Mitchell, T. J., & Wynn, H. P. (1989) · Statistical Science · doi:10.1214/ss/1177012413 · Cited by 6,919
Global Sensitivity Analysis. The Primer
Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., & Tarantola, S. (2007) · doi:10.1002/9780470725184 · Cited by 6,430
Using Effect Size—or Why the P Value Is Not Enough
Sullivan, G. M., & Feinn, R. (2012) · Journal of Graduate Medical Education · doi:10.4300/jgme-d-12-00156.1 · Cited by 5,553
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
Factorial Sampling Plans for Preliminary Computational Experiments
Morris, M. D. (1991) · Technometrics · doi:10.1080/00401706.1991.10484804 · Cited by 4,943
Bayesian t tests for accepting and rejecting the null hypothesis
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009) · Psychonomic Bulletin & Review · doi:10.3758/pbr.16.2.225 · Cited by 3,688
A standard protocol for describing individual-based and agent-based models
Grimm, V., Berger, U., Bastiansen, F., Eliassen, S., Ginot, V., Giske, J., Goss-Custard, J., Grand, T., Heinz, S. K., Huse, G., et al. (2006) · Ecological Modelling · doi:10.1016/j.ecolmodel.2006.04.023 · Cited by 2,981
The ODD protocol: A review and first update
Grimm, V., Berger, U., DeAngelis, D. L., Polhill, J. G., Giske, J., & Railsback, S. F. (2010) · Ecological Modelling · doi:10.1016/j.ecolmodel.2010.08.019 · Cited by 2,440
Death to Kappa: birth of quantity disagreement and allocation disagreement for accuracy assessment
Pontius, R. G., & Millones, M. (2011) · International Journal of Remote Sensing · doi:10.1080/01431161.2011.552923 · Cited by 2,046
Cited by
The Future of Sensitivity Analysis: An essential discipline for systems modeling and policy support
Razavi, S., Jakeman, A., Saltelli, A., Prieur, C., Iooss, B., Borgonovo, E., Plischke, E., Lo Piano, S., Iwanaga, T., Becker, W., et al. (2021) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2020.104954 · Cited by 628
Validation of Agent-Based Models in Economics and Finance
Fagiolo, G., Guerini, M., Lamperti, F., Moneta, A., & Roventini, A. (2019) · Simulation Foundations, Methods and Applications · doi:10.1007/978-3-319-70766-2_31 · Cited by 300
Eight grand challenges in socio-environmental systems modeling
Elsawah, S., Filatova, T., Jakeman, A. J., Kettner, A. J., Zellner, M. L., Athanasiadis, I. N., Hamilton, S. H., Axtell, R. L., Brown, D. G., Gilligan, J. M., et al. (2020) · Socio-Environmental Systems Modelling · doi:10.18174/sesmo.2020a16226 · Cited by 205
Agent-based modeling of environment-migration linkages: a review
Thober, J., Schwarz, N., & Hermans, K. (2018) · Ecology and Society · doi:10.5751/es-10200-230241 · Cited by 79
Agent-Based Modelling in Visitor Management of Protected Areas
Štekerová, K., Zelenka, J., & Kořínek, M. (2022) · Sustainability · doi:10.3390/su141912490 · Cited by 13
Metadata
- Authors
- Lee, J.-S., Filatova, T., Ligmann-Zielinska, A., Hassani-Mahmooei, B., Stonedahl, F., Lorscheid, I., Voinov, A., Polhill, G., Sun, Z., & Parker, D. C.
- Venue
- Journal of Artificial Societies and Social Simulation
- Published
- 2015
- Volume
- 18
- Issue
- 4
- ISSN
- 1460-7425 (ISSN-L)
- OpenAlex
- W2204468525
- Cited by
- 324
- References
- 102
- FWCI
- 33.25
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Complexities of Agent-Based Modeling Output Analysis. https://app.cochat.ai/papers/10.18564/jasss.2897
- The paper
- Lee, J.-S., Filatova, T., Ligmann-Zielinska, A., Hassani-Mahmooei, B., Stonedahl, F., Lorscheid, I., Voinov, A., Polhill, G., Sun, Z., & Parker, D. C. (2015). The Complexities of Agent-Based Modeling Output Analysis. Journal of Artificial Societies and Social Simulation. https://doi.org/10.18564/jasss.2897
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.