Reproducibility and replicability: opportunities and challenges for geospatial research
The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
Abstract
A cornerstone of the scientific method, the ability to reproduce and replicate the results of research has gained widespread attention across the sciences in recent years. A corresponding burst of energy into how to make research more reproducible and replicable has led to numerous innovations. This article outlines some of the opportunities for geospatial researchers to contribute to and learn from the broader reproducibility literature. We review practices developed in related disciplines to improve the reproducibility and replicability of research and outline current efforts to adapt those practices to geospatial analyses. The article then highlights the open questions, opportunities, and potential new directions in geospatial research related to R&R. We stress that the path ahead will likely require a mixture of computational, geospatial, and behavioral research that collectively addresses the many sides of reproducibility and replicability issues.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This is attributed to the model's pretraining on huge amounts of data to achieve good generalizability and, therefore, transferability compared to models learned from and (over)fitted on a small dataset.”
From Assessment of a new GeoAI foundation model for flood inundation mapping (2023) · cites this paper as [17]
Citations
Cites
The FAIR Guiding Principles for scientific data management and stewardship
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.18 · Cited by 18,423
Star, S. L., & Griesemer, J. R. (1989) · Social Studies of Science · doi:10.1177/030631289019003001 · Cited by 10,847
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702
1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209
The Sociology of Science: Theoretical and Empirical Investigations
Kaplan, N., Merton, R. K., & Storer, N. W. (1974) · Technology and Culture · doi:10.2307/3102264 · Cited by 4,005
The Fourth Paradigm: Data-Intensive Scientific Discovery
Buchan, I. E., Winn, J. M., & Bishop, C. (2009) · Cited by 1,439
Annals of the Association of American Geographers
(1921) · Geographical Review · doi:10.2307/208271 · Cited by 1,423
An introduction to Docker for reproducible research
Boettiger, C. (2015) · ACM SIGOPS Operating Systems Review · doi:10.1145/2723872.2723882 · Cited by 1,059
Research works printed but not linked: 6
- Barba, L. 2018. Terminologies for reproducible research. arXiv preprint: 1802.03311 . · arXiv
- Bollen K., 2015, Behavioral, and Economic Sciences
- Moen K., 2020, Cognitive Research, 5, 20
- Richardson D., 2019, ARC News, 41, 2
- Sui D., Annals of the Association of American Geographers
- Wilson J., Annals of the Association of American Geographers
Cited by
Replication across space and time must be weak in the social and environmental sciences
Goodchild, M. F., & Li, W. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2015759118 · Cited by 157
“Methods of spatial analysis are today implemented in geographic information systems (GISs) and in many packages of cartographic or statistical software, and issues of replicability and reproducibility in spatial analysis have been addressed in recent papers (14, 15).”
MethodSelf-citationGeoGPT: An assistant for understanding and processing geospatial tasks
Zhang, Y., Cheng, W., He, Z., & Yu, W. (2024) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2024.103976 · Cited by 328
Modeling and Analyzing Urban Networks and Amenities With OSMnx
Boeing, G. (2025) · Geographical Analysis · doi:10.1111/gean.70009 · Cited by 235
Big data in Earth science: Emerging practice and promise
Vance, T. C., Huang, T., & Butler, K. A. (2024) · Science · doi:10.1126/science.adh9607 · Cited by 143
“Reproducibility is achieved when the original data and computational methods produce the same scientific results, whereas replicability occurs when the same scientific conclusions are reached using new data ( 45 ).”
MethodUrban Visual Intelligence: Studying Cities with Artificial Intelligence and Street-Level Imagery
Zhang, F., Miranda, A. S., Duarte, F., Vale, L. J., Hack, G. A., Chen, M., Liu, Y., Batty, M., & Ratti, C. (2024) · Annals of the American Association of Geographers · doi:10.1080/24694452.2024.2313515 · Cited by 193
“Such spatial variability often limits the ability of models to make consistent predictions across different locations (Goodchild et al. 2020; Goodchild and Li 2021; Kedron et al. 2021; W. Li, Hsu, and Hu 2021).”
Metadata
- Authors
- Kedron, P., Li, W., Fotheringham, S., & Goodchild, M.
- Affiliations
- Arizona State University (US)
- School of Geographical Sciences and Urban Planning, Arizona State University, Tempe, AZ, USA
- University of California, Santa Barbara (US)
- Corresponding author
- Kedron, Peter
- Venue
- International Journal of Geographical Information Science
- Publisher
- Informa UK Limited
- Published
- 2020-08-04
- Volume
- 35
- Issue
- 3
- Pages
- 427-445
- ISSN
- 0269-3798 (ISSN-L)
- 1365-8816
- 1362-3087
- OpenAlex
- W3046703890
- Semantic Scholar
- 99c88dc4c0e15f8cb3bf55ec683c02f548c3a8ae
- Cited by
- 108
- References
- 67
- Citations by year
- 2026: 21 · 2025: 22 · 2024: 21 · 2023: 18 · 2022: 15 · 2021: 9 · 2020: 2
- FWCI
- 10.91
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- International Journal of Geographical Information Systems (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Scientific Computing and Data Management
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- reproducibility, replicability, geospatial research, research practices, open science, computational reproducibility
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Reproducibility and replicability: opportunities and challenges for geospatial research. https://app.cochat.ai/papers/10.1080/13658816.2020.1802032
- The paper
- Kedron, P., Li, W., Fotheringham, S., & Goodchild, M. (2021). Reproducibility and replicability: opportunities and challenges for geospatial research. International Journal of Geographical Information Science. https://doi.org/10.1080/13658816.2020.1802032
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.