The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
The effects of small sample size and sample bias on threshold selection and accuracy assessment of species distribution models
Abstract
Species distribution models are used for a range of ecological and evolutionary questions, but often are constructed from few and/or biased species occurrence records. Recent work has shown that the presence‐only model Maxent performs well with small sample sizes. While the apparent accuracy of such models with small samples has been studied, less emphasis has been placed on the effect of small or biased species records on the secondary modeling steps, specifically accuracy assessment and threshold selection, particularly with profile (presence‐only) modeling techniques. When testing the effects of small sample sizes on distribution models, accuracy assessment has generally been conducted with complete species occurrence data, rather than similarly limited (e.g. few or biased) test data. Likewise, selection of a probability threshold – a selection of probability that classifies a model into discrete areas of presences and absences – has also generally been conducted with complete data. In this study we subsampled distribution data for an endangered rodent across multiple years to assess the effects of different sample sizes and types of bias on threshold selection, and examine the differences between apparent and actual accuracy of the models. Although some previously recommended threshold selection techniques showed little difference in threshold selection, the most commonly used methods performed poorly. Apparent model accuracy calculated from limited data was much higher than true model accuracy, but the true model accuracy was lower than it could have been with a more optimal threshold. That is, models with thresholds and accuracy calculated from biased and limited data had inflated reported accuracy, but were less accurate than they could have been if better data on species distribution were available and an optimal threshold were used.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Maximum entropy modeling of species geographic distributions
Phillips, S. J., Anderson, R. P., & Schapire, R. E. (2006) · Ecological Modelling · doi:10.1016/j.ecolmodel.2005.03.026 · Cited by 15,396
Novel methods improve prediction of species’ distributions from occurrence data
Elith*, J., H. Graham*, C., P. Anderson, R., Dudík, M., Ferrier, S., Guisan, A., J. Hijmans, R., Huettmann, F., R. Leathwick, J., Lehmann, A., et al. (2006) · Ecography · doi:10.1111/j.2006.0906-7590.04596.x · Cited by 9,088
A review of methods for the assessment of prediction errors in conservation presence/absence models
FIELDING, A. H., & BELL, J. F. (1997) · Environmental Conservation · doi:10.1017/s0376892997000088 · Cited by 6,956
Species Distribution Models: Ecological Explanation and Prediction Across Space and Time
Elith, J., & Leathwick, J. R. (2009) · Annual Review of Ecology, Evolution, and Systematics · doi:10.1146/annurev.ecolsys.110308.120159 · Cited by 6,707
AUC: A misleading measure of the performance of predictive distribution models
Lobo, J. M., Jiménez‐Valverde, A., & Real, R. (2008) · Global Ecology and Biogeography · doi:10.1111/j.1466-8238.2007.00358.x · Cited by 3,499
Pearson, R. G., Raxworthy, C. J., Nakamura, M., & Townsend Peterson, A. (2007) · Journal of Biogeography · doi:10.1111/j.1365-2699.2006.01594.x · Cited by 3,138
Phillips, S. J., Dudík, M., Elith, J., Graham, C. H., Lehmann, A., Leathwick, J., & Ferrier, S. (2009) · Ecological Applications · doi:10.1890/07-2153.1 · Cited by 2,996
Selecting thresholds of occurrence in the prediction of species distributions
Liu, C., Berry, P. M., Dawson, T. P., & Pearson, R. G. (2005) · Ecography · doi:10.1111/j.0906-7590.2005.03957.x · Cited by 2,701
Hernandez, P. A., Graham, C. H., Master, L. L., & Albert, D. L. (2006) · Ecography · doi:10.1111/j.0906-7590.2006.04700.x · Cited by 2,534
Evaluating presence–absence models in ecology: the need to account for prevalence
Manel, S., Williams, H. C., & Ormerod, S. J. (2001) · Journal of Applied Ecology · doi:10.1046/j.1365-2664.2001.00647.x · Cited by 1,759
Threshold criteria for conversion of probability of species presence to either–or presence–absence
Jiménez-Valverde, A., & Lobo, J. M. (2007) · Acta Oecologica · doi:10.1016/j.actao.2007.02.001 · Cited by 892
Patterns and uncertainties of species' range shifts under climate change
Thuiller, W. (2004) · Global Change Biology · doi:10.1111/j.1365-2486.2004.00859.x · Cited by 873
Graham, C. H., Ron, S. R., Santos, J. C., Schneider, C. J., & Moritz, C. (2004) · Evolution · doi:10.1111/j.0014-3820.2004.tb00461.x · Cited by 627
Freeman, E. A., & Moisen, G. G. (2008) · Ecological Modelling · doi:10.1016/j.ecolmodel.2008.05.015 · Cited by 578
Aligning Conservation Priorities Across Taxa in Madagascar with High-Resolution Planning Tools
Kremen, C., Cameron, A., Moilanen, A., Phillips, S. J., Thomas, C. D., Beentje, H., Dransfield, J., Fisher, B. L., Glaw, F., Good, T. C., et al. (2008) · Science · doi:10.1126/science.1155193 · Cited by 561
Avoiding Pitfalls of Using Species Distribution Models in Conservation Planning
LOISELLE, B. A., HOWELL, C. A., GRAHAM, C. H., GOERCK, J. M., BROOKS, T., SMITH, K. G., & WILLIAMS, P. H. (2003) · Conservation Biology · doi:10.1111/j.1523-1739.2003.00233.x · Cited by 517
PresenceAbsence : An R Package for Presence Absence Analysis
Freeman, E. A., & Moisen, G. (2008) · Journal of Statistical Software · doi:10.18637/jss.v023.i11 · Cited by 461
Manel, S., Dias, J.-M., & Ormerod, S. J. (1999) · Ecological Modelling · doi:10.1016/s0304-3800(99)00113-1 · Cited by 435
Raxworthy, C. J., Ingram, C. M., Rabibisoa, N., & Pearson, R. G. (2007) · Systematic Biology · doi:10.1080/10635150701775111 · Cited by 433
Huntley, B., Berry, P. M., Cramer, W., & McDonald, A. P. (1995) · Journal of Biogeography · doi:10.2307/2845830 · Cited by 405
Historical bias in biodiversity inventories affects the observed environmental niche of the species
Hortal, J., Jiménez‐Valverde, A., Gómez, J. F., Lobo, J. M., & Baselga, A. (2008) · Oikos · doi:10.1111/j.0030-1299.2008.16434.x · Cited by 388
A comparison of methods for mapping species ranges and species richness
Graham, C. H., & Hijmans, R. J. (2006) · Global Ecology and Biogeography · doi:10.1111/j.1466-8238.2006.00257.x · Cited by 377
Wisz, M. S., & Guisan, A. (2009) · BMC Ecology · doi:10.1186/1472-6785-9-8 · Cited by 338
Papeş, M., & Gaubert, P. (2007) · Diversity and Distributions · doi:10.1111/j.1472-4642.2007.00392.x · Cited by 309
Predicting the potential distribution of plant species in an alpine environment
Guisan, A., Theurillat, J., & Kienast, F. (1998) · Journal of Vegetation Science · doi:10.2307/3237224 · Cited by 293
Thorn, J. S., Nijman, V., Smith, D., & Nekaris, K. A. I. (2009) · Diversity and Distributions · doi:10.1111/j.1472-4642.2008.00535.x · Cited by 286
Logit Models from Economics and Other Fields
Cramer, J. S. (2003) · Cambridge University Press eBooks · doi:10.1017/cbo9780511615412 · Cited by 285
Modelling the potential geographic distribution of invasive ant species in New Zealand
Ward, D. F. (2007) · Biological Invasions · doi:10.1007/s10530-006-9072-y · Cited by 233
A Comparison of C/B Ratios from Studies Using Receiver Operating Characteristic Curve Analysis
Cantor, S. B., Sun, C. C., Tortolero-Luna, G., Richards-Kortum, R., & Follen, M. (1999) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00075-x · Cited by 218
Spatial analysis improves species distribution modelling during range expansion
De Marco, P., Diniz-Filho, J. A. F., & Bini, L. M. (2008) · Biology Letters · doi:10.1098/rsbl.2008.0210 · Cited by 174
Equilibrium modeling of alpine plant distribution: how far can we go?
Guisan, A., & Theurillat, J.-P. (2000) · Phytocoenologia · doi:10.1127/phyto/30/2000/353 · Cited by 172
Waltari, E., & Guralnick, R. (2008) · Journal of Biogeography · doi:10.1111/j.1365-2699.2008.01959.x · Cited by 154
Dealing with uncertain absences in habitat modelling: a case study of a rare ground‐dwelling parrot
Gibson, L. A., Barrett, B., & Burbidge, A. H. (2007) · Diversity and Distributions · doi:10.1111/j.1472-4642.2007.00365.x · Cited by 129
DeMatteo, K. E., & Loiselle, B. A. (2008) · Biological Conservation · doi:10.1016/j.biocon.2008.07.010 · Cited by 84
Brown, K. A., Spector, S., & Wu, W. (2008) · Journal of Applied Ecology · doi:10.1111/j.1365-2664.2008.01550.x · Cited by 77
Cited by
Selecting thresholds for the prediction of species occurrence with presence‐only data
Liu, C., White, M., & Newell, G. (2013) · Journal of Biogeography · doi:10.1111/jbi.12058 · Cited by 1,613
Minimum required number of specimen records to develop accurate species distribution models
van Proosdij, A. S. J., Sosef, M. S. M., Wieringa, J. J., & Raes, N. (2016) · Ecography · doi:10.1111/ecog.01509 · Cited by 791
On the selection of thresholds for predicting species occurrence with presence‐only data
Liu, C., Newell, G., & White, M. (2016) · Ecology and Evolution · doi:10.1002/ece3.1878 · Cited by 727
Shcheglovitova, M., & Anderson, R. P. (2013) · Ecological Modelling · doi:10.1016/j.ecolmodel.2013.08.011 · Cited by 631
Overcoming limitations of modelling rare species by using ensembles of small models
Breiner, F. T., Guisan, A., Bergamini, A., & Nobis, M. P. (2015) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12403 · Cited by 541
Metadata
- Authors
- Bean, W. T., Stafford, R., & Brashares, J. S.
- Venue
- Ecography
- Publisher
- Wiley
- Published
- 2012-03
- Volume
- 35
- Issue
- 3
- Pages
- 250-258
- ISSN
- 0906-7590 (ISSN-L)
- 1600-0587
- OpenAlex
- W2040247202
- Cited by
- 303
- References
- 37
- FWCI
- 7.06
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.