The DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index.
A review of evidence about use and performance of species distribution modelling ensembles like BIOMOD
Open accessAbstract
Aim: The idea of combining predictions from different models into an ensemble has gained considerable popularity in species distribution modelling, partly due to free and comprehensive software such as the R package BIOMOD. However, despite proliferation of ensemble models, we lack oversight of how and where they are used for modelling distributions, and how well they perform. Here, we present such an overview. Location: Global. Methods: Since BIOMOD is freely available and widely used by ensemble species distribution modellers, we focused on articles that apply BIOMOD, filtering the initial 852 papers identified in our structured literature search to a relevant final subset of 224 eligible peer‐reviewed journal articles. Results: BIOMOD‐based ensembles are used across many taxa and locations, with terrestrial plants being the most represented group of species (n = 72) and Europe being the most represented continent (n = 106). These studies often focus on forecasting distributions in the future (n = 109), and commonly use presence‐only species data (n = 139) and climatic environmental predictors (n = 219). An average of six models are used in ensembles, and approximately half of ensembles weight contributions of models by their cross‐validation performance. However, discussion about choices made in the modelling process and unambiguous information on the performance of ensemble models versus individual models are limited. The use of independent data to validate model performance is particularly uncommon. Main conclusions: We document the breadth of ensemble applications, but could not draw strong quantitative conclusions about the predictive performance of ensemble models, due to lack of unambiguous information reported. Understanding how and where ensembles are best used when modelling species distributions is important for enabling best choices for different applications. To enable this objective to be achieved, we provide recommendations for thorough reporting practices in a BIOMOD‐based ensemble workflow.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
Model selection and multimodel inference: a practical information-theoretic approach.
Guthery, F. S., Burnham, K. P., & Anderson, D. R. (2003) · The Journal of Wildlife Management · doi:10.2307/3802723 · Cited by 41,495
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,522
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856 · cited 1×
“…including sensitivity (proportion of true positives correctly iden‐ tified), specificity (proportion of true negatives correctly identi‐ fied), area under curve (AUC) of receiver operating characteristics (Hanley & McNeil, 1982), and true skill statistics (TSS; Allouche, Tsoar, & Kadmon, 2006).”
BackgroundMaximum entropy modeling of species geographic distributions
Phillips, S. J., Anderson, R. P., & Schapire, R. E. (2006) · Ecological Modelling · doi:10.1016/j.ecolmodel.2005.03.026 · Cited by 15,396
Novel methods improve prediction of species’ distributions from occurrence data
Elith*, J., H. Graham*, C., P. Anderson, R., Dudík, M., Ferrier, S., Guisan, A., J. Hijmans, R., Huettmann, F., R. Leathwick, J., Lehmann, A., et al. (2006) · Ecography · doi:10.1111/j.2006.0906-7590.04596.x · Cited by 9,088
Self-citationEnsemble methods in machine learning
Dietterich, T. G. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45014-9_1 · Cited by 8,408
Multivariate Adaptive Regression Splines
Friedman, J. H. (1991) · The Annals of Statistics · doi:10.1214/aos/1176347963 · Cited by 8,000
Species Distribution Models: Ecological Explanation and Prediction Across Space and Time
Elith, J., & Leathwick, J. R. (2009) · Annual Review of Ecology, Evolution, and Systematics · doi:10.1146/annurev.ecolsys.110308.120159 · Cited by 6,707
Self-citationA statistical explanation of MaxEnt for ecologists
Elith, J., Phillips, S. J., Hastie, T., Dudík, M., Chee, Y. E., & Yates, C. J. (2011) · Diversity and Distributions · doi:10.1111/j.1472-4642.2010.00725.x · Cited by 6,699
Self-citationPredicting species distribution: offering more than simple habitat models
Guisan, A., & Thuiller, W. (2005) · Ecology Letters · doi:10.1111/j.1461-0248.2005.00792.x · Cited by 6,587
A working guide to boosted regression trees
Elith, J., Leathwick, J. R., & Hastie, T. (2008) · Journal of Animal Ecology · doi:10.1111/j.1365-2656.2008.01390.x · Cited by 6,484
Self-citationPattern Recognition and Neural Networks
Ripley, B. D. (1996) · Cambridge University Press eBooks · doi:10.1017/cbo9780511812651 · Cited by 6,459
ALLOUCHE, O., TSOAR, A., & KADMON, R. (2006) · Journal of Applied Ecology · doi:10.1111/j.1365-2664.2006.01214.x · Cited by 5,172
Random forests for classification in ecology
Cutler, D. R., Edwards, T. C., Beard, K. H., Cutler, A., Hess, K. T., Gibson, J., & Lawler, J. J. (2007) · Ecology · doi:10.1890/07-0539.1 · Cited by 4,950
Cross‐validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure
Roberts, D. R., Bahn, V., Ciuti, S., Boyce, M. S., Elith, J., Guillera‐Arroita, G., Hauenstein, S., Lahoz‐Monfort, J. J., Schröder, B., Thuiller, W., et al. (2017) · Ecography · doi:10.1111/ecog.02881 · Cited by 3,689
Self-citationEnsemble forecasting of species distributions
ARAUJO, M., & NEW, M. (2007) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2006.09.010 · Cited by 3,640
BIOMOD – a platform for ensemble forecasting of species distributions
Thuiller, W., Lafourcade, B., Engler, R., & Araújo, M. B. (2009) · Ecography · doi:10.1111/j.1600-0587.2008.05742.x · Cited by 2,749
Five (or so) challenges for species distribution modelling
Araújo, M. B., & Guisan, A. (2006) · Journal of Biogeography · doi:10.1111/j.1365-2699.2006.01584.x · Cited by 1,901
Ecological Niches and Geographic Distributions (MPB-49)
Peterson, A. T., Soberón, J., Pearson, R. G., Anderson, R. P., Martínez-Meyer, E., Nakamura, M., & Araújo, M. B. (2011) · Princeton University Press eBooks · doi:10.23943/princeton/9780691136868.001.0001 · Cited by 1,828
Mapping Species Distributions: Spatial Inference and Prediction
Franklin, J. (2010) · doi:10.1017/cbo9780511810602 · Cited by 1,534
Habitat Suitability and Distribution Models: With Applications in R
Guisan, A., Thuiller, W., & Zimmermann, N. E. (2017) · Cambridge University Press eBooks · doi:10.1017/9781139028271 · Cited by 1,479
Evaluation of consensus methods in predictive species distribution modelling
Marmion, M., Parviainen, M., Luoto, M., Heikkinen, R. K., & Thuiller, W. (2009) · Diversity and Distributions · doi:10.1111/j.1472-4642.2008.00491.x · Cited by 1,435
Predictive learning via rule ensembles
Friedman, J. H., & Popescu, B. E. (2008) · The Annals of Applied Statistics · doi:10.1214/07-aoas148 · Cited by 1,315
HIJMANS, R. J., & GRAHAM, C. H. (2006) · Global Change Biology · doi:10.1111/j.1365-2486.2006.01256.x · Cited by 1,179
sdm: a reproducible and extensible R platform for species distribution modelling
Naimi, B., & Araújo, M. B. (2016) · Ecography · doi:10.1111/ecog.01881 · Cited by 1,137
Thuiller, W. (2003) · Global Change Biology · doi:10.1046/j.1365-2486.2003.00666.x · Cited by 1,079
Valavi, R., Elith, J., Lahoz‐Monfort, J. J., & Guillera‐Arroita, G. (2019) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13107 · Cited by 1,056
Self-citationModel‐based uncertainty in species range prediction
Pearson, R. G., Thuiller, W., Araújo, M. B., Martinez‐Meyer, E., Brotons, L., McClean, C., Miles, L., Segurado, P., Dawson, T. P., & Lees, D. C. (2006) · Journal of Biogeography · doi:10.1111/j.1365-2699.2006.01460.x · Cited by 981
Is my species distribution model fit for purpose? Matching data and models to applications
Guillera‐Arroita, G., Lahoz‐Monfort, J. J., Elith, J., Gordon, A., Kujala, H., Lentini, P. E., McCarthy, M. A., Tingley, R., & Wintle, B. A. (2015) · Global Ecology and Biogeography · doi:10.1111/geb.12268 · Cited by 961
Self-citationFlexible Discriminant Analysis by Optimal Scoring
Hastie, T., Tibshirani, R., & Buja, A. (1994) · Journal of the American Statistical Association · doi:10.1080/01621459.1994.10476866 · Cited by 883
An evaluation of methods for modelling species distributions
Segurado, P., & Araújo, M. B. (2004) · Journal of Biogeography · doi:10.1111/j.1365-2699.2004.01076.x · Cited by 875 · cited 1×
“However, no one method is consistently superior in perfor‐ mance across species, regions and applications (Elith et al., 2006; Pearson et al., 2006; Segurado & Araujo, 2004).”
MethodPatterns and uncertainties of species' range shifts under climate change
Thuiller, W. (2004) · Global Change Biology · doi:10.1111/j.1365-2486.2004.00859.x · Cited by 873
Are niche‐based species distribution models transferable in space?
Randin, C. F., Dirnböck, T., Dullinger, S., Zimmermann, N. E., Zappa, M., & Guisan, A. (2006) · Journal of Biogeography · doi:10.1111/j.1365-2699.2006.01466.x · Cited by 768
Diniz‐Filho, J. A. F., Mauricio Bini, L., Fernando Rangel, T., Loyola, R. D., Hof, C., Nogués‐Bravo, D., & Araújo, M. B. (2009) · Ecography · doi:10.1111/j.1600-0587.2009.06196.x · Cited by 615
Anderson, R. P., & Gonzalez, I. (2011) · Ecological Modelling · doi:10.1016/j.ecolmodel.2011.04.011 · Cited by 592
Reducing uncertainty in projections of extinction risk from climate change
Araújo, M. B., Whittaker, R. J., Ladle, R. J., & Erhard, M. (2005) · Global Ecology and Biogeography · doi:10.1111/j.1466-822x.2005.00182.x · Cited by 553
Overcoming limitations of modelling rare species by using ensembles of small models
Breiner, F. T., Guisan, A., Bergamini, A., & Nobis, M. P. (2015) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12403 · Cited by 541
Ensemble Methods in Data Mining: Improving Accuracy Through Combining Predictions
Seni, G., & Elder, J. F. (2010) · Synthesis lectures on data mining and knowledge discovery · doi:10.2200/s00240ed1v01y200912dmk002 · Cited by 492
McCullagh, P. (1984) · European Journal of Operational Research · doi:10.1016/0377-2217(84)90282-0 · Cited by 487
Morales, N. S., Fernández, I. C., & Baca-González, V. (2017) · PeerJ · doi:10.7717/peerj.3093 · Cited by 464
biomod2: Ensemble Platform for Species Distribution Modeling
Guéguen, M., Blancheteau, H., Lemaire-Patin, R., & Thuiller, W. (2012) · CRAN: Contributed Packages · doi:10.32614/cran.package.biomod2 · Cited by 446
The projection of species distribution models and the problem of non-analog climate
Fitzpatrick, M. C., & Hargrove, W. W. (2009) · Biodiversity and Conservation · doi:10.1007/s10531-009-9584-8 · Cited by 415
Dormann, C. F., Calabrese, J. M., Guillera‐Arroita, G., Matechou, E., Bahn, V., Bartoń, K., Beale, C. M., Ciuti, S., Elith, J., Gerstner, K., et al. (2018) · Ecological Monographs · doi:10.1002/ecm.1309 · Cited by 400
Self-citationNature Conservation: Cost Effective Biological Surveys and Data Analysis
Csuti, B., Margules, C. R., & Austin, M. P. (1992) · The Journal of Wildlife Management · doi:10.2307/3808885 · Cited by 253
Transferring biodiversity models for conservation: Opportunities and challenges
Sequeira, A. M. M., Bouchet, P. J., Yates, K. L., Mengersen, K. L., & Caley, M. J. (2018) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12998 · Cited by 142
Meller, L., Cabeza, M., Pironon, S., Barbet‐Massin, M., Maiorano, L., Georges, D., & Thuiller, W. (2013) · Diversity and Distributions · doi:10.1111/ddi.12162 · Cited by 141
Engler, R., Waser, L. T., Zimmermann, N. E., SCHAUB, M. C., Berdos, S., Ginzler, C., & Psomas, A. (2013) · Forest Ecology and Management · doi:10.1016/j.foreco.2013.07.059 · Cited by 104
Evaluating ensemble forecasts of plant species distributions under climate change
Crimmins, S., Dobrowski, S., & Mynsberge, A. R. (2013) · Ecological Modelling · doi:10.1016/j.ecolmodel.2013.07.006 · Cited by 95
Evolution of climate niches in European mammals?
Dormann, C. F., Gruber, B., Winter, M., & Herrmann, D. (2009) · Biology Letters · doi:10.1098/rsbl.2009.0688 · Cited by 73
Consensus forecasting of intertidal seagrass habitat in the Wadden Sea
Folmer, E. O., van Beusekom, J., Dolch, T., Gräwe, U., van Katwijk, M. M., Kolbe, K., & Philippart, C. (2016) · Journal of Applied Ecology · doi:10.1111/1365-2664.12681 · Cited by 47
Testing for Forecast Consensus
Gregory, A. W., Smith, G. W., & Yetman, J. (2001) · Journal of Business and Economic Statistics · doi:10.1198/07350010152472599 · Cited by 39
The zoon r package for reproducible and shareable species distribution modelling
Golding, N., August, T., Lucas, T. C. D., Gavaghan, D. J., van Loon, E. E., & McInerny, G. (2017) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.12858 · Cited by 35
On Subjective Probability Forecasting
(1963) · doi:10.1175/1520-0450(1963)002<0191:ospf>2.0.co;2
Cited by
Valavi, R., Guillera‐Arroita, G., Lahoz‐Monfort, J. J., & Elith, J. (2022) · Ecological Monographs · doi:10.1002/ecm.1486 · Cited by 830
Self-citationHao, T., Elith, J., Lahoz‐Monfort, J. J., & Guillera‐Arroita, G. (2020) · Ecography · doi:10.1111/ecog.04890 · Cited by 477
Self-citationWant to model a species niche? A step-by-step guideline on correlative ecological niche modelling
Sillero, N., Arenas-Castro, S., Enriquez‐Urzelai, U., Vale, C. G., Sousa-Guedes, D., Martínez-Freiría, F., Real, R., & Barbosa, A. M. (2021) · Ecological Modelling · doi:10.1016/j.ecolmodel.2021.109671 · Cited by 456
Kaky, E., Nolan, V., Alatawi, A., & Gilbert, F. (2020) · Ecological Informatics · doi:10.1016/j.ecoinf.2020.101150 · Cited by 432
Park, S.-J., & Lee, D.-K. (2020) · Environmental Research Letters · doi:10.1088/1748-9326/aba5b3 · Cited by 100
Metadata
- Authors
- Hao, T., Elith, J., Guillera‐Arroita, G., & Lahoz‐Monfort, J. J.
- Venue
- Diversity and Distributions
- Publisher
- Wiley
- Published
- 2019-01-22
- Volume
- 25
- Issue
- 5
- Pages
- 839-852
- ISSN
- 0967-9952 (ISSN-L)
- 1366-9516
- 1472-4642
- OpenAlex
- W2912766985
- Cited by
- 674
- References
- 58
- FWCI
- 73.79
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.