Cited by 307

The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.

Modelling species presence‐only data with random forests

Valavi, Roozbeh, Elith, Jane, Lahoz‐Monfort, José J., Guillera‐Arroita, Gurutzeta·2021·Ecography

Published version of a preprint

Abstract

The random forest (RF) algorithm is an ensemble of classification or regression trees and is widely used, including for species distribution modelling (SDM). Many researchers use implementations of RF in the R programming language with default parameters to analyse species presence‐only data together with ‘background' samples. However, there is good evidence that RF with default parameters does not perform well for such ‘presence‐background' modelling. This is often attributed to the disparity between the number of presence and background samples, also known as 'class imbalance', and several solutions have been proposed. Here, we first set the context: the background sample should be large enough to represent all environments in the region. We then aim to understand the drivers of poor performance of RF when models are fitted to presence‐only species data alongside background samples. We show that 'class overlap' (where both classes occur in the same environment) is an important driver of poor performance, alongside class imbalance. Class overlap can even degrade performance for presence–absence data. We explain, test and evaluate suggested solutions. Using simulated and real presence‐background data, we compare performance of default RF with other weighting and sampling approaches. Our results demonstrate clear evidence of improvement in the performance of RFs when techniques that explicitly manage imbalance are used. We show that these either limit or enforce tree depth. Without compromising the environmental representativeness of the sampled background, we identify approaches to fitting RF that ameliorate the effects of imbalance and overlap and allow excellent predictive performance. Understanding the problems of RF in presence‐background modelling allows new insights into how best to fit models, and should guide future efforts to best deal with such data.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1111/ecog.05615

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Two species distributions simulated for assessing the impact of class overlap on RF. The left plots (a & c) show the probability of species occurrence (  ) as a function of a single predictor. We assume the predictor is equally represented across the landscape. Plots b & d show the predictive performance of RF with default settings vs RF using a method for dealing with class imbalance (down-sampling, see section 'suggested solutions'). See 'data availability' section for code.
Figure 1 Two species distributions simulated for assessing the impact of class overlap on RF. The left plots (a & c) show the probability of species occurrence (  ) as a function of a single predictor. We assume the predictor is equally represented across the landscape. Plots b & d show the predictive performance of RF with default settings vs RF using a method for dealing with class imbalance (down-sampling, see section 'suggested solutions'). See 'data availability' section for code.CC-BY · From the PDF
Figure 2 Visualisation of recursive partitioning with different number of background points. The 0s (red) and 1s (blue) show background and presence samples, respectively. The blue sections are the terminal nodes predicted as presence and red are predicted as background. The lines in the plots show the split
Figure 2 Visualisation of recursive partitioning with different number of background points. The 0s (red) and 1s (blue) show background and presence samples, respectively. The blue sections are the terminal nodes predicted as presence and red are predicted as background. The lines in the plots show the splitCC-BY · From the PDF
Figure 3 Performance of different RF models on different regions of NCEAS data. The values in cells are AUC and COR for the left and right plots respectively. Colour varies with values, yellow (low) to dark navy (high).
Figure 3 Performance of different RF models on different regions of NCEAS data. The values in cells are AUC and COR for the left and right plots respectively. Colour varies with values, yellow (low) to dark navy (high).CC-BY · From the PDF
Image 4
CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 56

  1. Ali A., 2015, Classification with class imbalance problem: a review, Int. J. Adv. Soft Comput. Appl., 7, 176
  2. 10.1126/sciadv.aat4858 · DOI
  3. 10.1111/j.2041-210X.2011.00172.x · DOI · Paper page
  4. 10.1007/978-3-642-52307-6_8 · DOI
  5. 10.1023/A:1010933404324 · DOI · Paper page
  6. Chen C., 2004, Using random forest to learn imbalanced data, 1
  7. Cieslak D. A., 2008, Learning decision trees for unbalanced data, Daelemans, W. et al. (eds), Joint European conference on machine learning and knowledge discovery in databases, 241
  8. 10.1007/s10618-011-0222-1 · DOI
  9. Demšar J., 2006, Statistical comparisons of classifiers over multiple data sets, J. Mach. Learn. Res., 7
  10. Denil M., 2010, Overlap versus imbalance, Advances in artificial intelligence lecture notes in computer science, 220
  11. Drummond C., 2000, Exploiting the cost (In) sensitivity of decision tree splitting criteria, ICML, 8
  12. 10.1201/9780429246593 · DOI · Paper page
  13. Elith J., 2019, Machine learning, random forests and boosted regression trees, Brennan, L. et al. (eds), Quantitative analyses in wildlife science wildlife management and conservation, 281
  14. 10.1111/j.2006.0906-7590.04596.x · DOI · Paper page
  15. 10.17161/bi.v15i2.13384 · DOI
  16. 10.1007/s10980-009-9341-0 · DOI
  17. 10.1111/2041-210X.12242 · DOI
  18. 10.1017/CBO9780511810602 · DOI
  19. 10.1016/j.ecolmodel.2012.03.007 · DOI
  20. 10.1139/cjfr-2014-0562 · DOI
  21. García S., 2008, An extension on ‘statistical comparisons of classifiers over multiple data sets' for all pairwise comparisons, J. Mach. Learn. Res., 9, 2677
  22. 10.1111/geb.12268 · DOI
  23. 10.1016/S0304-3800(02)00204-1 · DOI · Paper page
  24. 10.1017/9781139028271 · DOI
  25. Hastie T., 2009, The elements of statistical learning: data mining, inference and prediction, 10.1007/978-0-387-84858-7 · DOI · Paper page
  26. 10.1109/TKDE.2008.239 · DOI · Paper page
  27. Japkowicz N., 2003, Class imbalances: are we focusing on the right issue, Workshop on learning from imbalanced data sets II
  28. 10.3233/IDA-2002-6504 · DOI · Paper page
  29. 10.1109/CIDU.2012.6382186 · DOI
  30. 10.1186/1472-6947-11-51 · DOI
  31. Kujala H., 2015, Identifying conservation priorities and assessing impacts and trade‐offs of potential future development in the Lower Hunter Valley in New South Wales. Univ. of Melbourne, Melbourne, Victoria, 106
  32. 10.1111/ecog.01388 · DOI
  33. Liaw A., 2002, Classification and regression by randomForest, R News, 2, 18
  34. 10.1016/j.ecolmodel.2012.07.003 · DOI
  35. 10.1145/3347711 · DOI
  36. 10.3414/ME00-01-0052 · DOI
  37. 10.1111/ecog.00845 · DOI
  38. 10.1111/j.1600-0587.2011.07157.x · DOI
  39. Mthembu L., 2008, A note on the separability index, arXiv:0812.1107. · arXiv
  40. 10.1086/587826 · DOI · Paper page
  41. 10.1016/S0304-3800(00)00322-7 · DOI
  42. 10.1890/07-2153.1 · DOI · Paper page
  43. Prati R. C., 2004, Class imbalances versus class overlapping: an analysis of a learning system behavior, Mexican international conference on artificial intelligence, 312
  44. Probst P., 2019, Tunability: importance of hyperparameters of machine learning algorithms, J. Mach. Learn. Res., 20, 1
  45. Probst P., 2019, Hyperparameters and tuning strategies for random forest, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 9, e1301
  46. Rabinowitz D., 1981, Seven forms of rarity, The biological aspects of rare plant conservation (ed. by H. Synge). Wiley, 205
  47. 10.1111/2041-210X.12352 · DOI
  48. 10.1111/ecog.02881 · DOI · Paper page
  49. 10.1111/ddi.12698 · DOI
  50. 10.1002/ece3.2332 · DOI
  51. 10.1029/2019WR024884 · DOI
  52. 10.1186/1471-2105-8-25 · DOI
  53. 10.1002/ecm.1486 · DOI · Paper page
  54. 10.1214/10-AOAS331 · DOI
  55. Wright M. N., 2017, ranger: a fast implementation of random forests for high dimensional data in C++ and R, J. Stat. Softw., 77, 1, 10.18637/jss.v077.i01 · DOI
  56. 10.1016/j.ecoinf.2019.05.003 · DOI

Cited by

Sorted by citations

See all 307

Metadata

Authors
Valavi, R., Elith, J., Lahoz‐Monfort, J. J., & Guillera‐Arroita, G.
Venue
Ecography
Publisher
Wiley
Published
2021-10-27
Volume
44
Issue
12
Pages
1731-1742
ISSN
  • 0906-7590
  • 1600-0587
Cited by
307
References
4
Language
en

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review