Cited by 315Species Distribution and Climate ChangeSoil Geostatistics and MappingEcology and Vegetation Dynamics Studies

Machine learning-based global maps of ecological variables and the challenge of assessing them

Open access
Meyer, Hanna, Pebesma, Edzer·2022·Nature Communications

Abstract

The recent wave of published global maps of ecological variables has caused as much excitement as it has received criticism. Here we look into the data and methods mostly used for creating these maps, and discuss whether the quality of predicted values can be assessed, globally and locally.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41467-022-29838-9

Cites

25 of 25 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • If the null hypothesis of no clustering is not rejected, a random k-fold CV is the appropriate option for random and regular samples.

    “If the null hypothesis is not rejected, the algorithm returns a random kfold CV, since it is the appropriate option for random and regular samples (Meyer and Pebesma, 2022;Wadoux et al., 2021;de Bruin et al., 2022).”

    From kNNDM CV: <i>k</i>-fold nearest-neighbour distance matching cross-validation for map accuracy estimation (2024) · cites this paper as Meyer and Pebesma, 2022

    The “Map quality: global or local assessment?” section says random n-fold cross-validation makes sense when data are independent and identically distributed, as can occur with completely spatially random sampling. It does not say that failing to reject a null hypothesis of no clustering establishes that condition, or that random k-fold cross-validation is appropriate for regular samples. Page numbers were not available in our copy. (LLM, conf=0.84)

    Experimental — accuracy not yet measured
  • When samples are randomly distributed within the prediction area, the distribution of NNDs during random 10-fold CV resembles the distribution of NNDs during prediction, so W is small.

    “As shown in Meyer and Pebesma (2022), when samples are randomly distributed within the prediction area, the distribution of NNDs during the random 10-fold CV Ĝ * j (r, L) resembles the distribution of NNDs during prediction Ĝij (r), and therefore the value of W is small.”

    From kNNDM CV: <i>k</i>-fold nearest-neighbour distance matching cross-validation for map accuracy estimation (2024) · cites this paper as Meyer and Pebesma, 2022

    The “Map quality: global or local assessment?” section says cross-validation prediction situations naturally resemble map-prediction situations when reference data are sampled completely at random. Figure 1 in “Global reference data used in machine learning applications” compares nearest-neighbor distance distributions for a simulated spatially random sample. The paper does not specify random 10-fold cross-validation or define and report a small value of W. (LLM, conf=0.83)

    Experimental — accuracy not yet measured
  • The design of a CV method to provide a reliable estimate for map accuracy should be prediction-oriented; predictive conditions during CV should resemble conditions when the trained model is applied to the prediction area.

    “In our past work, we argued that the design of a CV method to provide a reliable estimate for map accuracy should be prediction-oriented; i.e. predictive conditions created during CV should resemble the conditions found when the trained model is applied to the prediction area (Milà et al., 2022;Meyer and Pebesma, 2022;Ludwig et al., 2023).”
  • When standard LOO CV is used for reference data randomly distributed within the prediction area, the distribution of NNDs between test and training locations during CV is similar to the distribution of NNDs between prediction and training locations.

    “In Milà et al. (2022) we showed that when standard LOO CV is used for reference data randomly distributed within the prediction area, the distribution of NNDs between test and training locations during CV is similar to the distribution of NNDs between prediction and training locations (see Meyer and Pebesma, 2022, for similar results for random kfold CV).”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Meyer, H., & Pebesma, E.
Affiliations
University of Münster (DE)
Corresponding author
  • Meyer, Hanna
  • Pebesma, Edzer
Venue
Nature Communications
Published
2022-04-22
Volume
13
Issue
1
Pages
2208-2208
ISSN
2041-1723 (ISSN-L)
OpenAlex
W4224247190
Cited by
310
References
25
Citations by year
2026: 97 · 2025: 61 · 2024: 72 · 2023: 60 · 2022: 19 · 2021: 1
FWCI
124.96
Type
journal article
Language
en
Open access
Locations
  • Nature Communications (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Environmental Science › Ecological Modeling › Species Distribution and Climate Change
  • Physical Sciences › Environmental Science › Environmental Engineering › Soil Geostatistics and Mapping
  • Physical Sciences › Environmental Science › Nature and Landscape Conservation › Ecology and Vegetation Dynamics Studies
Keywords
machine learning, uncertainty quantification, spatial prediction, biodiversity data
MeSH terms
Machine Learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review