Assessing the accuracy of land cover change with imperfect ground reference data
Open accessThe DOI matches its Crossref record; 86 of the 86 references it cites were found in a registry or an index.
Abstract
The ground data used as a reference in the validation of land cover change products are often not an ideal gold standard but degraded by error. The effects of ground reference data error on the accuracy of land cover change detection and the accuracy of estimates of the extent of change was evaluated. Twelve data sets were simulated to allow the exploration of the impacts of a spectrum of ground data imperfections on the estimation of the producer’s and user’s accuracy of change as well as of change extent. Simulated data were used since this ensured that the actual properties of the data were known and to exclude effects due to other sources of ground reference data error; although the impacts of simulated reference data on two real confusion matrices is also illustrated. The imperfections evaluated ranged from the inclusion of small amounts of known error into the ground reference data through to the extreme situation in which ground data were absent. The results show that even small amounts of error in the ground reference data can introduce large error into studies of land cover change by remote sensing and reinforce the desire to avoid the expression ground truth as this might imply that the data are a gold standard reference. The effect of reference data imperfections was dependent on the degree of association between the errors in the cross tabulated data sets. For example, in the scenarios investigated, a 10% error in the reference data set introduced an under-estimation of the producer’s accuracy of 18.5% if the errors were independent but an over-estimation of the producer’s accuracy of 12.3% if the errors were correlated. The magnitude of the mis-estimation of the producer’s accuracy was also a function of the amount of change and greatest at low levels of change. The amount of land cover change estimated also varied greatly as a function of ground reference data error. Some possible methods to reduce or even remove the impacts of ground reference data error were illustrated. These ranged from simple algebraic means to estimate the actual values of accuracy and change extent if the imperfections were known through to a latent class analysis that allowed the assessment of classification accuracy and estimation of change extent without the use of ground reference data if the underlying model is defined appropriately.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A review of assessing the accuracy of classifications of remotely sensed data
Congalton, R. G. (1991) · Remote Sensing of Environment · doi:10.1016/0034-4257(91)90048-b · Cited by 7,724
A review of methods for the assessment of prediction errors in conservation presence/absence models
FIELDING, A. H., & BELL, J. F. (1997) · Environmental Conservation · doi:10.1017/s0376892997000088 · Cited by 6,956 · cited 11×
“Thus, when the conditional independence assumption cannot be made researchers may need to adopt an alternative approach to address the impacts of imperfect ground reference data and one possibility is discussed below.”
BackgroundMethodInfluentialStatus of land cover classification accuracy assessment
Foody, G. M. (2002) · Remote Sensing of Environment · doi:10.1016/s0034-4257(01)00295-4 · Cited by 4,580
Self-citationAssessing the Accuracy of Remotely Sensed Data: Principles and Practices
Congalton, R., & Green, K. (1998) · Mapping Science · doi:10.1201/9781420048568 · Cited by 3,424
Assessing the Accuracy of Remotely Sensed Data: Principles and Practices
Foody, G. (2010) · The Photogrammetric Record · doi:10.1111/j.1477-9730.2010.00574_2.x · Cited by 2,806
Self-citationThe emergence of land change science for global environmental change and sustainability
Turner, B. L., Lambin, E. F., & Reenberg, A. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0704119104 · Cited by 2,216
Determination of Deforestation Rates of the World's Humid Tropical Forests
Achard, F., Eva, H. D., Stibig, H.-J., Mayaux, P., Gallego, J., Richards, T., & Malingreau, J.-P. (2002) · Science · doi:10.1126/science.1070656 · Cited by 1,993 · cited 1×
“For example, Skole and Tucker (1993) highlighted substantial, ~50%, over-estimation of deforestation arising through the use of coarse spatial resolution data sets and Achard et al. (2002) suggest that some estimates of deforestation rates in the humid tropics may have been exaggerated by ~23%.”
BackgroundBeyond Global Warming: Ecology and Global Change
Vitousek, P. M. (1994) · Ecology · doi:10.2307/1941591 · Cited by 1,839
Tropical Deforestation and Habitat Fragmentation in the Amazon: Satellite Data from 1978 to 1988
Skole, D., & Tucker, C. (1993) · Science · doi:10.1126/science.260.5116.1905 · Cited by 1,590
The SAGE Handbook of Quantitative Methodology for the Social Sciences
Kaplan, D. (2004) · doi:10.4135/9781412986311 · Cited by 1,454
Cited by
Good practices for estimating area and assessing accuracy of land change
Olofsson, P., Foody, G. M., Herold, M., Stehman, S. V., Woodcock, C. E., & Wulder, M. A. (2014) · Remote Sensing of Environment · doi:10.1016/j.rse.2014.02.015 · Cited by 3,078
Self-citationAtzberger, C. (2013) · Remote Sensing · doi:10.3390/rs5020949 · Cited by 1,031
Olofsson, P., Foody, G. M., Stehman, S. V., & Woodcock, C. E. (2013) · Remote Sensing of Environment · doi:10.1016/j.rse.2012.10.031 · Cited by 1,029
Self-citationFoody, G. M. (2020) · Remote Sensing of Environment · doi:10.1016/j.rse.2019.111630 · Cited by 665
Self-citationKey issues in rigorous accuracy assessment of land cover products
Stehman, S. V., & Foody, G. M. (2019) · Remote Sensing of Environment · doi:10.1016/j.rse.2019.05.018 · Cited by 651
Self-citationLiu, X., Hu, G., Chen, Y., Li, X., Xu, X., Li, S., Pei, F., & Wang, S. (2018) · Remote Sensing of Environment · doi:10.1016/j.rse.2018.02.055 · Cited by 608
Determination of tropical deforestation rates and related carbon losses from 1990 to 2010
Achard, F., Beuchle, R., Mayaux, P., Stibig, H., Bodart, C., Brink, A., Carboni, S., Desclée, B., Donnay, F., Eva, H. D., et al. (2014) · Global Change Biology · doi:10.1111/gcb.12605 · Cited by 563
Mapping global cropland and field size
Fritz, S., See, L., McCallum, I., You, L., Bun, A., Moltchanova, E., Duerauer, M., Albrecht, F., Schill, C., Perger, C., et al. (2015) · Global Change Biology · doi:10.1111/gcb.12838 · Cited by 559
Mapping farmland abandonment and recultivation across Europe using MODIS NDVI time series
Estel, S., Kuemmerle, T., Alcántara, C., Levers, C., Prishchepov, A., & Hostert, P. (2015) · Remote Sensing of Environment · doi:10.1016/j.rse.2015.03.028 · Cited by 547
Automated cropland mapping of continental Africa using Google Earth Engine cloud computing
Xiong, J., Thenkabail, P. S., Gumma, M. K., Teluguntla, P., Poehnelt, J., Congalton, R. G., Yadav, K., & Thau, D. (2017) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2017.01.019 · Cited by 488
Kattenborn, T., Lopatin, J., Förster, M., Braun, A. C., & Fassnacht, F. E. (2019) · Remote Sensing of Environment · doi:10.1016/j.rse.2019.03.025 · Cited by 258
Metadata
- Authors
- Foody, G. M.
- Venue
- Remote Sensing of Environment
- Publisher
- Elsevier BV
- Published
- 2010-10
- Volume
- 114
- Issue
- 10
- Pages
- 2271-2285
- ISSN
- 0034-4257 (ISSN-L)
- OpenAlex
- W2111743232
- Cited by
- 333
- References
- 86
- FWCI
- 37.76
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.