The DOI matches its Crossref record; 86 of the 86 references it cites were found in a registry or an index.
Assessing the accuracy of land cover change with imperfect ground reference data
Open accessAbstract
The ground data used as a reference in the validation of land cover change products are often not an ideal gold standard but degraded by error. The effects of ground reference data error on the accuracy of land cover change detection and the accuracy of estimates of the extent of change was evaluated. Twelve data sets were simulated to allow the exploration of the impacts of a spectrum of ground data imperfections on the estimation of the producer’s and user’s accuracy of change as well as of change extent. Simulated data were used since this ensured that the actual properties of the data were known and to exclude effects due to other sources of ground reference data error; although the impacts of simulated reference data on two real confusion matrices is also illustrated. The imperfections evaluated ranged from the inclusion of small amounts of known error into the ground reference data through to the extreme situation in which ground data were absent. The results show that even small amounts of error in the ground reference data can introduce large error into studies of land cover change by remote sensing and reinforce the desire to avoid the expression ground truth as this might imply that the data are a gold standard reference. The effect of reference data imperfections was dependent on the degree of association between the errors in the cross tabulated data sets. For example, in the scenarios investigated, a 10% error in the reference data set introduced an under-estimation of the producer’s accuracy of 18.5% if the errors were independent but an over-estimation of the producer’s accuracy of 12.3% if the errors were correlated. The magnitude of the mis-estimation of the producer’s accuracy was also a function of the amount of change and greatest at low levels of change. The amount of land cover change estimated also varied greatly as a function of ground reference data error. Some possible methods to reduce or even remove the impacts of ground reference data error were illustrated. These ranged from simple algebraic means to estimate the actual values of accuracy and change extent if the imperfections were known through to a latent class analysis that allowed the assessment of classification accuracy and estimation of change extent without the use of ground reference data if the underlying model is defined appropriately.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A review of assessing the accuracy of classifications of remotely sensed data
Congalton, R. G. (1991) · Remote Sensing of Environment · doi:10.1016/0034-4257(91)90048-b · Cited by 7,724
A review of methods for the assessment of prediction errors in conservation presence/absence models
FIELDING, A. H., & BELL, J. F. (1997) · Environmental Conservation · doi:10.1017/s0376892997000088 · Cited by 6,956
Status of land cover classification accuracy assessment
Foody, G. M. (2002) · Remote Sensing of Environment · doi:10.1016/s0034-4257(01)00295-4 · Cited by 4,580
Self-citationAssessing the Accuracy of Remotely Sensed Data: Principles and Practices
Congalton, R., & Green, K. (1998) · Mapping Science · doi:10.1201/9781420048568 · Cited by 3,424
Assessing the Accuracy of Remotely Sensed Data: Principles and Practices
Foody, G. (2010) · The Photogrammetric Record · doi:10.1111/j.1477-9730.2010.00574_2.x · Cited by 2,806
Self-citationThe emergence of land change science for global environmental change and sustainability
Turner, B. L., Lambin, E. F., & Reenberg, A. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0704119104 · Cited by 2,216
Determination of Deforestation Rates of the World's Humid Tropical Forests
Achard, F., Eva, H. D., Stibig, H.-J., Mayaux, P., Gallego, J., Richards, T., & Malingreau, J.-P. (2002) · Science · doi:10.1126/science.1070656 · Cited by 1,993 · cited 1×
“For example, Skole and Tucker (1993) highlighted substantial, ~50%, over-estimation of deforestation arising through the use of coarse spatial resolution data sets and Achard et al. (2002) suggest that some estimates of deforestation rates in the humid tropics may have been exaggerated by ~23%.”
BackgroundBeyond Global Warming: Ecology and Global Change
Vitousek, P. M. (1994) · Ecology · doi:10.2307/1941591 · Cited by 1,839
Tropical Deforestation and Habitat Fragmentation in the Amazon: Satellite Data from 1978 to 1988
Skole, D., & Tucker, C. (1993) · Science · doi:10.1126/science.260.5116.1905 · Cited by 1,590
The SAGE Handbook of Quantitative Methodology for the Social Sciences
Kaplan, D. (2004) · doi:10.4135/9781412986311 · Cited by 1,454
African Land-Cover Classification Using Satellite Data
Tucker, C. J., Townshend, J. R. G., & Goff, T. E. (1985) · Science · doi:10.1126/science.227.4685.369 · Cited by 1,055
ESTIMATING PREVALENCE FROM THE RESULTS OF A SCREENING TEST
ROGAN, W. J., & GLADEN, B. (1978) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a112510 · Cited by 1,001
Weng, Q. (2002) · Journal of Environmental Management · doi:10.1006/jema.2001.0509 · Cited by 890
Land-cover change detection using multi-temporal MODIS NDVI data
Lunetta, R. S., Knight, J. F., Ediriwickrema, J., Lyon, J. G., & Worthy, L. D. (2006) · Remote Sensing of Environment · doi:10.1016/j.rse.2006.06.018 · Cited by 830
DeFries, R. S., Houghton, R. A., Hansen, M. C., Field, C. B., Skole, D., & Townshend, J. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.182560099 · Cited by 715
Developing a science of land change: Challenges and methodological issues
Rindfuss, R. R., Walsh, S. J., Turner, B. L., Fox, J., & Mishra, V. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0401545101 · Cited by 657
THE RELATIONSHIP BETWEEN LAND-USE CHANGE AND CLIMATE CHANGE
Dale, V. H. (1997) · Ecological Applications · doi:10.1890/1051-0761(1997)007[0753:trbluc]2.0.co;2 · Cited by 654
Land cover mapping of large areas from satellites: Status and research priorities
Cihlar, J. (2000) · International Journal of Remote Sensing · doi:10.1080/014311600210092 · Cited by 643
Herold, M., Mayaux, P., Woodcock, C. E., Baccini, A., & Schmullius, C. (2008) · Remote Sensing of Environment · doi:10.1016/j.rse.2007.11.013 · Cited by 569
Estimation of diagnostic-test sensitivity and specificity through Bayesian modeling
Branscum, A. J., Gardner, I. A., & Johnson, W. O. (2005) · Preventive Veterinary Medicine · doi:10.1016/j.prevetmed.2004.12.005 · Cited by 533
Fuzzy set approach to assessing similarity of categorical maps
Hagen, A. (2003) · International Journal of Geographical Information Science · doi:10.1080/13658810210157822 · Cited by 501
Enøe, C., Georgiadis, M. P., & Johnson, W. O. (2000) · Preventive Veterinary Medicine · doi:10.1016/s0167-5877(00)00117-3 · Cited by 485
Random Effects Models in Latent Class Analysis for Evaluating Accuracy of Diagnostic Tests
Qu, Y., Tan, M., & Kutner, M. H. (1996) · Biometrics · doi:10.2307/2533043 · Cited by 449
Kennedy, R. E., Townsend, P. A., Gross, J. E., Cohen, W. B., Bolstad, P., Wang, Y. Q., & Adams, P. (2009) · Remote Sensing of Environment · doi:10.1016/j.rse.2008.07.018 · Cited by 411
Sampling designs for accuracy assessment of land cover
Stehman, S. V. (2009) · International Journal of Remote Sensing · doi:10.1080/01431160903131000 · Cited by 407
Land-Use/Land-Cover Change Detection Using Improved Change-Vector Analysis
Chen, J., Gong, P., He, C., Pu, R., & Shi, P. (2003) · Photogrammetric Engineering & Remote Sensing · doi:10.14358/pers.69.4.369 · Cited by 383
Measuring and modelling biodiversity from space
Gillespie, T. W., Foody, G. M., Rocchini, D., Giorgi, A. P., & Saatchi, S. (2008) · Progress in Physical Geography: Earth and Environment · doi:10.1177/0309133308093606 · Cited by 378
Self-citationVermunt, J. K. (2010) · International Encyclopedia of Education · doi:10.1016/b978-0-08-044894-7.01340-3 · Cited by 373
Harshness in image classification accuracy assessment
Foody, G. M. (2008) · International Journal of Remote Sensing · doi:10.1080/01431160701442120 · Cited by 357
Self-citationConditional dependence between tests affects the diagnosis and surveillance of animal diseases
Gardner, I. A., Stryhn, H., Lind, P., & Collins, M. T. (2000) · Preventive Veterinary Medicine · doi:10.1016/s0167-5877(00)00119-7 · Cited by 342
The Effect of Conditional Dependence on the Evaluation of Diagnostic Tests
Vacek, P. M. (1985) · Biometrics · doi:10.2307/2530967 · Cited by 317
Developments in the 'validation' of satellite sensor products for the study of the land surface
Justice, C. O., Belward, A. S., Morisette, J. T., Lewis, P., Privette, J. L., & Baret, F. (2000) · International Journal of Remote Sensing · doi:10.1080/014311600750020000 · Cited by 302
Remote sensing and land cover area estimation
Gallego, F. J. (2004) · International Journal of Remote Sensing · doi:10.1080/01431160310001619607 · Cited by 297
Mapping urban areas on a global scale: which of the eight maps now available is more accurate?
Potere, D., Schneider, A., Angel, S., & Civco, D. L. (2009) · International Journal of Remote Sensing · doi:10.1080/01431160903121134 · Cited by 295
Brannstrom, C., Jepson, W., Filippi, A. M., Redo, D., Xu, Z., & Ganesh, S. (2008) · Land Use Policy · doi:10.1016/j.landusepol.2007.11.008 · Cited by 281
The value of latent class analysis in medical diagnosis
Rindskopf, D., & Rindskopf, W. (1986) · Statistics in Medicine · doi:10.1002/sim.4780050105 · Cited by 280
Garrett, E. S., & Zeger, S. L. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.01055.x · Cited by 275
Results and implications of a study of fifteen years of satellite image classification experiments
Wilkinson, G. G. (2005) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2004.837325 · Cited by 255
Evaluation of diagnostic tests without gold standards
Hui, S. L., & Zhou, X. H. (1998) · Statistical Methods in Medical Research · doi:10.1177/096228029800700404 · Cited by 252
Development of a large area biodiversity monitoring system driven by remote sensing
Duro, D. C., Coops, N. C., Wulder, M. A., & Han, T. (2007) · Progress in Physical Geography: Earth and Environment · doi:10.1177/0309133307079054 · Cited by 251
Log-Linear Models for Event Histories
Holford, T. R., & Vermunt, J. K. (1998) · Journal of the American Statistical Association · doi:10.2307/2669879 · Cited by 231
Evaluating diagnostic tests with imperfect standards
Valenstein, P. N. (1990) · American Journal of Clinical Pathology · doi:10.1093/ajcp/93.2.252 · Cited by 220 · cited 5×
“…the assumption of conditional independence that underlies the standard latent class model means that its only parameters are the latent class probabilities and the conditional probabilities, which define the sensitivity and specificity of the classifications (Yang and Becker, 1997; Rindskopf,…”
BackgroundMethodInfluentialSources of error in accuracy assessment of thematic land-cover maps in the Brazilian Amazon
Powell, R. L., Matzke, N., de Souza, C., Clark, M., Numata, I., Hess, L. L., & Roberts, D. A. (2004) · Remote Sensing of Environment · doi:10.1016/j.rse.2003.12.007 · Cited by 212
Land-Cover Change Monitoring with Classification Trees Using Landsat TM and Ancillary Data
Rogan, J., Miller, J., Stow, D., Franklin, J., Levien, L., & Fischer, C. (2003) · Photogrammetric Engineering & Remote Sensing · doi:10.14358/pers.69.7.793 · Cited by 210
Albert, P. S., & Dodd, L. E. (2004) · Biometrics · doi:10.1111/j.0006-341x.2004.00187.x · Cited by 208
Ecosystems and Land Use Change
(2004) · Geophysical Monograph Series · doi:10.1029/gm153 · Cited by 206
Correlation-Adjusted Estimation of Sensitivity and Specificity of Two Diagnostic Tests
Georgiadis, M. P., Johnson, W. O., Gardner, I. A., & Singh, R. (2003) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/1467-9876.00389 · Cited by 188
A joint initiative for harmonization and validation of land cover datasets
Herold, M., Woodcock, C. E., di Gregorio, A., Mayaux, P., Belward, A. S., Latham, J., & Schmullius, C. C. (2006) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2006.871219 · Cited by 182
COMPARISON OF A SCREENING TEST AND A REFERENCE TEST IN EPIDEMIOLOGIC STUDIES
BUCK, A. A., & GART, J. J. (1966) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a120609 · Cited by 181
Insights into latent class analysis of diagnostic test performance
Pepe, M. S., & Janes, H. (2007) · Biostatistics · doi:10.1093/biostatistics/kxl038 · Cited by 181
A disappearing biome? Reconsidering land‐cover change in the Brazilian savanna
JEPSON, W. (2005) · The Geographical Journal · doi:10.1111/j.1475-4959.2005.00153.x · Cited by 177
COMPARISON OF A SCREENING TEST AND A REFERENCE TEST IN EPIDEMIOLOGIC STUDIES
GART, J. J., & BUCK, A. A. (1966) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a120610 · Cited by 177
Methodology for the assessment of new dichotomous diagnostic tests
Staquet, M., Rozencweig, M., Lee, Y. J., & Muggia, F. M. (1981) · Journal of Chronic Diseases · doi:10.1016/0021-9681(81)90059-x · Cited by 163
Effects of dependent errors in the assessment of diagnostic test performance
Torrance-Rynard, V. L., & Walter, S. D. (1997) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19971015)16:19<2157::aid-sim653>3.0.co;2-x · Cited by 159
Comber, A., Fisher, P., & Wadsworth, R. A. (2005) · Environment and Planning B Planning and Design · doi:10.1068/b31135 · Cited by 153
MCALPINE, C. A., SYKTUS, J., RYAN, J. G., DEO, R. C., MCKEON, G. M., MCGOWAN, H. A., & PHINN, S. R. (2009) · Global Change Biology · doi:10.1111/j.1365-2486.2009.01939.x · Cited by 152 · cited 1×
“One major role of land cover change is as a source of enhanced atmospheric CO2 that contributes to global warming and which may in turn result in further land cover changes (Dale, 1997; McAlpine et al., 2009).”
BackgroundMeasuring the accuracy of species distribution models: a review
Liu, C., White, M., & Newell, G. R. (2009) · MODSIM · Cited by 148
Latent Variable Modeling of Diagnostic Accuracy
Yang, I., & Becker, M. P. (1997) · Biometrics · doi:10.2307/2533555 · Cited by 146
A Novel Context-Sensitive Semisupervised SVM Classifier Robust to Mislabeled Training Samples
Bruzzone, L., & Persello, C. (2009) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2008.2011983 · Cited by 133
Detection of land-cover transitions by combining multidate classifiers
Bruzzone, L., Cossu, R., & Vernazza, G. L. (2004) · Pattern Recognition Letters · doi:10.1016/j.patrec.2004.06.002 · Cited by 132
Using Latent Class Models to Characterize and Assess Relative Error in Discrete Measurements
Espeland, M. A., & Handelman, S. L. (1989) · Biometrics · doi:10.2307/2531499 · Cited by 129
Latent class analysis of diagnostic agreement
Uebersax, J. S., & Grove, W. M. (1990) · Statistics in Medicine · doi:10.1002/sim.4780090509 · Cited by 125
Monitoring land use and cover around parks: A conceptual approach
Jones, D. A., Hansen, A. J., Bly, K., Doherty, K. E., Verschuyl, J. P., Paugh, J. I., Carle, R., & Story, S. J. (2009) · Remote Sensing of Environment · doi:10.1016/j.rse.2008.08.018 · Cited by 114
Bruzzone, L., & Marconcini, M. (2009) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2008.2007741 · Cited by 114
Sample size calculations for comparative studies of medical tests for detecting presence of disease
Alonzo, T. A., PEPE, M. S., & Moskowitz, C. S. (2002) · Statistics in Medicine · doi:10.1002/sim.1058 · Cited by 107
Frequentist and Bayesian approaches to prevalence estimation using examples from Johne's disease
Messam, L. L. M., Branscum, A. J., Collins, M. T., & Gardner, I. A. (2008) · Animal Health Research Reviews · doi:10.1017/s1466252307001314 · Cited by 93
Townshend, J. R. G. (1992) · International Journal of Remote Sensing · doi:10.1080/01431169208904193 · Cited by 93
Interpreting the change detection error matrix
van Oort, P. A. J. (2006) · Remote Sensing of Environment · doi:10.1016/j.rse.2006.10.012 · Cited by 85
Albert, P. S., McShane, L. M., & Shih, J. H. (2001) · Biometrics · doi:10.1111/j.0006-341x.2001.00610.x · Cited by 85
Quality assessment of classification and cluster maps without ground truth knowledge
Baraldi, A., Bruzzone, L., & Blonda, P. N. (2005) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2004.843074 · Cited by 84
Improved Monitoring of Rainforests Helps Pierce Haze of Deforestation
Kintisch, E. (2007) · Science · doi:10.1126/science.316.5824.536 · Cited by 83
Quantifying error in aerial survey data
Johnson, E. W., & Ross, J. (2008) · Australian Forestry · doi:10.1080/00049158.2008.10675038 · Cited by 80
See, L., & Fritz, S. (2006) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2006.874750 · Cited by 80
Some issues in resolution of diagnostic tests using an imperfect gold standard
Hawkins, D. M., Garrett, J. A., & Stephenson, B. D. (2001) · Statistics in Medicine · doi:10.1002/sim.819 · Cited by 76
Holocene land-use change and its impact on river basin dynamics in Great Britain and Ireland
Foulds, S. A., & Macklin, M. G. (2006) · Progress in Physical Geography Earth and Environment · doi:10.1177/0309133306071143 · Cited by 71
Broich, M., Stehman, S. V., Hansen, M. C., Potapov, P. V., & Shimabukuro, Y. E. (2009) · Remote Sensing of Environment · doi:10.1016/j.rse.2009.07.011 · Cited by 70
Diagnostic test analyses in search of their gold standard: latent class analyses with random effects
Goetghebeur, E. J. T., Liinev, J., Boelaert, M., & der Stuyft, P. V. (2000) · Statistical Methods in Medical Research · doi:10.1177/096228020000900304 · Cited by 69
The impact of imperfect ground reference data on the accuracy of land cover change estimation
Foody, G. M. (2009) · International Journal of Remote Sensing · doi:10.1080/01431160902755346 · Cited by 61
Self-citationAssessing a predictive model of land change using uncertain data
Pontius, R. G., & Petrova, S. (2009) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2009.09.005 · Cited by 59
Stehman, S. V. (2005) · Remote Sensing of Environment · doi:10.1016/j.rse.2005.04.002 · Cited by 59
Improving land cover change estimates by accounting for classification errors
van Oort, P. A. J. (2005) · International Journal of Remote Sensing · doi:10.1080/01431160500057848 · Cited by 46
Hydrological consequences of land use change: A review of the state-of-science
Eshleman, K. N. (2004) · Geophysical monograph · doi:10.1029/153gm03 · Cited by 40
Estimating the prevalence of a rare disease: adjusted maximum likelihood
Rahme, E., & Joseph, L. (1998) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.1111/1467-9884.00120 · Cited by 35
Enøe, C., Andersen, S., Sørensen, V., & Willeberg, P. W. (2001) · Preventive Veterinary Medicine · doi:10.1016/s0167-5877(01)00226-4 · Cited by 29
Editorial Response: Can We Do Better Than Discrepant Analysis for New Diagnostic Test Evaluation?
Miller, W. C. (1998) · Clinical Infectious Diseases · doi:10.1086/514996 · Cited by 26
Remote classification ofCerrado(Savanna) and agricultural land covers in northeastern Brazil
Brannstrom, C., & Filippi, A. M. (2008) · Geocarto International · doi:10.1080/10106040701596767 · Cited by 17
Cited by
Good practices for estimating area and assessing accuracy of land change
Olofsson, P., Foody, G. M., Herold, M., Stehman, S. V., Woodcock, C. E., & Wulder, M. A. (2014) · Remote Sensing of Environment · doi:10.1016/j.rse.2014.02.015 · Cited by 3,078
Self-citationAtzberger, C. (2013) · Remote Sensing · doi:10.3390/rs5020949 · Cited by 1,031
Olofsson, P., Foody, G. M., Stehman, S. V., & Woodcock, C. E. (2013) · Remote Sensing of Environment · doi:10.1016/j.rse.2012.10.031 · Cited by 1,029
Self-citationFoody, G. M. (2020) · Remote Sensing of Environment · doi:10.1016/j.rse.2019.111630 · Cited by 665
Self-citationKey issues in rigorous accuracy assessment of land cover products
Stehman, S. V., & Foody, G. M. (2019) · Remote Sensing of Environment · doi:10.1016/j.rse.2019.05.018 · Cited by 651
Self-citation
Metadata
- Authors
- Foody, G. M.
- Venue
- Remote Sensing of Environment
- Publisher
- Elsevier BV
- Published
- 2010-10
- Volume
- 114
- Issue
- 10
- Pages
- 2271-2285
- ISSN
- 0034-4257 (ISSN-L)
- OpenAlex
- W2111743232
- Cited by
- 333
- References
- 86
- FWCI
- 37.76
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.