Cited by 248Remote-Sensing Image ClassificationRemote Sensing and Land UseStatistical Methods in Epidemiology

The DOI matches its Crossref record; 73 of the 74 references it cites were found in a registry or an index.

Sample size determination for image classification accuracy assessment and comparison

Foody, Giles M.·2009·International Journal of Remote Sensing

Open this paper in CoChat — free account

Abstract

Many factors influence the quality and value of a classification accuracy assessment and evaluation programme. This paper focuses on the size of the testing set(s) used with particular regard to the impacts on accuracy assessment and comparison. Testing set size is important as the use of an inappropriately large or small sample could lead to limited and sometimes erroneous assessments of accuracy and of differences in accuracy. Here, some of the basic statistical principles of sample size determination are outlined, including a discussion of Type II errors and their control. The paper provides a discussion on some of the basic issues of sample size determination for accuracy assessment and includes factors linked to accuracy comparison. With the latter, the researcher should specify the effect size (minimum meaningful difference in accuracy), significance level and power used in an analysis and ideally also fit confidence limits to derived estimates. This will help design a study and aid the use of appropriate sample sizes, as well as facilitate interpretation of results. In particular, it will help avoid problems, such as under-powered analyses, and provide a richer information base for classification evaluation. The paper includes equations that could be used to determine sample sizes for common applications in remote sensing, using both independent and related samples.

Integrity signals

Registry

Crossref · 10.1080/01431160903130937

Cites

73 of 73 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Statistical Methods for Rates and Proportions
Fleiss, J. L., Levin, B., & Paik, M. C. (2003) · Wiley Series in Probability and Statistics · doi:10.1002/0471445428 · Cited by 18,646
Photogrammetric Engineering and Remote Sensing
Nicol, J., King, B., Quattrochi, D., Dowman, I. J., Ehlers, M., & Xiaoli, D. (2007) · UCL Discovery (University College London) · Cited by 9,325
Status of land cover classification accuracy assessment
Foody, G. M. (2002) · Remote Sensing of Environment · doi:10.1016/s0034-4257(01)00295-4 · Cited by 4,655
Assessing the Accuracy of Remotely Sensed Data
Congalton, R. G., & Green, K. (1998) · Mapping sciences series/Mapping science · doi:10.1201/9781420048568 · Cited by 3,434
Assessing the Accuracy of Remotely Sensed Data: Principles and Practices
Foody, G. M. (2010) · The Photogrammetric Record · doi:10.1111/j.1477-9730.2010.00574_2.x · Cited by 2,836
American Journal of Neuroradiology
(2020) · American Journal of Neuroradiology · doi:10.3174/ajnr · Cited by 2,468
The Abuse of Power: The Pervasive Fallacy of Power Calculations for Data Analysis
Hoenig, J. M., & Heisey, D. M. (2001) · The American Statistician · doi:10.1198/000313001300339897 · Cited by 2,046
Statistics notes: Absence of evidence is not evidence of absence
Altman, D. G., & Bland, J. M. (1995) · BMJ · doi:10.1136/bmj.311.7003.485 · Cited by 1,946
Selecting and interpreting measures of thematic classification accuracy
Stehman, S. V. (1997) · Remote Sensing of Environment · doi:10.1016/s0034-4257(97)00083-7 · Cited by 1,745
Thematic Map Comparison
Foody, G. M. (2004) · Photogrammetric Engineering & Remote Sensing · doi:10.14358/pers.70.5.627 · Cited by 1,065
A coefficient of agreement as a measure of thematic classification accuracy.
Rosenfield, G. H., & Fitzpatrick-Lins, K. (1986) · Photogrammetric Engineering & Remote Sensing · Cited by 936
Classification and feature extraction for remote sensing images from urban areas based on morphological transformations
Benediktsson, J. A., Pesaresi, M., & Arnason, K. (2003) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2003.814625 · Cited by 767
Sample Size Determination
Dell, R. B., Holleran, S., & Ramakrishnan, R. (2002) · ILAR Journal · doi:10.1093/ilar.43.4.207 · Cited by 758
Assessing Landsat classification accuracy using discrete multivariate analysis statistical techniques.
Congalton, R. G., Oderwald, R. G., & Mead, R. A. (1983) · University of New Hampshire Scholars Repository (University of New Hampshire at Manchester) · Cited by 680
PCA‐based land‐use change detection and analysis using multitemporal and multisensor satellite data
Deng, J., Wang, K., Deng, Y., & Qi, G. J. (2008) · International Journal of Remote Sensing · doi:10.1080/01431160801950162 · Cited by 659
The Use of Predicted Confidence Intervals When Planning Experiments and the Misuse of Power When Interpreting Results
Goodman, S. N., & Berlin, J. A. (1994) · Annals of Internal Medicine · doi:10.7326/0003-4819-121-3-199408010-00008 · Cited by 658
Comparative assessment of the measures of thematic classification accuracy
Liu, C., Frazier, P. S., & Kumar, L. (2006) · Remote Sensing of Environment · doi:10.1016/j.rse.2006.10.010 · Cited by 465
Retrospective Power Analysis
Thomas, L. (1997) · Conservation Biology · doi:10.1046/j.1523-1739.1997.96102.x · Cited by 404
A combined fuzzy pixel-based and object-based approach for classification of high-resolution multispectral data over urban areas
Shackelford, A. K., & Davis, C. H. (2003) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2003.815972 · Cited by 368
Harshness in image classification accuracy assessment
Foody, G. M. (2008) · International Journal of Remote Sensing · doi:10.1080/01431160701442120 · Cited by 358
Multispectral landuse classification using neural networks and support vector machines: one or the other, or both?
Dixon, B., & Candade, N. (2007) · International Journal of Remote Sensing · doi:10.1080/01431160701294661 · Cited by 308
Meta-analysis of multiple outcomes by regression with random effects
Berkey, C. S., Hoaglin, D. C., Antczak‐Bouckoms, A., Mosteller, F. C., & Colditz, G. A. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19981130)17:22 · Cited by 274
Sub‐pixel mapping of rural land cover objects from fine spatial resolution satellite sensor imagery using super‐resolution pixel‐swapping
Thornton, M. W., Atkinson, P. M., & Holland, D. A. (2006) · International Journal of Remote Sensing · doi:10.1080/01431160500207088 · Cited by 263
Confidence intervals are a more useful complement to nonsignificant tests than are power calculations
Colegrave, N. (2003) · Behavioral Ecology · doi:10.1093/beheco/14.3.446 · Cited by 251
Classification accuracy comparison: Hypothesis tests and the use of confidence intervals in evaluations of difference, equivalence and non-inferiority
Foody, G. M. (2009) · Remote Sensing of Environment · doi:10.1016/j.rse.2009.03.014 · Cited by 232
Land cover classification in the Argentine Pampas using multi-temporal Landsat TM data
Guerschman, J. P., Paruelo, J. M., Bella, C. M. D., Giallorenzi, M. C., & Pacín, F. (2003) · International Journal of Remote Sensing · doi:10.1080/0143116021000021288 · Cited by 223
Sample Survey: Principles and Methods.
RS, & Barnett, V. (1995) · Journal of the American Statistical Association · doi:10.2307/2291109 · Cited by 209
Sample Size for Testing Differences in Proportions for the Paired-Sample Design
Connor, R. (1987) · Biometrics · doi:10.2307/2531961 · Cited by 189
Comparing accuracy assessments to infer superiority of image classification methods
de Leeuw, J., Jia, H., Yang, L. X., Liu, X., Schmidt, K. S., & Skidmore, A. K. (2006) · International Journal of Remote Sensing · doi:10.1080/01431160500275762 · Cited by 166
Basic probability sampling designs for thematic map accuracy assessment
Stehman, S. V. (1999) · International Journal of Remote Sensing · doi:10.1080/014311699212100 · Cited by 162
No Significant Difference?
Ewell, P. T. (1999) · Change The Magazine of Higher Learning · doi:10.1080/00091389909602682 · Cited by 141
The Matched Pairs Design in the Case of All-or-None Responses
Miettinen, O. S. (1968) · Biometrics · doi:10.2307/2528039 · Cited by 133
Sampling designs to test land-use map accuracy
HAY, A. M. (1979) · Cited by 128
An accuracy assessment framework for large‐area land cover classification products derived from medium‐resolution satellite data
Wulder, M. A., Franklin, S. E., White, J. C., Linke, J., & Magnussen, S. (2005) · International Journal of Remote Sensing · doi:10.1080/01431160500185284 · Cited by 118
Sample Survey Principles and Methods.
Skinner, C. J., & Barnett, V. (1992) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2982900 · Cited by 113
A confidence interval approach to data analysis
Stefano, J. D. (2003) · Forest Ecology and Management · doi:10.1016/s0378-1127(03)00331-1 · Cited by 110
Determination of sample size.
Naing, N. N. (2003) · PubMed · Cited by 106
RVM‐based multi‐class classification of remotely sensed data
Foody, G. M. (2008) · International Journal of Remote Sensing · doi:10.1080/01431160701822115 · Cited by 84
Practical Sampling Techniques
Som, R. K. (1995) · CRC Press eBooks · doi:10.1201/9781482273465 · Cited by 83
Expert system classification of urban land use/cover for Delhi, India
Wentz, E. A., Nelson, D., Rahman, A., Stefanov, W. L., & Roy, S. S. (2008) · International Journal of Remote Sensing · doi:10.1080/01431160801905497 · Cited by 77
Interpreting Null Results: Improving Presentation and Conclusions with Confidence Intervals 1
Aberson, C. L. (2002) · Cited by 74
A TREE-BASED METHOD OF ANALYSIS FOR PROSPECTIVE STUDIES
Heping Zhang, D., Holford, T. R., & Bracken, M. B. (1996) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19960115)15:1 · Cited by 62
Multisource Classification Using Support Vector Machines
Watanachaturaporn, P., Arora, M. K., & Varshney, P. K. (2008) · Photogrammetric Engineering & Remote Sensing · doi:10.14358/pers.74.2.239 · Cited by 59
SVM‐based segmentation and classification of remotely sensed data
Lizarazo, I. (2008) · International Journal of Remote Sensing · doi:10.1080/01431160802326081 · Cited by 55
An improved SVM method P‐SVM for classification of remotely sensed data
Zhang, R., & Ma, J. (2008) · International Journal of Remote Sensing · doi:10.1080/01431160802220151 · Cited by 54
Using a hybrid fuzzy classifier (HFC) to map typical grassland vegetation in Xilin River Basin, Inner Mongolia, China
Sha, Z., Bai, Y., Xie, Y., Yu, M., & Zhang, L. (2008) · International Journal of Remote Sensing · doi:10.1080/01431160701408436 · Cited by 52
Sample size
Carlin, J., & Doyle, L. (2002) · Journal of Paediatrics and Child Health · doi:10.1046/j.1440-1754.2002.00855.x · Cited by 51
A multiscale feature fusion approach for classification of very high resolution satellite imagery based on wavelet transform
Huang, X., Zhang, L., & Li, P. (2008) · International Journal of Remote Sensing · doi:10.1080/01431160802139922 · Cited by 50
Mapping an inland wetland complex using hyperspectral imagery
Jollineau, M., & Howarth, P. J. (2008) · International Journal of Remote Sensing · doi:10.1080/01431160701469099 · Cited by 49
Evaluation of global land cover data sets over the tundra–taiga transition zone in northernmost Finland
Heiskanen, J. (2008) · International Journal of Remote Sensing · doi:10.1080/01431160701871104 · Cited by 46
Assessing the impact of the 2004 tsunami on mangroves using remote sensing and GIS techniques
Sirikulchayanon, P., Sun, W., & Oyana, T. J. (2008) · International Journal of Remote Sensing · doi:10.1080/01431160701646332 · Cited by 46
Accuracy Assessment
Stehman, S. V., & Foody, G. M. (2013) · Sage eBooks · doi:10.4135/9780857021052.n21 · Cited by 44
Long‐term monitoring of land cover changes based on Landsat imagery to improve hydrological modelling in West Africa
Ruelland, D., Dezetter, A., Puech, C., & Ardoin‐Bardin, S. (2008) · International Journal of Remote Sensing · doi:10.1080/01431160701758699 · Cited by 43
Using classification error matrices to improve the accuracy of weighted land-cover models
Prisley, S. P., & Smith, J. L. (1987) · Cited by 42
Technical Note. Mapping seagrass beds using IKONOS satellite image and side scan sonar measurements: a Japanese case study
Sagawa, T., Mikami, A., Komatsu, T., Kosaka, N., Kosako, A., Miyazaki, S., & Takahashi, M. (2007) · International Journal of Remote Sensing · doi:10.1080/01431160701269028 · Cited by 40
Are there any differences? A non-sensical question in ecology
Martínez‐Abraín, A. (2007) · Acta Oecologica · doi:10.1016/j.actao.2007.04.003 · Cited by 37
Evaluation of segment‐based gap‐filled Landsat ETM+ SLC‐off satellite data for land cover classification in southern Saskatchewan, Canada
Bédard, F., Reichert, G., Dobbins, R., & Trépanier, I. (2008) · International Journal of Remote Sensing · doi:10.1080/01431160701281064 · Cited by 37
Confidence intervals and sample sizes: don't throw out all your old sample size tables.
Daly, L. (1991) · BMJ · doi:10.1136/bmj.302.6772.333 · Cited by 36
Test Statistic and Sample Size for a Two-Sample McNemar Test
Feuer, E. J., & Kessler, L. G. (1989) · Biometrics · doi:10.2307/2531505 · Cited by 36
Near‐infrared discrimination of leafless saltcedar in wintertime Landsat TM
Groeneveld, D. P., & Watson, R. P. (2008) · International Journal of Remote Sensing · doi:10.1080/01431160701711078 · Cited by 34
Phase correlation based redundancy removal in feature weighting band selection for hyperspectral images
Demir, B., & Ertürk, Ş. M. (2008) · International Journal of Remote Sensing · doi:10.1080/01431160701802471 · Cited by 30
Statistics for clinicians: 7: Sample size
Carlin, J. B., & Doyle, L. W. (2002) · Journal of Paediatrics and Child Health · Cited by 28
Confidence Intervals from the Difference between Two Correlated Proportions
Lloyd, C. J. (1990) · Journal of the American Statistical Association · doi:10.1080/01621459.1990.10474989 · Cited by 27
Land use studies in drylands: an evaluation of object‐oriented classification of very high resolution panchromatic imagery
Elmqvist, B., Ardö, J., & Olsson, L. (2008) · International Journal of Remote Sensing · doi:10.1080/01431160802238419 · Cited by 24
Design, analysis, and inference for studies comparing thematic accuracy of classified remotely sensed data: a special case of map comparison
Stehman, S. V. (2006) · Journal of Geographical Systems · doi:10.1007/s10109-006-0022-0 · Cited by 19
Artificial immune‐based supervised classifier for land‐cover classification
Pal, M. (2008) · International Journal of Remote Sensing · doi:10.1080/01431160701408402 · Cited by 18
Bayesian Multi‐net Classifier for classification of remote sensing data
Ouyang, Y., Ma, J., & Dai, Q. (2006) · International Journal of Remote Sensing · doi:10.1080/01431160600794605 · Cited by 16
Landscape Complexity and Remote Classification in Eastern Coastal Mexico: Applications of Landsat‐7 ETM+Data
Crews‐Meyer, K. A., Hudson, P. F., & Colditz, R. R. (2004) · Geocarto International · doi:10.1080/10106040408542298 · Cited by 15
No significant difference ... Says who?
Trout, A. T., Kaufmann, T. J., & Kallmes, D. F. (2007) · PubMed · Cited by 14
Statistics for Clinicians
Bahna, S. L. (2009) · Annals of Allergy Asthma & Immunology · doi:10.1016/s1081-1206(10)60812-5 · Cited by 13
Sample-size needs for forestry herbicide trials
Zedaker, S. M., Grégoire, T. G., & Miller, J. H. (1993) · Canadian Journal of Forest Research · doi:10.1139/x93-268 · Cited by 12
Some Practical Considerations and a Crude Formula for Estimating Sample Size for McNemar’s Test
Lehr, R. G. (2001) · Drug Information Journal · doi:10.1177/009286150103500419 · Cited by 7
Untitled work
Buyse, M. (1999) · doi:10.1002/(sici)1097-0258(19991230)18:24

Research works printed but not linked: 7

Aberson C., 2002, Journal of Articles in Support of the Null Hypothesis, 1, 36
Cochran W. G., 1977, Sampling Techniques
Congalton R. G., 1983, Photogrammetric Engineering and Remote Sensing, 49, 1671
Hay A. M., 1979, Photogrammetric Engineering and Remote Sensing, 45, 529
Prisley S. P., 1987, Photogrammetric Engineering and Remote Sensing, 53, 1259
Rosenfield G. H., 1986, Photogrammetric Engineering and Remote Sensing, 52, 223
Saeed N., 2006, European Journal of Scientific Research, 14, 319

Sorted by citations

An assessment of the effectiveness of a random forest classifier for land-cover classification
Rodriguez-Galiano, V. F., Ghimire, B., Rogan, J. S., Chica‐Olmo, M., & Rigol-Sánchez, J. P. (2011) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2011.11.002 · Cited by 3,068
GLC_FCS30: global land-cover product with fine classification system at 30 m using time-series Landsat imagery
Zhang, X., Liu, L., Chen, X., Gao, Y., Xie, S., & Mi, J. (2021) · Earth system science data · doi:10.5194/essd-13-2753-2021 · Cited by 1,136
The global distribution and trajectory of tidal flats
Murray, N., Phinn, S., DeWitt, M., Ferrari, R., Johnston, R., Lyons, M., Clinton, N., Thau, D., & Fuller, R. A. (2018) · Nature · doi:10.1038/s41586-018-0805-8 · Cited by 1,120
A comparison of pixel-based and object-based image analysis with selected machine learning algorithms for the classification of agricultural landscapes using SPOT-5 HRG imagery
Duro, D. C., Franklin, S. E., & Dubé, M. G. (2011) · Remote Sensing of Environment · doi:10.1016/j.rse.2011.11.020 · Cited by 1,020
Key issues in rigorous accuracy assessment of land cover products
Stehman, S. V., & Foody, G. M. (2019) · Remote Sensing of Environment · doi:10.1016/j.rse.2019.05.018 · Cited by 727

See all 248

Authors: Foody, G. M.
Affiliations: University of Nottingham (GB)
Corresponding author: Foody, Giles M.
ORCID: Foody, Giles M.
Venue: International Journal of Remote Sensing
Publisher: Informa UK Limited
Published: 2009-09-23
Volume: 30
Issue: 20
Pages: 5273-5291
ISSN: 0143-1161 (ISSN-L)
1366-5901
DOI: 10.1080/01431160903130937
OpenAlex: W2004206930
Cited by: 248
References: 74
Citations by year: 2026: 16 · 2025: 15 · 2024: 16 · 2023: 26 · 2022: 27 · 2021: 20 · 2020: 15 · 2019: 18 · 2018: 16 · 2017: 6 · 2016: 13 · 2015: 15 · 2014: 15 · 2013: 7 · 2012: 10
FWCI: 17.32
Type: journal article
Language: en
Open access: closed
Locations: International Journal of Remote Sensing (publishedVersion)
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Remote-Sensing Image Classification
Social Sciences › Geography, Planning and Development › Remote Sensing and Land Use
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Keywords: sample size determination, classification accuracy assessment, accuracy comparison, type II error, statistical power, effect size, confidence limits, remote sensing image classification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Sample size determination for image classification accuracy assessment and comparison

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks