Cited by 111Precipitation Measurement and AnalysisEnvironmental Monitoring and Data ManagementData Quality and Management

Strategies for Evaluating Quality Assurance Procedures

Durre, Imke, Menne, Matthew J., Vose, Russell S.·2008·Journal of Applied Meteorology and Climatology

Abstract

The evaluation strategies outlined in this paper constitute a set of tools beneficial to the development and documentation of robust automated quality assurance (QA) procedures. Traditionally, thresholds for the QA of climate data have been based on target flag rates or statistical confidence limits. However, these approaches do not necessarily quantify a procedure’s effectiveness at detecting true errors in the data. Rather, as illustrated by way of an “extremes check” for daily precipitation totals, information on the performance of a QA test is best obtained through a systematic manual inspection of samples of flagged values combined with a careful analysis of geographical and seasonal patterns of flagged observations. Such an evaluation process not only helps to document the effectiveness of each individual test, but, when applied repeatedly throughout the development process, it also aids in choosing the optimal combination of QA procedures and associated thresholds. In addition, the approach described here constitutes a mechanism for reassessing system performance whenever revisions are made following initial development.

Integrity signals

Registry

Crossref · 10.1175/2007jamc1706.1

Cites

14 of 14 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The quality-control process was designed so that each check has a low false-positive rate, that is, a low probability that valid observations are flagged as errors according to the evaluation approach outlined in Durre et al. (2008).

    “As described in Lawrimore et al. (2011), the quality-control process was designed so that each check has a low false-positive rate, that is, a low probability that valid observations are flagged as errors according to the evaluation approach outlined in Durre et al. (2008).”

Cites

Flagged first, then by citations

1 of the 15 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Durre, I., Menne, M. J., & Vose, R. S.
Affiliations
National Oceanic and Atmospheric Administration (US)
Corresponding author
Durre, Imke
Venue
Journal of Applied Meteorology and Climatology
Published
2008-06-01
Volume
47
Issue
6
Pages
1785-1791
ISSN
  • 1558-8424 (ISSN-L)
  • 1558-8432
OpenAlex
W2153859231
Cited by
111
References
15
Citations by year
2026: 3 · 2025: 2 · 2024: 13 · 2023: 11 · 2022: 8 · 2021: 15 · 2020: 7 · 2019: 4 · 2018: 4 · 2017: 3 · 2016: 1 · 2015: 7 · 2014: 5 · 2013: 4 · 2012: 8
FWCI
2.95
Type
journal article
Language
en
Open access
closed
Locations
Journal of Applied Meteorology and Climatology (publishedVersion)
Topics
  • Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Precipitation Measurement and Analysis
  • Physical Sciences › Earth and Planetary Sciences › Geology › Environmental Monitoring and Data Management
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Data Quality and Management
Keywords
automated quality assurance, climate data quality

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review