CoChat
Sign Up
Cited by 201Scientific Measurement and Uncertainty EvaluationClimate variability and modelsCalibration and Measurement Techniques

Uncertainty information in climate data records from Earth observation

Open access
Merchant, Christopher J., Paul, Frank, Popp, Thomas, Ablain, Michael, Bontemps, Sophie, Defourny, Pierre, Hollmann, Rainer, Lavergne, Thomas +10 more·2017·Earth System Science Data
DOI 10.5194/essd-9-511-2017

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Abstract. The question of how to derive and present uncertainty information in climate data records (CDRs) has received sustained attention within the European Space Agency Climate Change Initiative (CCI), a programme to generate CDRs addressing a range of essential climate variables (ECVs) from satellite data. Here, we review the nature, mathematics, practicalities, and communication of uncertainty information in CDRs from Earth observations. This review paper argues that CDRs derived from satellite-based Earth observation (EO) should include rigorous uncertainty information to support the application of the data in contexts such as policy, climate modelling, and numerical weather prediction reanalysis. Uncertainty, error, and quality are distinct concepts, and the case is made that CDR products should follow international metrological norms for presenting quantified uncertainty. As a baseline for good practice, total standard uncertainty should be quantified per datum in a CDR, meaning that uncertainty estimates should clearly discriminate more and less certain data. In this case, flags for data quality should not duplicate uncertainty information, but instead describe complementary information (such as the confidence in the uncertainty estimate provided or indicators of conditions violating the retrieval assumptions). The paper discusses the many sources of error in CDRs, noting that different errors may be correlated across a wide range of timescales and space scales. Error effects that contribute negligibly to the total uncertainty in a single-satellite measurement can be the dominant sources of uncertainty in a CDR on the large space scales and long timescales that are highly relevant for some climate applications. For this reason, identifying and characterizing the relevant sources of uncertainty for CDRs is particularly challenging. The characterization of uncertainty caused by a given error effect involves assessing the magnitude of the effect, the shape of the error distribution, and the propagation of the uncertainty to the geophysical variable in the CDR accounting for its error correlation properties. Uncertainty estimates can and should be validated as part of CDR validation when possible. These principles are quite general, but the approach to providing uncertainty information appropriate to different ECVs is varied, as confirmed by a brief review across different ECVs in the CCI. User requirements for uncertainty information can conflict with each other, and a variety of solutions and compromises are possible. The concept of an ensemble CDR as a simple means of communicating rigorous uncertainty information to users is discussed. Our review concludes by providing eight concrete recommendations for good practice in providing and communicating uncertainty in EO-based climate data records.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.5194/essd-9-511-2017

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Estimates of standard uncertainty are provided for every SST at all product levels.

    “Estimates of standard uncertainty4 (which may be considered as the standard deviation of the estimated error distribution) are provided for every SST39 at all product levels.”

    From Satellite-based time-series of sea-surface temperature since 1981 for climate applications (2019) · cites this paper as [39]

    The SST example in the Conclusion says standard uncertainty is given at pixel level, and §5.1 defines it as the standard deviation of the estimated error distribution. But §9 describes per-datum uncertainty estimates at all product levels as an aspiration that is challenging and not fully solved. The source therefore does not support the claim that estimates are provided for every SST at all levels. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • The quality indicator is influenced by the confidence we have that the SST uncertainty estimate for a given SST is valid.

    “The quality indicator is influenced by the confidence we have that the SST uncertainty estimate for a given SST is valid39.”

    From Satellite-based time-series of sea-surface temperature since 1981 for climate applications (2019) · cites this paper as [39]

  • There is no unique way to derive nor to present uncertainties in EO data.

    “However, there is no unique way to derive nor to present uncertainties in EO data (Merchant et al., 2017).”

    From Version 2 of the EUMETSAT OSI SAF and ESA CCI sea-ice concentration climate data records (2019) · cites this paper as Merchant et al., 2017

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 38

  1. Ablain, M., Cazenave, A., Larnicol, G., Balmaseda, M., Cipollini, P., Faugère, Y., Fernandes, M. J., Henry, O., Johannessen, J. A., Knudsen, P., Andersen, O., Legeais, J., Meyssignac, B., Picot, N., Roca, M., Rudenko, S., Scharffenberg, M. G., 10 Stammer, D., Timms, G., and Benveniste, J.: Improved sea level record over the satellite altimetry era (1993-2010) from the Climate Change Initiative project, Ocean Sci., 11, 67-82, doi:10.5194/os-11-67-2015, 2015. · DOI
  2. Barnett, T., F. Zwiers, G. Hengerl, M. Allen, T. Crowly, N. Gillett, K. Hasselmann, P. Jones, B. Santer, R. Schnur, P. Scott, K. Taylor, and S. Tett, Detecting and Attributing External Influences on the Climate System: A Review of Recent Advances. J. Climate, 18, 1291-1314, doi: 10.1175/JCLI3329.1., 2005. · DOI
  3. Bates, J. J. and J. L.Privette, A maturity model for assessing the completeness of climate data records, Eos Trans. AGU, 93(44), 441, 2012.
  4. Bates, J., J. Privette, E. Kearns, W. Glance, and X. Zhao, 2016: Sustained Production of Multidecadal Climate Records: Lessons from the NOAA Climate Data Record Program. Bull. Amer. Meteor. Soc., 97, 1573-1581, doi: 10.1175/BAMS-D- 15-00015.1., 2016. · DOI
  5. Bojinski, S., M. Verstraete, T. Peterson, C. Richter, A. Simmons, and M. Zemp, The Concept of Essential Climate Variables in Support of Climate Research, Applications, and Policy. Bull. Amer. Meteor. Soc., 95, 1431-1443, doi: 10.1175/BAMS-D- 25 13-00047.1 · DOI
  6. Buchwitz, M., Reuter M., Bovensmann H., Pillai D., Heymann J., Schneising O., Rozanov V., Krings T., Burrows J. P., Boesch H., Gerbig C., Meijer Y., and Löscher A., Carbon Monitoring Satellite (CarbonSat): assessment of atmospheric CO2 and CH4 retrieval errors by error parameterization. Atmospheric Measurement Techniques, 6, 3477-3500, doi:10.5194/amt- 6-3477-2013, 2013. · DOI
  7. Earth Syst. Sci. Data Discuss., doi:10.5194/essd-2017-16, 2017 Open Access · DOI
  8. Bulgin, C. E., Embury, O., Corlett, G. and Merchant, C. J., Independent uncertainty estimates for coefficient based sea surface temperature retrieval from the Along-Trck Scanning Radiometer instruments. Remote Sensing of Environment, 178. pp. 213-222. ISSN 0034-4257 doi: 10.1016/j.rse.2016.02.022, 2016. · DOI · Paper page
  9. Bureau International des Poids et Mesures, The International System of Units (SI), 8 th edition, available from http://www.bipm.org/en/publications/si-brochure/, 2006.
  10. Dee, D. P., Uppala, S. M., Simmons, A. J., Berrisford, P., Poli, P., Kobayashi, S., Andrae, U., Balmaseda, M. A., Balsamo, G., Bauer, P., Bechtold, P., Beljaars, A. C. M., van de Berg, L., Bidlot, J., Bormann, N., Delsol, C., Dragani, R., Fuentes, M., Geer, A. J., Haimberger, L., Healy, S. B., Hersbach, H., Hólm, E. V., Isaksen, L., Kållberg, P., Köhler, M., Matricardi, M., McNally, A. P., Monge-Sanz, B. M., Morcrette, J.-J., Park, B.-K., Peubey, C., de Rosnay, P., Tavolato, C., Thépaut, J.-N. and Vitart, F., The ERA-Interim reanalysis: configuration and performance of the data assimilation system. Q.J.R. Meteorol. 10 Soc., 137: 553-597. doi:10.1002/qj.828, 2011. · DOI · Paper page
  11. Dils, B., Buchwitz, M., Reuter, M., Schneising, O., Boesch, H., Parker, R., Guerlet, S., Aben, I., Blumenstock, T., Burrows, J. P., Butz, A., Deutscher, N. M., Frankenberg, C., Hase, F., Hasekamp, O. P., Heymann, J., De Mazière, M., Notholt, J., Sussmann, R., Warneke, T., Griffith, D., Sherlock, V., and Wunch, D.: The Greenhouse Gas Climate Change Initiative (GHG-CCI): comparative validation of GHG-CCI SCIAMACHY/ENVISAT and TANSO-FTS/GOSAT CO2 and CH4 15 retrieval algorithm products with measurements from the TCCON, Atmos. Meas. Tech., 7, 1723-1744, doi:10.5194/amt-7- 1723-2014, 2014. · DOI
  12. Embury, O. and Merchant, C. J., A reprocessing for climate of sea surface temperature from the Along-Track Scanning Radiometers: a new retrieval scheme. Remote Sensing of Environment, 116. pp. 47-61. ISSN 0034-4257 doi: 10.1016/j.rse.2010.11.020, 2012. · DOI
  13. Flannaghan, T. J., S. Fueglistaler, I. M. Held, S. Po-Chedley, B. Wyman, and M. Zhao, Tropical temperature trends in Atmospheric General Circulation Model simulations and the impact of uncertainties in observed SSTs, J. Geophys. Res. Atmos., 119, 13,327-13,337, doi:10.1002/2014JD022365, 2014. · DOI
  14. Giacco F., Thiel C., Pugliese L., Scarpetta S., and Marinaro M., Uncertainty analysis for the classification of multispectral satellite images using SVMs and SOMs. IEEE Trans Geosci Remote Sens 48(10):3769-3779, 2010.
  15. Gruber, A., Su, C.H., Zwieback, S., Crowd, W., Dorigo, W., and Wagner, W., Recent advances in (soil moisture) triple collocation analysis, International Journal of Applied Earth Observation and Geoinformation, 45: 200-211, 2016.
  16. Hollmann, R., C. J. Merchant, R. Saunders, C. Downy, M. Buchwitz, A. Cazenave, E. Chuvieco, P. Defourny, G. de Leeuw, 30 R. Forsberg, T. Holzer-Popp, F. Paul, S. Sandven, S. Sathyendranath, M. van Roozendael, and W. Wagner, The ESA Climate Change Initiative: Satellite Data Records for Essential Climate Variables. Bull. Amer. Meteor. Soc., 94, 1541- 1552, doi: 10.1175/BAMS-D-11-00254. · DOI
  17. 2013. Earth Syst. Sci. Data Discuss., doi:10.5194/essd-2017-16, 2017 Open Access Joint Committee for Guides in Metrology, Evaluation of measurement data -Guide to the expression of uncertainty in measurement, JCGM 100:2008, available from http://www.bipm.org/en/publications/guides/gum.html, 2008. · DOI
  18. Immler, F. J., Dykema, J., Gardiner, T., Whiteman, D. N., Thorne, P. W., and Vömel, H.: Reference Quality Upper-Air Measurements: guidance for developing GRUAN data products, Atmos. Meas. Tech., 3, 1217-1231, doi:10.5194/amt-3- 1217-2010, 2010. · DOI
  19. Intergovernmental Panel on Climate Change, Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovern-mental Panel on Climate Change [Stocker, T.F., D. Qin, G.-K. Plattner, M. Tignor, S.K. Allen, J. Boschung, A. Nauels, Y. Xia, V. Bex and P.M. Midgley (eds.)]. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA, 1535 pp., 2013.
  20. Kennedy, J. J., A review of uncertainty in in situ measurements and data sets of sea surface temperature, Rev. 10 Geophys., 52, 1-32, doi:10.1002/2013RG000434, 2014. · DOI
  21. Loosvelt, L., Peters, J., Skriver, H., De Baets, B., Verhoest, N.E., Impact of reducing polarimetric SAR input on the uncertainty of crop classifications based on the random forests algorithm. IEEE Trans Geosci Remote Sens 50(10):4185- 4200, 2012.
  22. Mahlstein, I., G. Hegerl, and S. Solomon, Emerging local warming signals in observational data, Geophys. Res. Lett., 39, 15 L21711, doi:10.1029/2012GL053952, 2012. · DOI
  23. Kobayashi, S., Ota, Y., Harada, Y., Ebita, A., Moriya, M., Onoda, H., Onogi, K., Kamahori, H., Kobayashi, C., Endo, H., Miyaoka, K., Takahashi, K., The JRA-55 Reanalysis: General Specifications and Basic Characteristics. Journal of the Meteorological Society of Japan, 93 (1), 5-48, 2015
  24. Massonnet, F., Bellprat, O., Guemas, V., and Doblas-Reyes, F. J., Using climate models to estimate the quality of global 20 observational data sets, Science, Vol 354, Issue 6211, pp452-455, doi: 10.1126/science.aaf6369, 2016. · DOI
  25. McColl K.A., Vogelzang J., Konings A. G., Entekhabi D., Piles M., Stoffelen A., Extended triple collocation: Estimating errors and correlation coefficients with respect to an unknown target. Geophysical Research Letters 2014; 41: 6229-6236, 2014.
  26. Merchant, C. J. and Embury, O., Simulation and inversion of satellite thermal measurements. In: Zibordi, G., Donlon, C. J. 25 and Parr, A. C. (eds.) Optical radiometry for ocean climate measurements. Experimental methods in the physical sciences, 47 (47). Academic Press, pp. 489-526. ISBN 9780124170117 doi: 10.1016/B978-0-12-417011-7.00015-5, 2014. · DOI
  27. Merchant, C. J. and Le Borgne, P., Retrieval of sea surface temperature from space based on modeling of infrared radiative transfer: capabilities and limitations. Journal of Atmospheric and Oceanic Technology, 21 (11). pp. 1734-1746. ISSN 1520- 0426 doi: 10.1175/JTECH1667.1, 2004. · DOI
  28. Mittaz J., Woolliams, E. and Merchant, C. J., Applying Principles of Metrology to Historical Earth Observations from Satellites, Metrologia, in preparation, 2017. Earth Syst. Sci. Data Discuss., doi:10.5194/essd-2017-16, 2017 Open Access · DOI
  29. Morice, C. P., J. J. Kennedy, N. A. Rayner, and P. D. Jones, Quantifying uncertainties in global and regional temperature change using an ensemble of observational estimates: The HadCRUT4 data set, J. Geophys. Res., 117, D08101, doi:10.1029/2011JD017187, 2012. · DOI · Paper page
  30. Notz, D., How well must climate models agree with observations?, Phil. Trans. R. Soc. A 2015 373 20140164, DOI: 10.1098/rsta.2014.0164, 2015. · DOI
  31. Rayner, N. A., C. J. Merchant, and G. K. Corlett, Communicating uncertainties in sea surface temperature, Eos, 96, doi:10.1029/2015EO030289, 2015. · DOI
  32. Reuter, M., Bösch, H., Bovensmann, H., Bril, A., Buchwitz, M., Butz, A., Burrows, J. P., O'Dell, C. W., Guerlet, S., Hasekamp, O., Heymann, J., Kikuchi, N., Oshchepkov, S., Parker, R., Pfeifer, S., Schneising, O., Yokota, T., and Yoshida, Y.: A joint effort to deliver satellite retrieved atmospheric CO2 concentrations for surface flux inversions: the ensemble 10 median algorithm EMMA, Atmos. Chem. Phys., 13, 1771-1780, doi:10.5194/acp-13-1771-2013, 2013. · DOI
  33. Reuter M., Buchwitz M., Hilker M., Heymann J., Schneising O., Pillai D., Bovensmann H., Burrows J. P., Bösch H., Parker R., Butz A., Hasekamp O., O'Dell C. W., Yoshida Y., Gerbig C., Nehrkorn T., Deutscher N. M., Warneke T., Notholt J., Hase F., Kivi R., Sussmann R., Machida T., Matsueda H., and Sawa Y., Satellite-inferred European carbon sink larger than expected, Atmos. Chem. Phys., doi:10.5194/acp-14-13739-2014, 14, 13739-13753, 2014. · DOI
  34. Sofieva, V. F., Tamminen, J., Kyrölä, E., Laeng, A., von Clarmann, T., Dalaudier, F., Hauchecorne, A., Bertaux, J.-L., Barrot, G., Blanot, L., Fussen, D., and Vanhellemont, F.: Validation of GOMOS ozone precision estimates in the 20 stratosphere, Atmos. Meas. Tech., 7, 2147-2158, doi:10.5194/amt-7-2147-2014, 2014. · DOI
  35. Stoffelen A., Toward the true near-surface wind speed: Error modeling and calibration using triple collocation. Journal of Geophysical Research-Oceans, 103: 7755-7766, 1998.
  36. Wimmer, W., Robinson I. S., and Donlon, C. J., Long-term validation of AATSR SST data products using shipborne radiometry in the Bay of Biscay and English Channel, Remote Sensing of Environment, Volume 116, Pages 17-31, ISSN 25 0034-4257, http://dx.doi.org/10.1016/j.rse.2011.03.022., 2012. · DOI
  37. Woolliams E., Mittaz J., Merchant C. J. and Dilo A., Harmonization and Recalibration: A FIDUCEO perspective, Global Space-based Inter-calibration System Quarterly, Vol. 10 (2), doi: 10.7289/V5GT5K7S, 2016. · DOI
  38. World Meteorological Organisation and Bureau Internationale de Poids et Mesures, Measurement Challenges for Global Observation Systems for Climate Change Monitoring: Traceability, Stability and Uncertainty, WMO/TD-No. 1557, Rapport 30 BIPM-2010/08, ISBN 13 978-92-822-2239-3, available from http://www.bipm.org/en/conference-centre/bipm- workshops/wmo-bipm_workshop/, 2010. Earth Syst. Sci. Data Discuss., doi:10.5194/essd-2017-16, 2017 Open Access · DOI

Cited by

Sorted by citations

Authors
Merchant, C. J., Paul, F., Popp, T., Ablain, M., Bontemps, S., Defourny, P., Hollmann, R., Lavergne, T., Laeng, A., de Leeuw, G., Mittaz, J., Poulsen, C., Povey, A. C., Reuter, M., Sathyendranath, S., Sandven, S., Sofieva, V. F., & Wagner, W.
Affiliations
  • National Centre for Earth Observation (GB)
  • University of Zurich (CH)
  • Deutsches Zentrum für Luft- und Raumfahrt e. V. (DLR) (DE)
  • UCLouvain (BE)
  • Deutscher Wetterdienst (DE)
  • Norwegian Meteorological Institute (NO)
  • Karlsruhe Institute of Technology (DE)
  • National Physical Laboratory (GB)
  • Rutherford Appleton Laboratory (GB)
  • University of Oxford (GB)
  • Plymouth Marine Laboratory (GB)
  • Nansen Environmental and Remote Sensing Center (NO)
  • Finnish Meteorological Institute (FI)
  • TU Wien (AT)
Corresponding author
  • Merchant, Christopher J.
  • Paul, Frank
  • Popp, Thomas
  • Bontemps, Sophie
  • Defourny, Pierre
  • Hollmann, Rainer
  • Lavergne, Thomas
  • Laeng, Alexandra
  • Mittaz, Jonathan
  • Poulsen, Caroline
  • Povey, Adam C.
  • Sathyendranath, Shubha
  • Sandven, Stein
  • Sofieva, Viktoria F.
  • Wagner, Wolfgang
ORCID
Venue
Earth System Science Data
Publisher
Copernicus GmbH
Published
2017-07-25
Volume
9
Issue
2
Pages
511-527
ISSN
  • 1866-3508 (ISSN-L)
  • 1866-3516
DOI
10.5194/essd-9-511-2017
OpenAlex
W2592829405
Cited by
201
References
0
Citations by year
2026: 16 · 2025: 19 · 2024: 23 · 2023: 29 · 2022: 27 · 2021: 42 · 2020: 12 · 2019: 20 · 2018: 8 · 2017: 5
FWCI
11.01
Type
journal article
Language
en
Open access
Licence
Locations
  • Earth system science data (publishedVersion, cc-by)
  • Marine Biological Association (acceptedVersion)
  • Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, cc-by)
  • CentAUR (University of Reading) (acceptedVersion)
  • Repository KITopen (Karlsruhe Institute of Technology) (submittedVersion, cc-by)
  • elib (German Aerospace Center) (submittedVersion)
  • Digital Access to Libraries (Université catholique de Louvain (UCL), l'Université de Namur (UNamur) and the Université Saint-Louis (USL-B)) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • ePubs (Science and Technology Facilities Council, Research Councils UK) (submittedVersion)
  • Zurich Open Repository and Archive (University of Zurich) (submittedVersion, cc-by)
  • Zurich Open Repository and Archive (University of Zurich)
  • KITopen (cc-by)
Funders
European Space Agency (Climate Change Initiative (GMECV))
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Scientific Measurement and Uncertainty Evaluation
  • Physical Sciences › Environmental Science › Global and Planetary Change › Climate variability and models
  • Physical Sciences › Engineering › Aerospace Engineering › Calibration and Measurement Techniques
Keywords
climate data record, uncertainty quantification, satellite Earth observation, essential climate variables, climate change initiatives, data quality flags, error sources, error propagation, uncertainty communication

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review