CoChat

Benchmarking homogenization algorithms for monthly data

Open access
Venema, V. K. C., Mestre, O., Aguilar, E., Auer, I., Guijarro, J. A., Domonkos, P., Vertacnik, G., Szentimrey, T., Stepanek, P., Zahradnicek, P., Viarre, J., Müller-Westermeier, G., Lakatos, M., Williams, C. N., Menne, M. J., Lindau, R., Rasol, D., Rustemeier, E., Kolokythas, K., Marinova, T., Andresen, L., Acquaotta, F., Fratianni, S., Cheval, S., Klancar, M., Brunetti, M., Gruber, C., Prohom Duran, M., Likso, T., Esteban, P., Brandsma, T. · 2012 · Climate of the Past

Published version of a preprint

The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.

Abstract

The COST (European Cooperation in Science and Technology) Action ES0601: advances in homogenization methods of climate series: an integrated approach (HOME) has executed a blind intercomparison and validation study for monthly homogenization algorithms. Time series of monthly temperature and precipitation were evaluated because of their importance for climate studies and because they represent two important types of statistics (additive and multiplicative). The algorithms were validated against a realistic benchmark dataset. The benchmark contains real inhomogeneous data as well as simulated data with inserted inhomogeneities. Random independent break-type inhomogeneities with normally distributed breakpoint sizes were added to the simulated datasets. To approximate real world conditions, breaks were introduced that occur simultaneously in multiple station series within a simulated network of station data. The simulated time series also contained outliers, missing data periods and local station trends. Further, a stochastic nonlinear global (network-wide) trend was added. Participants provided 25 separate homogenized contributions as part of the blind study. After the deadline at which details of the imposed inhomogeneities were revealed, 22 additional solutions were submitted. These homogenized datasets were assessed by a number of performance metrics including (i) the centered root mean square error relative to the true homogeneous value at various averaging scales, (ii) the error in linear trend estimates and (iii) traditional contingency skill scores. The metrics were computed both using the individual station series as well as the network average regional series. The performance of the contributions depends significantly on the error metric considered. Contingency scores by themselves are not very informative. Although relative homogenization algorithms typically improve the homogeneity of temperature data, only the best ones improve precipitation data. Training the users on homogenization software was found to be very important. Moreover, state-of-the-art relative homogenization algorithms developed to work with an inhomogeneous reference are shown to perform best. The study showed that automatic algorithms can perform as well as manual ones.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.5194/cp-8-89-2012

Cites

50 of 50 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Scatterplot of the centered RMSE before and after homogenization for selected contributions. The squares display the errors of the stations; the dots show the errors of the network mean (regional climate) time series. Points on the bisect indicate no change, above the bisect the data is made more inhomogeneous, while below the bisect homogenization improved the homogeneity of the data.
Fig. 1 Scatterplot of the centered RMSE before and after homogenization for selected contributions. The squares display the errors of the stations; the dots show the errors of the network mean (regional climate) time series. Points on the bisect indicate no change, above the bisect the data is made more inhomogeneous, while below the bisect homogenization improved the homogeneity of the data.CC-BY · From the PDF
Image 2
CC-BY · From the PDF
Figure 3 Boxplot of the decadal CRMSE of the complete contributions, for temperature (top) and precipitation (bottom). For comparison the error metric for the inhomogeneous data is plotted at the top. The abscissa is the same as the one of Fig. 2 to emphasis the smaller errors and larger improvement over the inhomogeneous data for the decadal data. The conventions of the boxplots are explained in Fig. 2. Fig. 3. Boxplot of the decadal CRMSE of the complete contributions, for temperature (top) and precipitation (bottom). For comparison the error metric for the inhomogeneous data is plotted at the top. The abscissa is the same as the one of Fig. 2 to emphasis the smaller errors and larger improvement over the inhomogeneous data for the decadal data. The conventions of the boxplots are explained in Fig. 2.
Figure 3 Boxplot of the decadal CRMSE of the complete contributions, for temperature (top) and precipitation (bottom). For comparison the error metric for the inhomogeneous data is plotted at the top. The abscissa is the same as the one of Fig. 2 to emphasis the smaller errors and larger improvement over the inhomogeneous data for the decadal data. The conventions of the boxplots are explained in Fig. 2. Fig. 3. Boxplot of the decadal CRMSE of the complete contributions, for temperature (top) and precipitation (bottom). For comparison the error metric for the inhomogeneous data is plotted at the top. The abscissa is the same as the one of Fig. 2 to emphasis the smaller errors and larger improvement over the inhomogeneous data for the decadal data. The conventions of the boxplots are explained in Fig. 2.CC-BY · From the PDF
Figure 4 The average temporal behavior over all contributions of the monthly, yearly and decadal CRMSE of the station data for temperature (top) and precipitation (bottom). The stripped vertical line at 1925 indicates the end of the period in which not all stations have started observations. The two stripped vertical lines at 1940 and 1945 indicate the period of the Second World War with much missing data. Fig. 4. The average temporal behavior over all contributions of the monthly, yearly and decadal CRMSE of the station data for temperature (top) and precipitation (bottom). The stripped vertical line at 1925 indicates the end of the period in which not all stations have started observations. The two stripped vertical lines at 1940 and 1945 indicate the period of the Second World War with much missing data.
Figure 4 The average temporal behavior over all contributions of the monthly, yearly and decadal CRMSE of the station data for temperature (top) and precipitation (bottom). The stripped vertical line at 1925 indicates the end of the period in which not all stations have started observations. The two stripped vertical lines at 1940 and 1945 indicate the period of the Second World War with much missing data. Fig. 4. The average temporal behavior over all contributions of the monthly, yearly and decadal CRMSE of the station data for temperature (top) and precipitation (bottom). The stripped vertical line at 1925 indicates the end of the period in which not all stations have started observations. The two stripped vertical lines at 1940 and 1945 indicate the period of the Second World War with much missing data.CC-BY · From the PDF
Fig. 5 Trends in the original data versus the trends in the inhomogeneous or homogenized data. The top row shows trends for selected temperature contributions, the bottom row for precipitation. The open symbols denote the trends of homogenized stations, the closed black symbols the trend of the homogenized regional network averaged trend; every network has its own symbol, which shows that station trend errors are correlated. The vertical grey lines run from the trend in the inhomogeneous data to the trend in the homogenized data.
Fig. 5 Trends in the original data versus the trends in the inhomogeneous or homogenized data. The top row shows trends for selected temperature contributions, the bottom row for precipitation. The open symbols denote the trends of homogenized stations, the closed black symbols the trend of the homogenized regional network averaged trend; every network has its own symbol, which shows that station trend errors are correlated. The vertical grey lines run from the trend in the inhomogeneous data to the trend in the homogenized data.CC-BY · From the PDF
Image 6
CC-BY · From the PDF
Fig. 7 Scatterplot of the probability of false detection against the probability of detection for the surrogate temperature dataset. The slant lines represent the Peirce (true) skill score. The crosses are the mean detection values of all complete surrogate temperature contributions. For the abbreviations see Table 9.
Fig. 7 Scatterplot of the probability of false detection against the probability of detection for the surrogate temperature dataset. The slant lines represent the Peirce (true) skill score. The crosses are the mean detection values of all complete surrogate temperature contributions. For the abbreviations see Table 9.CC-BY · From the PDF
Fig. 8 The temporal behavior of the true breaks and predicted breaks (in top panel) and the probability of (false) detection (bottom panel) based on the homogeneous surrogate temperature stations and all complete contributions.
Fig. 8 The temporal behavior of the true breaks and predicted breaks (in top panel) and the probability of (false) detection (bottom panel) based on the homogeneous surrogate temperature stations and all complete contributions.CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 50 references

Research works printed but not linked: 8

  1. Domonkos, P.: Testing of homogenisation methods: purposes, tools and problems of implementation. Proceedings of the 5 th Semi- nar and Quality Control in Climatological Databases, edited by:
  2. HMS: Hungarian Meteorological Service. Proceedings of the First Seminar for homogenization of Surface Climatological Data, Bu- dapest, Hungary, 6-12 October 1996, p. 44, 1996.
  3. Mestre, O.: Step-by-step procedures for choosing a model with change-points. In Proceedings of the second seminar for ho- mogenisation of surface climatological data, Budapest, Hungary, WCDMP-No.41, WMO-TD No. 962, 15-26, 1999.
  4. Szentimrey, T.: Multiple Analysis of Series for homogenization (MASH). Proceedings of the second seminar for homogenization of surface climatological data, Budapest, Hungary, WMO, WCDMP-No. 41, 27-46, 1999.
  5. Szentimrey, T.: Development of MASH homogenization pro- cedure for daily data. Proceedings of the fifth seminar for homogenization and quality control in climatological databases, Budapest, Hungary, 2006, WCDMP-No. 71, 123-130, 2008.
  6. WMO: Proceedings of the Second Seminar for homogenization of Surface Climatological Data. Budapest, Hungary, 9-13 Novem- ber 1998, p. 214, 1999.
  7. WMO: Proceedings of the Fifth Seminar for homogenization and Quality Control in Climatological Databases. Budapest, Hungary, 29 May–2 June 2006. Climate Data and Monitoring WCDMP- No 71, WMO/TD- No. 1493, 2006.
  8. WMO: Fourth Seminar for homogenization and Quality Control in Climatological Databases. Budapest, Hungary, 6-10 October 2003, WCDMP-No 56, WMO-TD No. 1236, p. 243, 2004. WMO: Proceedings of the Fifth Seminar for homogenization and Quality Control in Climatological Databases. Budapest, Hun- gary, 29 May-2 June 2006. Climate Data and Monitoring WCDMP-No 71, WMO/TD-No. 1493, 2006. WMO: Proceedings of the Sixth Seminar for homogenization and Quality Control in Climatological Databases. Budapest, Hun- gary, 26-30 2008, edited by: Lakatos, M., Szentimrey, T., Bihari, Z., and Szalai, S., WCDMP-No. 76, WMO/TD-NO. 1576, 2011.

Cited by

Items 301–400 of 407

Metadata

Authors
Venema, V. K. C., Mestre, O., Aguilar, E., Auer, I., Guijarro, J. A., Domonkos, P., Vertacnik, G., Szentimrey, T., Stepanek, P., Zahradnicek, P., Viarre, J., Müller-Westermeier, G., Lakatos, M., Williams, C. N., Menne, M. J., Lindau, R., Rasol, D., Rustemeier, E., Kolokythas, K., Marinova, T., Andresen, L., Acquaotta, F., Fratianni, S., Cheval, S., Klancar, M., Brunetti, M., Gruber, C., Prohom Duran, M., Likso, T., Esteban, P., & Brandsma, T.
Affiliations
  • University of Bonn (DE)
  • École Nationale de la Météorologie (FR)
  • Universitat Rovira i Virgili (ES)
  • Central Institution for Meteorology and Geodynamics (AT)
  • Agencia Estatal de Meteorología (ES)
  • Hungarian Meteorological Service (HU)
  • National Oceanic and Atmospheric Administration (US)
  • Croatian Meteorological and Hydrological Service (HR)
  • University of Patras (GR)
  • Bulgarian Academy of Sciences (BG)
  • Norwegian Meteorological Institute (NO)
  • University of Turin (IT)
  • National Institute for Research and Development in Environmental Protection (RO)
  • Slovenian Environment Agency (SI)
  • Institute of Atmospheric Sciences and Climate (IT)
  • Universitat de Barcelona (ES)
  • Royal Netherlands Meteorological Institute (NL)
Venue
Climate of the Past
Published
2012-01-10
Volume
8
Issue
1
Pages
89-115
ISSN
  • 1814-9324 (ISSN-L)
  • 1814-9332
OpenAlex
W2734645652
Cited by
368
References
50
Citations by year
2026: 14 · 2025: 21 · 2024: 19 · 2023: 22 · 2022: 23 · 2021: 23 · 2020: 20 · 2019: 21 · 2018: 28 · 2017: 25 · 2016: 35 · 2015: 32 · 2014: 33 · 2013: 25 · 2012: 25
FWCI
30.58
Type
journal article
Language
en
Open access
Locations
  • Climate of the past (publishedVersion, cc-by)
  • Institutional Research Information System University of Turin (University of Turin) (submittedVersion, cc-by)
  • Institutional Research Information System University of Turin (University of Turin) (submittedVersion, other-oa)
  • ASEP (submittedVersion, public-domain)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • European Commission
  • Deutsche Forschungsgemeinschaft
Topics
  • Physical Sciences › Environmental Science › Global and Planetary Change › Climate variability and models
  • Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Precipitation Measurement and Analysis
  • Physical Sciences › Environmental Science › Environmental Engineering › Urban Heat Island Mitigation
Keywords
monthly temperature data, linear trend estimation, station network

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.