Benchmarking homogenization algorithms for monthly data
Open accessPublished version of a preprint
The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
Abstract
The COST (European Cooperation in Science and Technology) Action ES0601: advances in homogenization methods of climate series: an integrated approach (HOME) has executed a blind intercomparison and validation study for monthly homogenization algorithms. Time series of monthly temperature and precipitation were evaluated because of their importance for climate studies and because they represent two important types of statistics (additive and multiplicative). The algorithms were validated against a realistic benchmark dataset. The benchmark contains real inhomogeneous data as well as simulated data with inserted inhomogeneities. Random independent break-type inhomogeneities with normally distributed breakpoint sizes were added to the simulated datasets. To approximate real world conditions, breaks were introduced that occur simultaneously in multiple station series within a simulated network of station data. The simulated time series also contained outliers, missing data periods and local station trends. Further, a stochastic nonlinear global (network-wide) trend was added. Participants provided 25 separate homogenized contributions as part of the blind study. After the deadline at which details of the imposed inhomogeneities were revealed, 22 additional solutions were submitted. These homogenized datasets were assessed by a number of performance metrics including (i) the centered root mean square error relative to the true homogeneous value at various averaging scales, (ii) the error in linear trend estimates and (iii) traditional contingency skill scores. The metrics were computed both using the individual station series as well as the network average regional series. The performance of the contributions depends significantly on the error metric considered. Contingency scores by themselves are not very informative. Although relative homogenization algorithms typically improve the homogeneity of temperature data, only the best ones improve precipitation data. Training the users on homogenization software was found to be very important. Moreover, state-of-the-art relative homogenization algorithms developed to work with an inhomogeneous reference are shown to perform best. The study showed that automatic algorithms can perform as well as manual ones.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures








Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For instance, [5, 6] demonstrated that inconsistencies in data collection practices contribute to significant biases in climate datasets, affecting the accuracy of climate modelling and predictive analyses. These biases can propagate through climate models, reducing the reliability of future projections, especially in data-scarce regions.”
From Exploratory data analysis of Burkina Faso’s Raw daily meteorological records (1981–2021): data quality insights and potential outlier screening (2026) · cites this paper as [5]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The PHA has undergone extensive evaluation and benchmarking (e.g., Menne and Williams 2009; Venema et al. 2012; Williams et al. 2012) and has been shown to be effective at finding a wide variety of shifts in realistic simulated monthly temperature datasets.”
From The Global Historical Climatology Network Monthly Temperature Dataset, Version 4 (2018) · cites this paper as Venema et al. 2012
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Long-term station records are often subjected to inhomogeneities due to factors like station relocation, observation time change, instrument change, and surrounding environment change (Venema et al., 2012).”
From SCDNA: a serially complete precipitation and temperature dataset for North America from 1979 to 2018 (2020) · cites this paper as Venema et al., 2012
The cited source says what the claim attributes to it.
In the citing paper
“Different methods could generate different estimates of inhomogeneities as shown by many comparison studies (e.g., Beaulieu et al., 2008; Reeves et al., 2007; Venema et al., 2012).”
From SCDNA: a serially complete precipitation and temperature dataset for North America from 1979 to 2018 (2020) · cites this paper as Venema et al., 2012
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting
Cleveland, W. S., & Devlin, S. J. (1988) · Journal of the American Statistical Association · doi:10.1080/01621459.1988.10478639 · Cited by 5,742
Solomon, S. (2007) · Medical Entomology and Zoology · Cited by 4,149
IPCC (2007): Climate Change The Physical Science Basis
Solomon, S. (2007) · AGU Fall Meeting Abstracts · Cited by 3,476
Observations: Surface and Atmospheric Climate Change
Trenberth, K. E., Jones, P. D., Ambenje, P., Bojariu, R., Easterling, D. R., Tank, A. K., Parker, D., Rahimzadeh, F., Renwick, J. A., Rusticucci, M., et al. (2007) · elib (German Aerospace Center) · Cited by 2,344
A homogeneity test applied to precipitation data.
Alexandersson, H. (1986) · Journal of Climatology · doi:10.1002/joc.3370060607 · Cited by 1,858
Improved Surrogate Data for Nonlinearity Tests
Schreiber, T., & Schmitz, A. (1996) · Physical Review Letters · doi:10.1103/physrevlett.77.635 · Cited by 1,506
Some methods for testing the homogeneity of rainfall records
Buishand, T. A. (1982) · Journal of Hydrology · doi:10.1016/0022-1694(82)90066-x · Cited by 1,234
HISTALP—historical instrumental climatological surface time series of the Greater Alpine Region
Auer, I., Böhm, R., Jurkovic, A., Lipa, W., Orlik, A., Potzmann, R., Schöner, W., Ungersböck, M., Matulla, C., Briffa, K., et al. (2007) · International Journal of Climatology · doi:10.1002/joc.1377 · Cited by 1,133
Self-citationHomogeneity adjustments ofin situ atmospheric climate data: a review
Peterson, T. C., Easterling, D. R., Karl, T. R., Groisman, P., Nicholls, N., Plummer, N., Torok, S., Auer, I., Boehm, R., Gullett, D., et al. (1998) · International Journal of Climatology · doi:10.1002/(sici)1097-0088(19981115)18:13<1493::aid-joc329>3.0.co;2-t · Cited by 789
Self-citationHOMOGENIZATION OF SWEDISH TEMPERATURE DATA. PART I: HOMOGENEITY TEST FOR LINEAR TRENDS
ALEXANDERSSON, H., & MOBERG, A. (1997) · International Journal of Climatology · doi:10.1002/(sici)1097-0088(199701)17:1<25::aid-joc103>3.0.co;2-j · Cited by 669
A Review and Comparison of Changepoint Detection Techniques for Climate Data
Reeves, J., Chen, J., Wang, X. L., Lund, R., & Lu, Q. Q. (2007) · Journal of Applied Meteorology and Climatology · doi:10.1175/jam2493.1 · Cited by 638
Brunetti, M., Maugeri, M., Monti, F., & Nanni, T. (2006) · International Journal of Climatology · doi:10.1002/joc.1251 · Cited by 602
Self-citationA closer look at United States and global surface temperature change
Hansen, J., Ruedy, R., Sato, M., Imhoff, M., Lawrence, W., Easterling, D., Peterson, T., & Karl, T. (2001) · Journal of Geophysical Research: Atmospheres · doi:10.1029/2001jd000354 · Cited by 580
Böhm, R., Auer, I., Brunetti, M., Maugeri, M., Nanni, T., & Schöner, W. (2001) · International Journal of Climatology · doi:10.1002/joc.689 · Cited by 434
Self-citationWang, X. L. (2008) · Journal of Applied Meteorology and Climatology · doi:10.1175/2008jamc1741.1 · Cited by 427
A new method for detecting undocumented discontinuities in climatological time series
Easterling, D. R., & Peterson, T. C. (1995) · International Journal of Climatology · doi:10.1002/joc.3370150403 · Cited by 419
Homogeneous temperature and precipitation series of Switzerland from 1864 to 2000
Begert, M., Schlegel, T., & Kirchhofer, W. (2005) · International Journal of Climatology · doi:10.1002/joc.1118 · Cited by 411
The U.S. Historical Climatology Network Monthly Temperature Data, Version 2
Menne, M. J., Williams, C. N., & Vose, R. S. (2009) · Bulletin of the American Meteorological Society · doi:10.1175/2008bams2613.1 · Cited by 300
Self-citationDetection and Correction of Artificial Shifts in Climate Series
Caussinus, H., & Mestre, O. (2004) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/j.1467-9876.2004.05155.x · Cited by 281
Self-citationConrad, V. (1946) · Weather · doi:10.1002/j.1477-8696.1946.tb00041.x · Cited by 268
A Technique for the Identification of Inhomogeneities in Canadian Temperature Series
Vincent, L. A. (1998) · Journal of Climate · doi:10.1175/1520-0442(1998)011<1094:atftio>2.0.co;2 · Cited by 268
Auer, I., Böhm, R., Jurković, A., Orlik, A., Potzmann, R., Schöner, W., Ungersböck, M., Brunetti, M., Nanni, T., Maugeri, M., et al. (2005) · International Journal of Climatology · doi:10.1002/joc.1135 · Cited by 214
Self-citationUsing benchmarking to advance research: a challenge to software engineering
Sim, S. E., Easterbrook, S., & Holt, R. C. (2003) · doi:10.5555/776816.776826 · Cited by 199
Wallén, C. C., Conrad, V., Pollak, L. W., & Wallen, C. C. (1950) · Geografiska Annaler · doi:10.2307/520284 · Cited by 198
METHODS OF COMPARING ANNUAL RAINFALL RECORDS FOR CLIMATIC PURPOSES
Craddock, J. M. (1979) · Weather · doi:10.1002/j.1477-8696.1979.tb03465.x · Cited by 193
Comparison of techniques for detection of discontinuities in temperature series.
Ducré‐Robitaille, J., Vincent, L. A., & Boulet, G. (2003) · International Journal of Climatology · doi:10.1002/joc.924 · Cited by 182
Kohler, M. A. (1949) · Bulletin of the American Meteorological Society · doi:10.1175/1520-0477-30.5.188 · Cited by 178
Perreault, L., Bernier, J., Bobée, B., & Parent, E. (2000) · Journal of Hydrology · doi:10.1016/s0022-1694(00)00270-5 · Cited by 164
Detection of Undocumented Changepoints Using Multiple Test Statistics and Composite Reference Series
Menne, M. J., & Williams, C. N. (2005) · Journal of Climate · doi:10.1175/jcli3524.1 · Cited by 157
Self-citationExposure, instrumentation, and observing practice effects on land temperature measurements
Trewin, B. (2010) · WIREs Climate Change · doi:10.1002/wcc.46 · Cited by 153
Using benchmarking to advance research: a challenge to software engineering
Sim, S. E., Easterbrook, S., & Holt, R. C. (2003) · 25th International Conference on Software Engineering, 2003. Proceedings. · doi:10.1109/icse.2003.1201189 · Cited by 149
Benchmarking the performance of pairwise homogenization of surface temperatures in the United States
Williams, C. N., Menne, M. J., & Thorne, P. W. (2012) · Journal of Geophysical Research: Atmospheres · doi:10.1029/2011jd016761 · Cited by 121
Self-citationData rescue initiatives: bringing historical climate data into the 21st century
Brunet, M., & Jones, P. (2011) · Climate Research · doi:10.3354/cr00960 · Cited by 120
On the reliability of the U.S. surface temperature record
Menne, M. J., Williams, C. N., & Palecki, M. A. (2010) · Journal of Geophysical Research: Atmospheres · doi:10.1029/2009jd013094 · Cited by 118
Self-citationŠtěpánek, P., Zahradníček, P., & Skalák, P. (2009) · Advances in Science and Research · doi:10.5194/asr-3-23-2009 · Cited by 117
A Stochastic Iterative Amplitude Adjusted Fourier Transform algorithm with improved accuracy
Venema, V., Ament, F., & Simmer, C. (2006) · Nonlinear processes in geophysics · doi:10.5194/npg-13-321-2006 · Cited by 94
Self-citationAdapted Caussinus-Mestre Algorithm for Networks of Temperature series (ACMANT)
Domonkos, P. (2011) · International Journal of Geosciences · doi:10.4236/ijg.2011.23032 · Cited by 88
Self-citationThorne, P. W., Willett, K. M., Allan, R. J., Bojinski, S., Christy, J. R., Fox, N., Gilbert, S., Jolliffe, I., Kennedy, J. J., Kent, E., et al. (2011) · Bulletin of the American Meteorological Society · doi:10.1175/2011bams3124.1 · Cited by 76
Attributes of several methods for detecting discontinuities in mean temperature series.
DeGaetano, A. T. (2006) · Journal of Climate · doi:10.1175/jcli3662.1 · Cited by 72
Intercomparison of homogenization techniques for precipitation data
Beaulieu, C., Seidou, O., Ouarda, T. B. M. J., Zhang, X., Boulet, G., & Yagouti, A. (2008) · Water Resources Research · doi:10.1029/2006wr005615 · Cited by 65
Brunet, M., Asín, J., Sigró, J., Bañón, M., García, F., Aguilar, E., Palenzuela, J. E., Peterson, T. C., & Jones, P. D. (2010) · International Journal of Climatology · doi:10.1002/joc.2192 · Cited by 64
Self-citationEfficiency evaluation for detecting inhomogeneities by objective homogenisation methods
Domonkos, P. (2011) · Theoretical and Applied Climatology · doi:10.1007/s00704-011-0399-7 · Cited by 63
Self-citationStatistical characteristics of surrogate data based on geophysical measurements
Venema, V., Bachner, S., Rust, H. W., & Simmer, C. (2006) · Nonlinear processes in geophysics · doi:10.5194/npg-13-449-2006 · Cited by 51
Self-citationBenchmarking monthly homogenization algorithmsPreprint
Venema, K., Mestre, O., Aguilar, E., et al. (2011) · doi:10.5194/cpd-7-2655-2011 · Cited by 41
Self-citationFewer jumps, less memory: Homogenized temperature records and long memory
Rust, H. W., Mestre, O., & Venema, V. K. C. (2008) · Journal of Geophysical Research Atmospheres · doi:10.1029/2008jd009919 · Cited by 37
Self-citationDomonkos, P., Poza, R., & Efthymiadis, D. A. (2011) · Advances in science and research · doi:10.5194/asr-6-7-2011 · Cited by 21
Self-citationStatistical characteristics of detectable inhomogeneities in observed meteorological time series
Domonkos, P., & Štěpánek, P. (2009) · Studia Geophysica et Geodaetica · doi:10.1007/s11200-009-0015-9 · Cited by 19
Self-citationHow do we tell which estimates of past climate change are correct?
Sherwood, S. C., Titchner, H., Thorne, P., & McCarthy, M. (2008) · International Journal of Climatology · doi:10.1002/joc.1825 · Cited by 7
Description of the COST - HOME monthly benchmark dataset and the submitted homogenized contributionsPreprint
Venema, V., Mestre, O., Aguilar, E., Auer, I., Guijarro, J. A., Domonkos, P., Vertačnik, G., Szentimrey, T., Štěpánek, P., & Zahradníček, P. (2018) · doi:10.31223/osf.io/37w5p · Cited by 5
Self-citation(2003) · doi:10.1175/1520-0442(2003)016
Research works printed but not linked: 8
- Domonkos, P.: Testing of homogenisation methods: purposes, tools and problems of implementation. Proceedings of the 5 th Semi- nar and Quality Control in Climatological Databases, edited by:
- HMS: Hungarian Meteorological Service. Proceedings of the First Seminar for homogenization of Surface Climatological Data, Bu- dapest, Hungary, 6-12 October 1996, p. 44, 1996.
- Mestre, O.: Step-by-step procedures for choosing a model with change-points. In Proceedings of the second seminar for ho- mogenisation of surface climatological data, Budapest, Hungary, WCDMP-No.41, WMO-TD No. 962, 15-26, 1999.
- Szentimrey, T.: Multiple Analysis of Series for homogenization (MASH). Proceedings of the second seminar for homogenization of surface climatological data, Budapest, Hungary, WMO, WCDMP-No. 41, 27-46, 1999.
- Szentimrey, T.: Development of MASH homogenization pro- cedure for daily data. Proceedings of the fifth seminar for homogenization and quality control in climatological databases, Budapest, Hungary, 2006, WCDMP-No. 71, 123-130, 2008.
- WMO: Proceedings of the Second Seminar for homogenization of Surface Climatological Data. Budapest, Hungary, 9-13 Novem- ber 1998, p. 214, 1999.
- WMO: Proceedings of the Fifth Seminar for homogenization and Quality Control in Climatological Databases. Budapest, Hungary, 29 May–2 June 2006. Climate Data and Monitoring WCDMP- No 71, WMO/TD- No. 1493, 2006.
- WMO: Fourth Seminar for homogenization and Quality Control in Climatological Databases. Budapest, Hungary, 6-10 October 2003, WCDMP-No 56, WMO-TD No. 1236, p. 243, 2004. WMO: Proceedings of the Fifth Seminar for homogenization and Quality Control in Climatological Databases. Budapest, Hun- gary, 29 May-2 June 2006. Climate Data and Monitoring WCDMP-No 71, WMO/TD-No. 1493, 2006. WMO: Proceedings of the Sixth Seminar for homogenization and Quality Control in Climatological Databases. Budapest, Hun- gary, 26-30 2008, edited by: Lakatos, M., Szentimrey, T., Bihari, Z., and Szalai, S., WCDMP-No. 76, WMO/TD-NO. 1576, 2011.
Cited by
The European mountain cryosphere: a review of its current state, trends, and future challenges
Beniston, M., Farinotti, D., Stoffel, M., Andreassen, L. M., Coppola, E., Eckert, N., Fantini, A., Giacona, F., Hauck, C., Huss, M., et al. (2018) · The Cryosphere · doi:10.5194/tc-12-759-2018 · Cited by 734
Alexander, L. V. (2016) · Weather and Climate Extremes · doi:10.1016/j.wace.2015.10.007 · Cited by 453
U.S. Climate Reference Network after one decade of operations: Status and assessment
Diamond, H. J., Karl, T. R., Palecki, M. A., Baker, C. B., Bell, J. E., Leeper, R. D., Easterling, D. R., Lawrimore, J. H., Meyers, T. P., Helfert, M. R., et al. (2013) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-12-00170.1 · Cited by 415
Vincent, L. A., Wang, X. L., Milewska, E. J., Wan, H., Yang, F., & Swail, V. (2012) · Journal of Geophysical Research: Atmospheres · doi:10.1029/2012jd017859 · Cited by 372
Rauthe, M., Steiner, H., Riediger, U., Mazurkiewicz, A., & Gratzki, A. (2013) · Meteorologische Zeitschrift · doi:10.1127/0941-2948/2013/0436 · Cited by 270
Metadata
- Authors
- Venema, V. K. C., Mestre, O., Aguilar, E., Auer, I., Guijarro, J. A., Domonkos, P., Vertacnik, G., Szentimrey, T., Stepanek, P., Zahradnicek, P., Viarre, J., Müller-Westermeier, G., Lakatos, M., Williams, C. N., Menne, M. J., Lindau, R., Rasol, D., Rustemeier, E., Kolokythas, K., Marinova, T., Andresen, L., Acquaotta, F., Fratianni, S., Cheval, S., Klancar, M., Brunetti, M., Gruber, C., Prohom Duran, M., Likso, T., Esteban, P., & Brandsma, T.
- Affiliations
- University of Bonn (DE)
- École Nationale de la Météorologie (FR)
- Universitat Rovira i Virgili (ES)
- Central Institution for Meteorology and Geodynamics (AT)
- Agencia Estatal de Meteorología (ES)
- Hungarian Meteorological Service (HU)
- National Oceanic and Atmospheric Administration (US)
- Croatian Meteorological and Hydrological Service (HR)
- University of Patras (GR)
- Bulgarian Academy of Sciences (BG)
- Norwegian Meteorological Institute (NO)
- University of Turin (IT)
- National Institute for Research and Development in Environmental Protection (RO)
- Slovenian Environment Agency (SI)
- Institute of Atmospheric Sciences and Climate (IT)
- Universitat de Barcelona (ES)
- Royal Netherlands Meteorological Institute (NL)
- ORCID
- Venue
- Climate of the Past
- Publisher
- Copernicus GmbH
- Published
- 2012-01-10
- Volume
- 8
- Issue
- 1
- Pages
- 89-115
- ISSN
- 1814-9324 (ISSN-L)
- 1814-9332
- OpenAlex
- W2734645652
- Cited by
- 368
- References
- 50
- Citations by year
- 2026: 14 · 2025: 21 · 2024: 19 · 2023: 22 · 2022: 23 · 2021: 23 · 2020: 20 · 2019: 21 · 2018: 28 · 2017: 25 · 2016: 35 · 2015: 32 · 2014: 33 · 2013: 25 · 2012: 25
- FWCI
- 30.58
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Climate of the past (publishedVersion, cc-by)
- Institutional Research Information System University of Turin (University of Turin) (submittedVersion, cc-by)
- Institutional Research Information System University of Turin (University of Turin) (submittedVersion, other-oa)
- ASEP (submittedVersion, public-domain)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- European Commission
- Deutsche Forschungsgemeinschaft
- Topics
- Physical Sciences › Environmental Science › Global and Planetary Change › Climate variability and models
- Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Precipitation Measurement and Analysis
- Physical Sciences › Environmental Science › Environmental Engineering › Urban Heat Island Mitigation
- Keywords
- monthly temperature data, linear trend estimation, station network
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.