Cited by 13Precipitation Measurement and AnalysisHydrological Forecasting Using AIMeteorological Phenomena and Simulations

Deep-learning-based precipitation observation quality control

Open access
Sha, Yingkai, Gagne, David John, West, Gregory, Stull, Roland·2021·Journal of Atmospheric and Oceanic Technology

Abstract

We present a novel approach for the automated quality control (QC) of precipitation for a sparse station observation network within the complex terrain of British Columbia, Canada. Our QC approach uses Convolutional Neural Networks (CNNs) to classify bad observation values, incorporating a multi-classifier ensemble to achieve better QC performance. We train CNNs using human QC’d labels from 2016 to 2017 with gridded precipitation and elevation analyses as inputs. Based on the classification evaluation metrics, our QC approach shows reliable and robust performance across different geographical environments (e.g., coastal and inland mountains), with 0.927 Area Under Curve (AUC) and type I/type II error lower than 15%. Based on the saliency-map-based interpretation studies, we explain the success of CNN-based QC by showing that it can capture the precipitation patterns around, and upstream of the station locations. This automated QC approach is an option for eliminating bad observations for various applications, including the pre-processing of training datasets for machine learning. It can be used in conjunction with human QC to improve upon what could be accomplished with either method alone.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1175/jtech-d-20-0081.1

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Incorporating topographic elevation data through deep neural networks and multi-scale ensemble learning techniques enhanced the capability to detect precipitation quality anomalies in data-sparse regions such as coastal and mountainous areas.

    “By incorporating topographic elevation data through deep neural networks and multi-scale ensemble learning techniques, the capability to detect precipitation quality anomalies in data-sparse regions such as coastal and mountainous areas has been enhanced [42].”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 58

  1. Adam, J. C., and D. P. Lettenmaier, 2003: Adjustment of global gridded precipitation for systematic bias: Global gridded precipitation. J. Geophys. Res., 108, 4257, https://doi.org/ 10.1029/2002JD002499. · DOI
  2. Adler, R. F., and Coauthors, 2003: The version-2 Global Precipitation Climatology Project (GPCP) monthly precipitation analysis (1979-present). J. Hydrometeor., 4, 1147-1167, https://doi.org/ 10.1175/1525-7541(2003)004,1147:TVGPCP.2.0.CO;2. · DOI
  3. Banta, R. M., and Coauthors, 2013: Observational techniques: Sampling the mountain atmosphere. Mountain Weather Research and Forecasting: Recent Progress and Current Challenges, F. K. Chow, S. F. De Wekker, and B. J. Snyder, Eds., Springer Atmospheric Sciences, 409-530, https:// doi.org/10.1007/978-94-007-4098-3_8. · DOI
  4. Breiman, L., 1996: Bagging predictors. Mach. Learn., 24, 123-140, https://doi.org/10.1007/BF00058655. · DOI · Paper page
  5. --, J. H. Friedman, R. A. Olshen, and C. J. Stone, 2017: Classification and Regression Trees. 1st ed. Routledge, 368 pp., https://doi.org/10.1201/9781315139470. · DOI · Paper page
  6. Carrera, M. L., S. Bélair, V. Fortin, B. Bilodeau, D. Charpentier, and I. Doré, 2010: Evaluation of snowpack simulations over the Canadian Rockies with an experimental hydrometeoro- logical modeling system. J. Hydrometeor., 11, 1123-1140, https://doi.org/10.1175/2010JHM1274. · DOI
  7. Chilton, R. R. H., 1981: A summary of climatic regimes of British Columbia. Ministry of Environment Assessment and Planning Division Rep., 44 pp. · DOI
  8. CISL, 2020: Cheyenne: HPE/SGI ICE XA system (NCAR Community Computing). National Center for Atmospheric Research, https:// doi.org/10.5065/d6rx99hx. · DOI
  9. Eischeid, J. K., C. B. Baker, T. R. Karl, and H. F. Diaz, 1995: The quality control of long-term climatological data using objective data analysis. J. Appl. Meteor., 34, 2787-2795, https://doi.org/ 10.1175/1520-0450(1995)034,2787:TQCOLT.2.0.CO;2. · DOI
  10. J. Appl. Meteor., 39, 1580-1591, https://doi.org/10.1175/1520- 0450(2000)039,1580:CASCND.2.0.CO;2. · DOI
  11. Fortin, V., G. Roy, N. Donaldson, and A. Mahidjiba, 2015: Assimilation of radar quantitative precipitation estimations in the Canadian Precipitation Analysis (CaPA). J. Hydrol., 531, 296-307, https://doi.org/10.1016/j.jhydrol.2015.08.003. · DOI
  12. --, --, T. Stadnyk, K. Koenig, N. Gasset, and A. Mahidjiba, 2018: Ten years of science based on the Canadian Precipitation Analysis: A CaPA system overview and literature review. Atmos.-Ocean, 56, 178-196, https://doi.org/10.1080/07055900.2018.1474728. · DOI
  13. Goodfellow, I., Y. Bengio, and A. Courville, 2016: Deep Learning: Adaptive Computation and Machine Learning. MIT Press, 775 pp.
  14. Groisman, P. Ya., and D. R. Legates, 1994: The accuracy of United States precipitation data. Bull. Amer. Meteor. Soc., 75, 215-228, https://doi.org/10.1175/1520-0477(1994) 075,0215:TAOUSP.2.0.CO;2. · DOI
  15. Gu, J., and Coauthors, 2018: Recent advances in convolutional neural networks. Pattern Recognit., 77, 354-377, https://doi.org/ 10.1016/j.patcog.2017.10.013. · DOI · Paper page
  16. --, --, --, and --, 2015b: Delving deep into rectifiers: surpassing human-level performance on ImageNet clas- sification. 2015 IEEE Int. Conf. on Computer Vision, IEEE, Santiago, Chile, 1026-1034, https://doi.org/10.1109/ ICCV.2015.123. · DOI · Paper page
  17. Hubbard, K. G., S. Goddard, W. D. Sorensen, N. Wells, and T. T. Osugi, 2005: Performance of quality assurance procedures for an applied climate information system. J. Atmos. Oceanic Technol., 22, 105-112, https://doi.org/10.1175/JTECH-1657. · DOI
  18. Ioffe, S., and C. Szegedy, 2015: Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv, http://arxiv.org/abs/1502.03167. · DOI
  19. Jiang, S., M. Lian, C. Lu, Q. Gu, S. Ruan, and X. Xie, 2018: Ensemble prediction algorithm of anomaly monitoring based on big data analysis platform of open-pit mine slope. Complexity, 2018, 1048756, https://doi.org/10.1155/2018/ 1048756. · DOI
  20. Jørgensen, H. K., S. Rosenørn, H. Madsen, and P. S. Mikkelsen, 1998: Quality control of rain data used for urban runoff sys- tems. Water Sci. Technol., 37, 113-120, https://doi.org/10.2166/ wst.1998.0448. · DOI
  21. Kienzle, S. W., 2008: A new temperature based method to separate rain and snow. Hydrol. Processes, 22, 5067-5085, https:// doi.org/10.1002/hyp.7131. · DOI
  22. Lakshmanan, V., A. Fritz, T. Smith, K. Hondl, and G. Stumpf, 2007: An automated technique to quality control radar re- flectivity data. J. Appl. Meteor. Climatol., 46, 288-305, https:// doi.org/10.1175/JAM2460. · DOI
  23. --, C. Karstens, J. Krause, and L. Tang, 2014: Quality control of weather radar data using polarimetric variables. J. Atmos. Oceanic Technol., 31, 1234-1249, https://doi.org/10.1175/JTECH- D-13-00073. · DOI
  24. Lespinas, F., V. Fortin, G. Roy, P. Rasmussen, and T. Stadnyk, 2015: Performance evaluation of the Canadian Precipitation Analysis (CaPA). J. Hydrometeor., 16, 2045-2064, https:// doi.org/10.1175/JHM-D-14-0191. · DOI
  25. Liu, Y., and Coauthors, 2016: Application of deep convolutional neural networks for detecting extreme weather in climate datasets. arXiv, http://arxiv.org/abs/1605.01156. · DOI
  26. --, G. Ren, X. Sun, and X. Li, 2018: A new method to separate precipitation phases. Hydrol. Earth Syst. Sci. Discuss., https:// doi.org/10.5194/hess-2018-307. · DOI
  27. Mahfouf, J.-F., B. Brasnett, and S. Gagnon, 2007: A Canadian Precipitation Analysis (CaPA) project: Description and pre- liminary results. Atmos.-Ocean, 45, 1-17, https://doi.org/ 10.3137/ao.v450101. · DOI
  28. Martinaitis, S. M., S. B. Cocks, Y. Qi, B. T. Kaney, J. Zhang, and K. Howard, 2015: Understanding winter precipitation impacts on automated gauge observations within a real-time system. J. Hydrometeor., 16, 2345-2363, https://doi.org/10.1175/JHM- D-15-0020. · DOI · Paper page
  29. Maul-Kötter, B., and T. Einfalt, 1998: Correction and preparation of continuously measured raingauge data: A standard method in North Rhine-Westphalia. Water Sci. Technol., 37, 155-162, https://doi.org/10.2166/wst.1998.0458. · DOI
  30. Meek, D., and J. Hatfield, 1994: Data quality checking for single station meteorological databases. Agric. For. Meteor., 69, 85-109, https://doi.org/10.1016/0168-1923(94)90083-3. · DOI
  31. Motoyama, H., 1990: Simulation of seasonal snowcover based on air temperature and precipitation. J. Appl. Meteor., 29, 1104-1110, https://doi.org/10.1175/1520-0450(1990)029,1104: SOSSBO.2.0.CO;2. · DOI
  32. Mourad, M., and J.-L. Bertrand-Krajewski, 2002: A method for automatic validation of long time series of data in urban hy- drology. Water Sci. Technol., 45, 263-270, https://doi.org/ 10.2166/wst.2002.0601. · DOI · Paper page
  33. Nearing, M. A., and Coauthors, 2005: Modeling response of soil erosion and runoff to changes in precipitation and cover. Catena, 61, 131-154, https://doi.org/10.1016/j.catena.2005.03.007. · DOI
  34. Neiman, P. J., F. M. Ralph, G. A. Wick, J. D. Lundquist, and M. D. Dettinger, 2008: Meteorological characteristics and overland precipitation impacts of atmospheric rivers affecting the west coast of North America based on eight years of SSM/I satellite observations. J. Hydrometeor., 9, 22-47, https://doi.org/10.1175/ 2007JHM855. · DOI
  35. Niculescu-Mizil, A., and R. Caruana, 2005: Predicting good prob- abilities with supervised learning. Proc. 22nd Int. Conf. on Machine Learning, Bonn, Germany, ACM, 625-632, https:// doi.org/10.1145/1102351.1102430. · DOI · Paper page
  36. Null, S. E., J. H. Viers, and J. F. Mount, 2010: Hydrologic response and watershed sensitivity to climate warming in California's Sierra Nevada. PLOS ONE, 5, e9932, https://doi.org/10.1371/ journal.pone.0009932. · DOI
  37. Odon, P., G. West, and R. Stull, 2018: Evaluation of reanalyses over British Columbia. Part I: Daily and extreme 2-m tem- perature. J. Appl. Meteor. Climatol., 57, 2091-2112, https:// doi.org/10.1175/JAMC-D-18-0058. · DOI
  38. Piatyszek, E., P. Voignier, and D. Graillot, 2000: Fault detection on a sewer network by a combination of a Kalman filter and a binary sequential probability ratio test. J. Hydrol., 230, 258- 268, https://doi.org/10.1016/S0022-1694(00)00213-4. · DOI
  39. Platt, J. C., 1999: Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in Large Margin Classifiers, MIT Press, 61-74.
  40. Qi, Y., S. Martinaitis, J. Zhang, and S. Cocks, 2016: A real-time automated quality control of hourly rain gauge data based on multiple sensors in MRMS system. J. Hydrometeor., 17, 1675-1691, https://doi.org/10.1175/JHM-D-15-0188. · DOI · Paper page
  41. Raileanu, L. E., and K. Stoffel, 2004: Theoretical comparison between the Gini index and information gain criteria. Ann. Math. Artif. Intell., 41, 77-93, https://doi.org/10.1023/B: AMAI.0000018580.96245. · DOI
  42. Rasmussen, R., and Coauthors, 2012: How well are we measuring snow: The NOAA/FAA/NCAR winter precipitation test bed. Bull. Amer. Meteor. Soc., 93, 811-829, https://doi.org/10.1175/ BAMS-D-11-00052. · DOI
  43. Read, W. A., 2015: The climatology and meteorology of wind- storms that affect southwest British Columbia, Canada, and associated tree-related damage to the power distribution grid. Ph.D. thesis, University of British Columbia, 383 pp., https:// doi.org/10.14288/1.0166485. · DOI
  44. Schneider, U., A. Becker, P. Finger, A. Meyer-Christoffer, M. Ziese, and B. Rudolf, 2014: GPCC's new land surface precipitation climatology based on quality-controlled in situ data and its role in quantifying the global water cycle. Theor. Appl. Climatol., 115, 15-40, https://doi.org/10.1007/s00704-013-0860-x. · DOI · Paper page
  45. Sciuto, G., B. Bonaccorso, A. Cancelliere, and G. Rossi, 2009: Quality control of daily rainfall data with neural net- works. J. Hydrol., 364, 13-22, https://doi.org/10.1016/ j.jhydrol.2008.10.008. · DOI · Paper page
  46. Simonyan, K., and A. Zisserman, 2015: Very deep convolutional networks for large-scale image recognition. arXiv, http:// arxiv.org/abs/1409.1556. · arXiv
  47. Springenberg, J. T., A. Dosovitskiy, T. Brox, and M. Riedmiller, 2015: Striving for simplicity: The all convolutional net. arXiv, http://arxiv.org/abs/1412.6806. · arXiv
  48. St epánek, P., P. Zahradnícȇk, and P. Skalák, 2009: Data quality control and homogenization of air temperature and precipi- tation series in the area of the Czech Republic in the period 1961-2007. Adv. Sci. Res., 3, 23-26, https://doi.org/10.5194/asr- 3-23-2009. · DOI
  49. Tompson, J., R. Goroshin, A. Jain, Y. LeCun, and C. Bregler, 2015: Efficient object localization using convolutional networks. 2015 IEEE Conf. on Computer Vision and Pattern Recognition, Boston, MA, IEEE, 648-656, https:// doi.org/10.1109/CVPR.2015.7298664. · DOI · Paper page
  50. Vandal, T., E. Kodra, S. Ganguly, A. Michaelis, R. Nemani, and A. R. Ganguly, 2018: Generating high resolution climate change projec- tions through single image super-resolution: an abridged version. Proc. 27th Int. Joint Conf. on Artificial Intelligence, Stockholm, Sweden, International Joint Conferences on Artificial Intelligence Organization, 5389-5393, https://doi.org/10.24963/ijcai.2018/759. · DOI
  51. Wilks, D. S., 2011: Statistical Methods in the Atmospheric Sciences. 3rd ed. Elsevier, 676 pp.
  52. Wong, J. S., S. Razavi, B. R. Bonsal, H. S. Wheater, and Z. E. Asong, 2017: Inter-comparison of daily precipitation products for large-scale hydro-climatic applications over Canada. Hydrol. Earth Syst. Sci., 21, 2163-2185, https:// doi.org/10.5194/hess-21-2163-2017. · DOI
  53. Xie, P., and P. A. Arkin, 1996: Analyses of global monthly precipita- tion using gauge observations, satellite estimates, and numerical model predictions. J. Climate, 9, 840-858, https://doi.org/10.1175/ 1520-0442(1996)009,0840:AOGMPU.2.0.CO;2. · DOI
  54. Xu, C.-D., J.-F. Wang, M.-G. Hu, and Q.-X. Li, 2014: Estimation of uncertainty in temperature observations made at meteoro- logical stations using a probabilistic spatiotemporal approach. J. Appl. Meteor. Climatol., 53, 1538-1546, https://doi.org/ 10.1175/JAMC-D-13-0179. · DOI
  55. Yang, D., D. Kane, Z. Zhang, D. Legates, and B. Goodison, 2005: Bias corrections of long-term (1973-2004) daily precipitation data over the northern regions. Geophys. Res. Lett., 32, L19501, https://doi.org/10.1029/2005GL024057. · DOI · Paper page
  56. You, J., K. G. Hubbard, S. Nadarajah, and K. E. Kunkel, 2007: Performance of quality assurance procedures on daily pre- cipitation. J. Atmos. Oceanic Technol., 24, 821-834, https:// doi.org/10.1175/JTECH2002. · DOI
  57. Zeiler, M. D., and R. Fergus, 2014: Visualizing and understanding convolutional networks. 13th European Conf. on Computer Vision, Zurich, Switzerland, ECCV, https://doi.org/10.1007/ 978-3-319-10590-1_53. · DOI · Paper page
  58. Zhao, Q., Y. Zhu, D. Wan, Y. Yu, and X. Cheng, 2018: Research on the data-driven quality control method of hydrological time series data. Water, 10, 1712, https://doi.org/10.3390/w10121712. · DOI · Paper page

Cited by

Sorted by citations

Authors
Sha, Y., Gagne, D. J., West, G., & Stull, R.
Affiliations
  • University of British Columbia (CA)
  • NSF National Center for Atmospheric Research (US)
  • BC Hydro (Canada) (CA)
Corresponding author
Sha, Yingkai
Venue
Journal of Atmospheric and Oceanic Technology
Published
2021-04-05
ISSN
  • 0739-0572 (ISSN-L)
  • 1520-0426
OpenAlex
W3139945928
Cited by
13
References
2
Citations by year
2025: 2 · 2024: 5 · 2023: 2 · 2022: 4
FWCI
0.96
Type
journal article
Language
en
Open access
Locations
Journal of Atmospheric and Oceanic Technology (publishedVersion, other-oa)
Funders
  • National Science Foundation
  • National Center for Atmospheric Research
  • University Corporation for Atmospheric Research
  • BC Hydro
  • Mitacs
  • University of British Columbia
  • Environment and Climate Change Canada
  • Natural Sciences and Engineering Research Council of Canada
Topics
  • Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Precipitation Measurement and Analysis
  • Physical Sciences › Environmental Science › Environmental Engineering › Hydrological Forecasting Using AI
  • Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
Keywords
automated quality control, complex terrain, convolutional neural networks, multi-classifier ensemble, saliency maps, gridded precipitation, British Columbia

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review