CoChat

A Machine Learning-Based Quality Control Algorithm for Heavy Rainfall Using Multi-Source Data

Open access
Sun, Hao, Zhou, Qing, Shi, Lijuan, Li, Cuina, Qin, Shiguang, Yao, Dan, Xu, Mingyi, Huang, Yang, Hu, Qin, Guan, Yunong · 2025 · Remote Sensing

The DOI matches its Crossref record; 39 of the 85 references it cites were found in a registry or an index, and 6 are not checked yet; 8 of the 62 sources its claims cite were checked for support.

Abstract

In this study, a machine learning-based quality control algorithm for heavy rainfall was developed by integrating automatic weather station observations with remote sensing data, minute-level data, and metadata.

Verified 2026-11-22 · 6 references not checked yet

Overview

CoChat Summary

TL;DR

This study develops a machine-learning quality-control algorithm for heavy rainfall by fusing automatic weather station observations with radar, satellite, minute-level, and metadata inputs. Gradient-boosting models outperformed conventional MDOS threshold checks, with XGBoost achieving the best test-set precision, recall, and F1-score.

Key findings

  • All four gradient-boosting models (XGBoost, LightGBM, CatBoost, GBRT) outperformed the conventional MDOS QC method on the testing set. XGBoost achieved precision 0.844, recall 0.789, and F1-score 0.816, corresponding to increases over MDOS of 0.110 in precision, 0.162 in recall, and 0.140 in F1-score.
  • Remote sensing and minute-level precipitation features dominated model predictions. Mean normalized feature importance was 0.332 for composite reflectivity (CR) and 0.220 for cloud-top temperature (CTT); among precipitation features, PRE_1m_Max, PRE_1h, and PRE_1m_Min had mean importance values of 0.089, 0.064, and 0.055.
  • SHAP analysis showed physically consistent decision logic: low CR (<20 dBZ) combined with hourly precipitation >70 mm or CTT >260 K, high CTT with low CR or high precipitation, PRE_1m_Max <0.5 mm or >3.5 mm, and negative PRE_1m_Min were flagged as anomalous.
  • Performance varied seasonally and spatially: monthly accuracy ranged 0.990–0.997 and precision 0.839–0.866; summer recall/F1 were lower (0.763–0.772 and 0.802–0.807) than winter (0.932–0.947 and 0.886–0.901). Northwestern and high-latitude regions had F1-scores above 0.93, while South China had the lowest recall (0.737) and F1-score (0.786).

Methods

  • Data covered 1 June 2022 to 31 December 2024 across China, focusing on hourly precipitation >40 mm. The total dataset contained 127,889 samples, split into 74,176 training, 21,741 validation, and 31,972 testing samples (ratio 5.8:1.7:2.5). Inputs included AWS hour-level and minute-level observations, radar CR, FY-4B CTT, and metadata.
  • Feature engineering used a sliding average temporal window for remote sensing matching and spatial nearest-neighbor averaging of five grid points. Pearson correlation with |r| > 0.5 removed 7 of 22 features, retaining 15; Z-score normalization was applied. Four gradient-boosting models were trained with Optuna hyperparameter optimization using 5-fold cross-validation.
  • Evaluation used confusion-matrix metrics (accuracy, precision, recall, F1-score), PR and ROC curves, comparison against MDOS QC, independent validation, monthly and seven-region performance assessment, and Tree SHAP interpretability.

Limitations

  • The paper states that surface observational data, radar data, and metadata are not publicly available due to Chinese national security and confidentiality policies; only satellite data are available. It also states future work is needed for spatiotemporally adaptive algorithms, expanded multi-year data, a unified multi-class QC system for all precipitation types, and systematic false-alarm analysis.
  • The paper notes that significant class imbalance inflated accuracy (>0.99) because normal samples greatly outnumbered anomalous ones, making precision, recall, and F1-score more diagnostic. It also notes that the manually corrected hourly precipitation QC benchmark, while treated as most accurate, has relatively poor timeliness.
  • Evident from the text: performance degraded in warm-season and complex-terrain regions with heterogeneous precipitation mechanisms, especially South China, indicating limited current generalization across seasons and climatic regions.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/rs17243976

Cites

39 of 85 references verified0 citation errors · 40 not checkable

Statistics

Nothing to recompute.

Figures

Image 1
CC-BY · From the PDF
Figure 1 Surface station density over the seven climatic regions of China.
Figure 1 Surface station density over the seven climatic regions of China.CC-BY · From the PDF
Figure 2 Workflow of the machine learning-based QC algorithm for heavy rainfall.
Figure 2 Workflow of the machine learning-based QC algorithm for heavy rainfall.CC-BY · From the PDF
Figure 3 Temporal matching of hourly precipitation data with remote sensing data through a sliding average temporal window method with a case from the 05:00-06:00 period.
Figure 3 Temporal matching of hourly precipitation data with remote sensing data through a sliding average temporal window method with a case from the 05:00-06:00 period.CC-BY · From the PDF
Figure 4 Absolute Pearson correlation matrix of 22 input features.
Figure 4 Absolute Pearson correlation matrix of 22 input features.CC-BY · From the PDF
Figure 5 Normalized feature importance values for the four models (XGBoost, LightGBM, CatBoost, GBRT) and their average across models.
Figure 5 Normalized feature importance values for the four models (XGBoost, LightGBM, CatBoost, GBRT) and their average across models.CC-BY · From the PDF
Figure 6 Normalized feature importance ranking of the ( a ) XGBoost, ( b ) LightGBM, ( c ) CatBoost and ( d ) GBRT models.
Figure 6 Normalized feature importance ranking of the ( a ) XGBoost, ( b ) LightGBM, ( c ) CatBoost and ( d ) GBRT models.CC-BY · From the PDF
Figure 7 ( a ) PR curves with APs and ( b ) ROC curves with AUCs of the four models.
Figure 7 ( a ) PR curves with APs and ( b ) ROC curves with AUCs of the four models.CC-BY · From the PDF
Figure 8 SHAP summary plot of the XGBoost model.
Figure 8 SHAP summary plot of the XGBoost model.CC-BY · From the PDF
Figure 9 CR SHAP dependence on ( a ) CTT, ( b ) PRE_1m_Max, ( c ) PRE_1m_Min, ( d ) PRE_1h, ( e ) WIN_D_Avg_10mi, ( f ) WIN_S_INST_Max, ( g ) TEM, and ( h ) PRS.
Figure 9 CR SHAP dependence on ( a ) CTT, ( b ) PRE_1m_Max, ( c ) PRE_1m_Min, ( d ) PRE_1h, ( e ) WIN_D_Avg_10mi, ( f ) WIN_S_INST_Max, ( g ) TEM, and ( h ) PRS.CC-BY · From the PDF
Figure 10 CTT SHAP dependence on ( a ) CR, ( b ) PRE_1m_Max, ( c ) PRE_1m_Min, ( d ) PRE_1h, ( e ) WIN_D_Avg_10mi, ( f ) WIN_S_INST_Max, ( g ) TEM, and ( h ) PRS.
Figure 10 CTT SHAP dependence on ( a ) CR, ( b ) PRE_1m_Max, ( c ) PRE_1m_Min, ( d ) PRE_1h, ( e ) WIN_D_Avg_10mi, ( f ) WIN_S_INST_Max, ( g ) TEM, and ( h ) PRS.CC-BY · From the PDF
Figure 11 PRE_1m_Max SHAP dependence on ( a ) CR, ( b ) CTT, ( c ) PRE_1m_Min, ( d ) PRE_1h, ( e ) WIN_D_Avg_10mi, ( f ) WIN_S_INST_Max, ( g ) TEM, and ( h ) PRS.
Figure 11 PRE_1m_Max SHAP dependence on ( a ) CR, ( b ) CTT, ( c ) PRE_1m_Min, ( d ) PRE_1h, ( e ) WIN_D_Avg_10mi, ( f ) WIN_S_INST_Max, ( g ) TEM, and ( h ) PRS.CC-BY · From the PDF
Image 13
CC-BY · From the PDF

Verification

Claims the paper makes

5 Valid · 1 Not valid · 23 Not checked · 86 Own results and interpretations

76 more claims

Create a free account to see all 115 claims and to verify the 22 references still open.

See all claims — free account
  • Probably not a substantial claim. 5
  • It may state two separate claims as one. 31

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

4 of 4

Citations

Cites

Show all 85 references

Research works printed but not linked: 2

  1. 2. Remote Sens. 2025, 17, 3976
  2. 79. J. Meteorol. Soc. Jpn. Ser. II 2020, 98, 19-42. [CrossRef]

Metadata

Authors
Sun, H., Zhou, Q., Shi, L., Li, C., Qin, S., Yao, D., Xu, M., Huang, Y., Hu, Q., & Guan, Y.
Affiliations
  • China Meteorological Administration (CN)
  • Anqing Meteorological Bureau, Anqing 246001, China
  • Jiangxi Meteorological Observation Centre, Nanchang 330096, China
  • Shanghai Meteorological Bureau (CN)
Corresponding author
Sun, Hao
Venue
Remote Sensing
Publisher
MDPI AG
Published
2025-12-09
Volume
17
Issue
24
Pages
3976
ISSN
2072-4292 (ISSN-L)
OpenAlex
W4417159468
Cited by
4
References
78
Citations by year
2026: 4
FWCI
1.73
Type
journal article
Language
en
Open access
Locations
  • Remote Sensing (publishedVersion, cc-by)
  • Preprints.org (acceptedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
National Natural Science Foundation of China (U2342216)
Topics
  • Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Precipitation Measurement and Analysis
  • Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
  • Physical Sciences › Environmental Science › Management, Monitoring, Policy and Law › Environmental Monitoring and Data Management
Keywords
heavy rainfall, quality control, automatic weather station, gradient boosting, XGBoost, LightGBM, CatBoost, cloud top temperature, SHAP, false alarm reduction, multi-source data fusion

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.