Storm-Based Probabilistic Hail Forecasting with Machine Learning Applied to Convection-Allowing Ensembles
Abstract
Forecasting severe hail accurately requires predicting how well atmospheric conditions support the development of thunderstorms, the growth of large hail, and the minimal loss of hail mass to melting before reaching the surface. Existing hail forecasting techniques incorporate information about these processes from proximity soundings and numerical weather prediction models, but they make many simplifying assumptions, are sensitive to differences in numerical model configuration, and are often not calibrated to observations. In this paper a storm-based probabilistic machine learning hail forecasting method is developed to overcome the deficiencies of existing methods. An object identification and tracking algorithm locates potential hailstorms in convection-allowing model output and gridded radar data. Forecast storms are matched with observed storms to determine hail occurrence and the parameters of the radar-estimated hail size distribution. The database of forecast storms contains information about storm properties and the conditions of the prestorm environment. Machine learning models are used to synthesize that information to predict the probability of a storm producing hail and the radar-estimated hail size distribution parameters for each forecast storm. Forecasts from the machine learning models are produced using two convection-allowing ensemble systems and the results are compared to other hail forecasting methods. The machine learning forecasts have a higher critical success index (CSI) at most probability thresholds and greater reliability for predicting both severe and significant hail.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Gagne et al. recently used an ensemble of random forests to improve hail predictions within a major weather model.
“For example, Gagne et al. recently used an ensemble of random forests to improve hail predictions within a major weather model [262].”
From Tackling climate change with machine learning (2023) · cites this paper as [262]
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · see citing works
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957 · see citing works
Statistical Methods in the Atmospheric Sciences
Ramsey, P. H., & Wilks, D. S. (1996) · Technometrics · doi:10.2307/1271312 · Cited by 7,681 · see citing works
Development of a turbulence closure model for geophysical fluid problems
Mellor, G. L., & Yamada, T. (1982) · Reviews of Geophysics · doi:10.1029/rg020i004p00851 · Cited by 7,065 · see citing works
A New Vertical Diffusion Package with an Explicit Treatment of Entrainment Processes
Hong, S.-Y., Noh, Y., & Dudhia, J. (2006) · Monthly Weather Review · doi:10.1175/mwr3199.1 · Cited by 6,661 · see citing works
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636 · see citing works
Algorithms for the Assignment and Transportation Problems
Munkres, J. R. (1957) · Journal of the Society for Industrial and Applied Mathematics · doi:10.1137/0105003 · Cited by 4,001 · see citing works
Morrison, H., Thompson, G., & Tatarskii, V. (2009) · Monthly Weather Review · doi:10.1175/2008mwr2556.1 · Cited by 2,183 · see citing works
Statistical Methods in the Atmospheric Sciences
(2011) · International geophysics/International geophysics series · doi:10.1016/c2010-0-65519-2 · Cited by 1,078 · see citing works
Lim, K. S., & Hong, S. (2009) · Monthly Weather Review · doi:10.1175/2009mwr2968.1 · Cited by 966 · see citing works
An Improved Mellor–Yamada Level-3 Model with Condensation Physics: Its Design and Verification
中西, 幹., & Niino, H. (2004) · Boundary-Layer Meteorology · doi:10.1023/b:boun.0000020164.04146.98 · Cited by 827 · see citing works
Milbrandt, J. A., & Yau, M. K. (2005) · Journal of the Atmospheric Sciences · doi:10.1175/jas3534.1 · Cited by 774 · see citing works
The Data Assimilation Research Testbed: A Community Facility
Anderson, J. G., Hoar, T. J., Raeder, K., Liu, H., Collins, N. S., Torn, R. D., & Arellano, A. F. (2009) · Bulletin of the American Meteorological Society · doi:10.1175/2009bams2618.1 · Cited by 732 · see citing works
Morrison, H., & Milbrandt, J. A. (2014) · Journal of the Atmospheric Sciences · doi:10.1175/jas-d-14-0065.1 · Cited by 718 · see citing works
The Influence of Gravity Wave Breaking on the General Circulation of the Middle Atmosphere
Holton, J. R. (1983) · Journal of the Atmospheric Sciences · doi:10.1175/1520-0469(1983)040 · Cited by 654 · see citing works
Effects of Vertical Wind Shear on the Intensity and Structure of Numerically Simulated Hurricanes
Frank, W. M., & Ritchie, E. A. (2001) · Monthly Weather Review · doi:10.1175/1520-0493(2001)129 · Cited by 637 · see citing works
Visualizing Multiple Measures of Forecast Quality
Roebber, P. J. (2008) · Weather and Forecasting · doi:10.1175/2008waf2222159.1 · Cited by 596 · see citing works
Experiences with 0–36-h Explicit Convective Forecasts with the WRF-ARW Model
Weisman, M., Davis, C., Wang, W., Manning, K. W., & Klemp, J. (2008) · Weather and Forecasting · doi:10.1175/2007waf2007005.1 · Cited by 489 · see citing works
Kain, J., Weiss, S., Bright, D., Baldwin, M., Levit, J., Carbin, G. W., Schwartz, C., Weisman, M., Droegemeier, K., Weber, D., et al. (2008) · Weather and Forecasting · doi:10.1175/waf2007106.1 · Cited by 476 · see citing works
On Summary Measures of Skill in Rare Event Forecasting Based on Contingency Tables
Doswell, C. A., Davies-Jones, R. P., & Keller, D. L. (1990) · Weather and Forecasting · doi:10.1175/1520-0434(1990)005 · Cited by 473 · see citing works
National Mosaic and Multi-Sensor QPE (NMQ) System: Description, Results, and Future Plans
Zhang, J., Howard, K. W., Langston, C., et al. (2011) · Bulletin of the American Meteorological Society · doi:10.1175/2011bams-d-11-00047.1 · Cited by 461 · see citing works
Using Artificial Intelligence to Improve Real-Time Decision-Making for High-Impact Weather
McGovern, A., Elmore, K. L., Gagne, D. J., Haupt, S. E., Karstens, C. D., Lagerquist, R., Smith, T. M., & Williams, J. K. Y. (2017) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-16-0123.1 · Cited by 455 · see citing works
Self-citationSmith, B. T., Thompson, R. L., Grams, J. S., et al. (2012) · Weather and Forecasting · doi:10.1175/waf-d-11-00115.1 · Cited by 361 · see citing works
Sukoriansky, S., Galperin, B., & Perov, V. (2005) · Boundary-Layer Meteorology · doi:10.1007/s10546-004-6848-4 · Cited by 334 · see citing works
Thompson, R. L., Smith, B. T., Grams, J. S., et al. (2012) · Weather and Forecasting · doi:10.1175/waf-d-11-00116.1 · Cited by 310 · see citing works
An Overview of the 2010 Hazardous Weather Testbed Experimental Forecast Program Spring Experiment
Clark, A. J., Weiss, S. J., Kain, J. S., Jirak, I. L., Coniglio, M. C., Melick, C. J., Siewert, C., Sobash, R. A., Marsh, P. T., Dean, A. R., et al. (2011) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-11-00040.1 · Cited by 243 · see citing works
Self-citationMorrison, H., Milbrandt, J. A., Bryan, G. H., et al. (2014) · Journal of the Atmospheric Sciences · doi:10.1175/jas-d-14-0066.1 · Cited by 231 · see citing works
An Objective High-Resolution Hail Climatology of the Contiguous United States
Cintineo, J. L., Smith, T. M., Lakshmanan, V., et al. (2012) · Weather and Forecasting · doi:10.1175/waf-d-11-00151.1 · Cited by 208 · see citing works
The attributes diagram A geometrical framework for assessing the quality of probability forecasts
Hsu, W.-R., & Murphy, A. H. (1986) · International Journal of Forecasting · doi:10.1016/0169-2070(86)90048-8 · Cited by 191 · see citing works
The Impact of Vertical Wind Shear on Hail Growth in Simulated Supercells
Dennis, E. J., & Kumjian, M. R. (2016) · Journal of the Atmospheric Sciences · doi:10.1175/jas-d-16-0066.1 · Cited by 173 · see citing works
Kuligowski, R. J., & Barros, A. P. (1998) · Weather and Forecasting · doi:10.1175/1520-0434(1998)013 · Cited by 164 · see citing works
An Efficient, General-Purpose Technique for Identifying Storm Cells in Geospatial Images
Lakshmanan, V., Hondl, K. D., & Rabin, R. M. (2008) · Journal of Atmospheric and Oceanic Technology · doi:10.1175/2008jtecha1153.1 · Cited by 150 · see citing works
NCAR’s Experimental Real-Time Convection-Allowing Ensemble Prediction System
Schwartz, C. S., Romine, G. S., Sobash, R. A., et al. (2015) · Weather and Forecasting · doi:10.1175/waf-d-15-0103.1 · Cited by 149 · see citing works
Self-citationPower Outage Estimation for Tropical Cyclones: Improved Accuracy with Simpler Models
Nateghi, R., Guikema, S. D., & Quiring, S. M. (2013) · Risk Analysis · doi:10.1111/risa.12131 · Cited by 147 · see citing works
Sobash, R., Kain, J., Bright, D., Dean, A., Coniglio, M., & Weiss, S. (2011) · Weather and Forecasting · doi:10.1175/waf-d-10-05046.1 · Cited by 141 · see citing works
Self-citationUsing random forests to diagnose aviation turbulence
Williams, J. K. Y. (2013) · Machine Learning · doi:10.1007/s10994-013-5346-7 · Cited by 127 · see citing works
Self-citationForecasting Hail Using a One-Dimensional Hail Growth Model within WRF
Adams-Selin, R. D., & Ziegler, C. L. (2016) · Monthly Weather Review · doi:10.1175/mwr-d-16-0027.1 · Cited by 125 · see citing works
Climatology, Persistence, and Their Linear Combination as Standards of Reference in Skill Scores
Murphy, A. H. (1992) · Weather and Forecasting · doi:10.1175/1520-0434(1992)007 · Cited by 120 · see citing works
Kain, J., Dembek, S., Weiss, S., Case, J., Levit, J., & Sobash, R. (2010) · Weather and Forecasting · doi:10.1175/2010waf2222430.1 · Cited by 112 · see citing works
Self-citationEvaluating Hail Damage Using Property Insurance Claims Data
Brown, T. M., Pogorzelski, W. H., & Giammanco, I. M. (2015) · Weather Climate and Society · doi:10.1175/wcas-d-15-0011.1 · Cited by 108 · see citing works
A Bayesian Neural Network for Severe-Hail Size Prediction
Marzban, C., & Witt, A. (2001) · Weather and Forecasting · doi:10.1175/1520-0434(2001)016 · Cited by 105 · see citing works
An Objective Method of Evaluating and Devising Storm-Tracking Algorithms
Lakshmanan, V., & Smith, T. M. (2009) · Weather and Forecasting · doi:10.1175/2009waf2222330.1 · Cited by 100 · see citing works
Ahijevych, D. A., Pinto, J. O., Williams, J. K. Y., et al. (2016) · Weather and Forecasting · doi:10.1175/waf-d-15-0113.1 · Cited by 88 · see citing works
Self-citationMullen, S., & Buizza, R. (2002) · Weather and Forecasting · doi:10.1175/1520-0434(2002)017 · Cited by 87 · see citing works
Severe Weather Prediction Using Storm Surrogates from an Ensemble Forecasting System
Sobash, R., Schwartz, C., Romine, G., Fossell, K. R., & Weisman, M. (2016) · Weather and Forecasting · doi:10.1175/waf-d-15-0138.1 · Cited by 82 · see citing works
Self-citationTornado Pathlength Forecasts from 2010 to 2011 Using Ensemble Updraft Helicity
Clark, A. J., Gao, J.-D., Marsh, P. T., Smith, T. M., Kain, J., Correia, J., Xue, M., & Kong, F. (2013) · Weather and Forecasting · doi:10.1175/waf-d-12-00038.1 · Cited by 81 · see citing works
Self-citationEvaluation of Alberta Hail Growth Model Using Severe Hail Proximity Soundings from the United States
Jewell, R. E., & Brimelow, J. (2009) · Weather and Forecasting · doi:10.1175/2009waf2222230.1 · Cited by 76 · see citing works
A Method for Forecasting Hailstone Size at the Earth's Surface*
Fawbush, E. J., & Miller, R. C. (1953) · Bulletin of the American Meteorological Society · doi:10.1175/1520-0477-34.6.235 · Cited by 75 · see citing works
Increasing major hail losses in the U.S.
Changnon, S. A. (2009) · Climatic Change · doi:10.1007/s10584-009-9597-z · Cited by 72 · see citing works
Classification of Convective Areas Using Decision Trees
Gagne, D. J., McGovern, A., & Brotzge, J. A. (2009) · Journal of Atmospheric and Oceanic Technology · doi:10.1175/2008jtecha1205.1 · Cited by 58 · see citing works
Self-citationThe May 2003 Extended Tornado Outbreak
Hamill, T. M., Schneider, R. S., Brooks, H. E., et al. (2005) · Bulletin of the American Meteorological Society · doi:10.1175/bams-86-4-531 · Cited by 48 · see citing works
Hail in Northeast Italy: A Neural Network Ensemble Forecast Using Sounding-Derived Indices
Manzato, A. (2012) · Weather and Forecasting · doi:10.1175/waf-d-12-00034.1 · Cited by 38 · see citing works
Spatiotemporal analysis of tornado exposure in five US metropolitan areas
Rosencrants, T., & Ashley, W. S. (2015) · Natural Hazards · doi:10.1007/s11069-015-1704-z · Cited by 34 · see citing works
Day-Ahead Hail Prediction Integrating Machine Learning with Storm-Scale Numerical Weather Models
Gagne, D. J., McGovern, A., Jerald, J., et al. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i2.19053 · Cited by 30 · see citing works
Self-citationReaching Scientific Consensus Through a Competition
Lakshmanan, V., Elmore, K. L., & Richman, M. B. (2010) · Bulletin of the American Meteorological Society · doi:10.1175/2010bams2870.1 · Cited by 12 · see citing works
Gagne, D. J. (2016) · SHAREOK (University of Oklahoma; Oklahoma State University; Central Oklahoma University) · Cited by 9 · see citing works
Self-citationThorpe (1993) · doi:10.1175/1520-0426(1993)010
VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY
BRIER, G. W. (1950) · Monthly Weather Review · doi:10.1175/1520-0493(1950)078
Verification of forecasts expressed in terms of probability
BRIER, G. W. (1950) · Monthly Weather Review · doi:10.1175/1520-0493(1950)078%3c0001:vofeit%3e2.0.co;2 · cited 1×
“Information about reliability and resolution is contained within the Brier score (BS; Brier 1950), which can be decomposed into reliability, resolution, and uncertainty components (Murphy 1973).”
Background
Flagged first, then by citations
Cited by
Sorted by citations
Tackling climate change with machine learning
Rolnick, D., Donti, P. L., Kaack, L. H., Kochanski, K., Lacoste, A., Sankaran, K., Ross, A. S., Milojevic-Dupont, N., Jaques, N., Waldman-Brown, A., et al. (2023) · ACM Computing Surveys · doi:10.1145/3485128 · Cited by 336 · see citing works
A Machine Learning Tutorial for Operational Meteorology. Part I: Traditional Machine Learning
Chase, R. J., Harrison, D. R., Burke, A., Lackmann, G. M., & McGovern, A. (2022) · Weather and Forecasting · doi:10.1175/waf-d-22-0070.1 · Cited by 25 · see citing works
Self-citation
- Authors
- Gagne, D. J., McGovern, A., Haupt, S. E., Sobash, R. A., Williams, J. K., & Xue, M.
- Affiliations
- University of Oklahoma (US)
- NSF National Center for Atmospheric Research (US)
- Massachusetts School of Law (US)
- Venue
- Weather and Forecasting
- Publisher
- American Meteorological Society
- Published
- 2017-09-22
- Volume
- 32
- Issue
- 5
- Pages
- 1819-1840
- ISSN
- 0882-8156 (ISSN-L)
- 1520-0434
- OpenAlex
- W2743153755
- Cited by
- 169
- References
- 65
- Citations by year
- 2026: 10 · 2025: 12 · 2024: 25 · 2023: 20 · 2022: 25 · 2021: 25 · 2020: 17 · 2019: 19 · 2018: 15 · 2017: 1
- FWCI
- 8.23
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Weather and Forecasting (publishedVersion)
- NOAA Institutional Repository (submittedVersion)
- Funders
- Division of Atmospheric and Geospace Sciences (AGS-1261776)
- National Oceanic and Atmospheric Administration (DOC-NOAA NA13NWS4680001)
- Topics
- Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
- Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Precipitation Measurement and Analysis
- Physical Sciences › Physics and Astronomy › Astronomy and Astrophysics › Lightning and Electromagnetic Phenomena
- Keywords
- hail forecasting, probabilistic forecasting, machine learning, severe hail, thunderstorm development, forecast reliability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
