The DOI matches its Crossref record; 79 of the 79 references it cites were found in a registry or an index.
Making the Black Box More Transparent: Understanding the Physical Implications of Machine Learning
Open accessAbstract
This paper synthesizes multiple methods for machine learning (ML) model interpretation and visualization (MIV) focusing on meteorological applications. ML has recently exploded in popularity in many fields, including meteorology. Although ML has been successful in meteorology, it has not been as widely accepted, primarily due to the perception that ML models are “black boxes,” meaning the ML methods are thought to take inputs and provide outputs but not to yield physically interpretable information to the user. This paper introduces and demonstrates multiple MIV techniques for both traditional ML and deep learning, to enable meteorologists to understand what ML models have learned. We discuss permutation-based predictor importance, forward and backward selection, saliency maps, class-activation maps, backward optimization, and novelty detection. We apply these methods at multiple spatiotemporal scales to tornado, hail, winter precipitation type, and convective-storm mode. By analyzing such a wide variety of applications, we intend for this work to demystify the black box of ML, offer insight in applying MIV techniques, and serve as a MIV toolbox for meteorologists and other physical scientists.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,522
Quinlan, J. R. (1986) · Machine Learning · doi:10.1007/bf00116251 · Cited by 21,439
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
(2010) · International Conference on Machine Learning · Cited by 13,995
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Wrappers for feature subset selection
Kohavi, R., & John, G. H. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(97)00043-x · Cited by 8,947
Statistical Methods in the Atmospheric Sciences
Ramsey, P. H. (1996) · Technometrics · doi:10.1080/00401706.1996.10484553 · Cited by 7,601
Friedman, J. H. (2002) · Computational Statistics & Data Analysis · doi:10.1016/s0167-9473(01)00065-2 · Cited by 7,067
Basic principles of ROC analysis
Metz, C. E. (1978) · Seminars in Nuclear Medicine · doi:10.1016/s0001-2998(78)80014-2 · Cited by 6,026
Lipton, Z. C. (2018) · Queue · doi:10.1145/3236386.3241340 · Cited by 5,427
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370
Support Vector Regression Machines
Drucker, H., Burges, C. J. C., Kaufman, L., Smola, A. J., & Vapnik, V. N. (1996) · Cited by 4,658
Thompson, G., Field, P. R., Rasmussen, R. M., & Hall, W. D. (2008) · Monthly Weather Review · doi:10.1175/2008mwr2387.1 · Cited by 3,156
Methods for interpreting and understanding deep neural networks
Montavon, G., Samek, W., & Müller, K.-R. (2018) · Digital Signal Processing · doi:10.1016/j.dsp.2017.10.011 · Cited by 2,665
Goldstein, A., Kapelner, A., Bleich, J., & Pitkin, E. (2015) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2014.907095 · Cited by 1,861
AN OPERATIONAL OBJECTIVE ANALYSIS SYSTEM
CRESSMAN, G. P. (1959) · Monthly Weather Review · doi:10.1175/1520-0493(1959)087<0367:aooas>2.0.co;2 · Cited by 1,798
Rotation of principal components
Richman, M. B. (1986) · Journal of Climatology · doi:10.1002/joc.3370060305 · Cited by 1,760
Statistical Pattern Recognition
(2001) · Computational Statistics Handbook with MATLAB · doi:10.1201/9781420035636-10 · Cited by 1,740
Statistical Methods in the Atmospheric Sciences
Briggs, W. (. (2007) · Journal of the American Statistical Association · doi:10.1198/jasa.2007.s163 · Cited by 1,531
Statistical Pattern Recognition
Webb, A. R. (2002) · Wiley eBooks · doi:10.1002/0470854774 · Cited by 1,459
Comparing support vector machines with Gaussian kernels to radial basis function classifiers
Scholkopf, B., Sung, K.-K., Burges, C. J. C., Girosi, F., Niyogi, P., Poggio, T., & Vapnik, V. (1997) · IEEE Transactions on Signal Processing · doi:10.1109/78.650102 · Cited by 1,400
Understanding variable importances in forests of randomized trees
Louppe, G., Wehenkel, L. A., Sutera, A., & Geurts, P. (2013) · ORBi (University of Liège) · Cited by 1,364
Sanity Checks for Saliency Maps
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., & Kim, B. (2018) · arXiv · doi:10.48550/arxiv.1810.03292 · Cited by 1,294
A North American Hourly Assimilation and Model Forecast Cycle: The Rapid Refresh
Benjamin, S. G., Weygandt, S. S., Brown, J. M., Hu, M., Alexander, C. R., Smirnova, T. G., Olson, J. B., James, E. P., Dowell, D. C., Grell, G. A., et al. (2016) · Monthly Weather Review · doi:10.1175/mwr-d-15-0242.1 · Cited by 1,164
Self-citationThompson, G., Rasmussen, R. M., & Manning, K. (2004) · Monthly Weather Review · doi:10.1175/1520-0493(2004)132<0519:efowpu>2.0.co;2 · Cited by 1,159
Chollet, F. (2017) · Cited by 1,158
Pattern recognition using generalized portrait method
Vapnik, V. N. (1963) · Automation and Remote Control · Cited by 1,151
Samek, W., Wiegand, T., & Müller, K.-R. (2017) · arXiv · doi:10.48550/arxiv.1708.08296 · Cited by 1,080
Olah, C., Mordvintsev, A., & Schubert, L. (2017) · Distill · doi:10.23915/distill.00007 · Cited by 880
Rotation-invariant convolutional neural networks for galaxy morphology prediction
Dieleman, S., Willett, K. W., & Dambre, J. (2015) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stv632 · Cited by 781
A NOTE ON GENETIC ALGORITHMS FOR LARGE-SCALE FEATURE SELECTION
SIEDLECKI, W., & SKLANSKY, J. (1993) · Handbook of Pattern Recognition and Computer Vision · doi:10.1142/9789814343138_0005 · Cited by 765
Fukushima, K., & Miyake, S. (1982) · Pattern Recognition · doi:10.1016/0031-3203(82)90024-3 · Cited by 742
An Hourly Assimilation–Forecast Cycle: The RUC
Benjamin, S. G., Dévényi, D., Weygandt, S. S., Brundage, K. J., Brown, J. M., Grell, G. A., Kim, D., Schwartz, B. E., Smirnova, T. G., Smith, T. L., et al. (2004) · Monthly Weather Review · doi:10.1175/1520-0493(2004)132<0495:ahactr>2.0.co;2 · Cited by 634
Self-citationInverting Visual Representations with Convolutional Networks
Dosovitskiy, A., & Brox, T. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.522 · Cited by 603
Tree Induction for Probability-Based Ranking
Provost, F., & Domingos, P. (2003) · Machine Learning · doi:10.1023/a:1024099825458 · Cited by 541
Using Artificial Intelligence to Improve Real-Time Decision-Making for High-Impact Weather
McGovern, A., Elmore, K. L., Gagne, D. J., Haupt, S. E., Karstens, C. D., Lagerquist, R., Smith, T., & Williams, J. K. (2017) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-16-0123.1 · Cited by 443
Self-citationAbility of a Poor Man's Ensemble to Predict the Probability and Distribution of Precipitation
Ebert, E. E. (2001) · Monthly Weather Review · doi:10.1175/1520-0493(2001)129<2461:aoapms>2.0.co;2 · Cited by 386
Atmospheric Temperature Prediction using Support Vector Machines
Radhika, Y., & Shashi, M. (2009) · International Journal of Computer Theory and Engineering · doi:10.7763/ijcte.2009.v1.9 · Cited by 383
Polarimetric Signatures in Supercell Thunderstorms
Kumjian, M. R., & Ryzhkov, A. V. (2008) · Journal of Applied Meteorology and Climatology · doi:10.1175/2007jamc1874.1 · Cited by 371
Smith, B. T., Thompson, R. L., Grams, J. S., Broyles, C., & Brooks, H. E. (2012) · Weather and Forecasting · doi:10.1175/waf-d-11-00115.1 · Cited by 358
Thompson, R. L., Smith, B. T., Grams, J. S., Dean, A. R., & Broyles, C. (2012) · Weather and Forecasting · doi:10.1175/waf-d-11-00116.1 · Cited by 304
Application of Deep Convolutional Neural Networks for Detecting Extreme Weather in Climate Datasets
Liu, Y., Racah, E., Prabhat, Correa, J., Khosrowshahi, A., Lavers, D. A., Kunkel, K. E., Wehner, M. F., & Collins, W. D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.01156 · Cited by 292
Numerical Weather Map Analysis
Bergthórsson, P., & Döös, B. R. (1955) · Tellus A: Dynamic Meteorology and Oceanography · doi:10.3402/tellusa.v7i3.8902 · Cited by 226
Interpretable Deep Learning for Spatial Analysis of Severe Hailstorms
Gagne II, D. J., Haupt, S. E., Nychka, D. W., & Thompson, G. (2019) · Monthly Weather Review · doi:10.1175/mwr-d-18-0316.1 · Cited by 215
Multi-band Polarization Imaging and Applications
Zhao, Y., Yi, C., Kong, S. G., Pan, Q., & Cheng, Y. (2016) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-662-49373-1 · Cited by 180
A Neural Network for Tornado Prediction Based on Doppler Radar-Derived Attributes
Marzban, C., & Stumpf, G. J. (1996) · Journal of Applied Meteorology · doi:10.1175/1520-0450(1996)035<0617:annftp>2.0.co;2 · Cited by 175
Gagne, D. J., McGovern, A., Haupt, S. E., Sobash, R. A., Williams, J. K., & Xue, M. (2017) · Weather and Forecasting · doi:10.1175/waf-d-17-0010.1 · Cited by 169
Self-citationMPING: Crowd-Sourcing Weather Reports for Research
Elmore, K. L., Flamig, Z. L., Lakshmanan, V., Kaney, B. T., Farmer, V., Reeves, H. D., & Rothfusz, L. P. (2014) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-13-00014.1 · Cited by 159
Self-citationOBJECTIVE WEATHER-MAP ANALYSIS
Panofsky, R. A. (1949) · Journal of Meteorology · Cited by 145
Deep Learning for Spatially Explicit Prediction of Synoptic-Scale Fronts
Lagerquist, R., McGovern, A., & Gagne, D. J. (2019) · Weather and Forecasting · doi:10.1175/waf-d-18-0183.1 · Cited by 139
Self-citationMachine Learning for Real-Time Prediction of Damaging Straight-Line Convective Wind
Lagerquist, R., McGovern, A., & Smith, T. M. (2017) · Weather and Forecasting · doi:10.1175/waf-d-17-0038.1 · Cited by 136
Self-citationUsing random forests to diagnose aviation turbulence
Williams, J. K. (2014) · Machine Learning · doi:10.1007/s10994-013-5346-7 · Cited by 124
A Bayesian Neural Network for Severe-Hail Size Prediction
Marzban, C., & Witt, A. (2001) · Weather and Forecasting · doi:10.1175/1520-0434(2001)016<0600:abnnfs>2.0.co;2 · Cited by 114
Genetic Algorithms in Feature Selection
Leardi, R. (1996) · Elsevier eBooks · doi:10.1016/b978-012213810-2/50004-9 · Cited by 101
An Empirical Model for Assessing the Severe Weather Potential of Developing Convection
Cintineo, J. L., Pavolonis, M. J., Sieglaff, J. M., & Lindsey, D. T. (2014) · Weather and Forecasting · doi:10.1175/waf-d-13-00113.1 · Cited by 82
Schwartz, C. S., Romine, G. S., Weisman, M. L., Sobash, R. A., Fossell, K. R., Manning, K. W., & Trier, S. B. (2015) · Weather and Forecasting · doi:10.1175/waf-d-15-0013.1 · Cited by 71
Classifying Convective Storms Using Machine Learning
Jergensen, G. E., McGovern, A., Lagerquist, R., & Smith, T. M. (2019) · Weather and Forecasting · doi:10.1175/waf-d-19-0170.1 · Cited by 71
Self-citationMcGovern, A., Gagne, D. J., Williams, J. K. Y., Brown, R. A., & Basara, J. B. (2013) · Machine Learning · doi:10.1007/s10994-013-5343-x · Cited by 66
Self-citationWhich Polarimetric Variables Are Important for Weather/No-Weather Discrimination?
Lakshmanan, V., Karstens, C. D., Krause, J. M., Elmore, K. L., Ryzhkov, A. V., & Berkseth, S. (2015) · Journal of Atmospheric and Oceanic Technology · doi:10.1175/jtech-d-13-00205.1 · Cited by 66
Self-citationClassification of Convective Areas Using Decision Trees
Gagne, D. J., McGovern, A., & Brotzge, J. A. (2009) · Journal of Atmospheric and Oceanic Technology · doi:10.1175/2008jtecha1205.1 · Cited by 63
Self-citationBecoming the expert - interactive multi-class machine teaching
Johns, E., Aodha, O. M., & Brostow, G. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298877 · Cited by 57
A Neural Network for Damaging Wind Prediction
Marzban, C., & Stumpf, G. J. (1998) · Weather and Forecasting · Cited by 55
The NOAA/CIMSS ProbSevere Model: Incorporation of Total Lightning and Validation
Cintineo, J. L., Pavolonis, M. J., Sieglaff, J. M., Lindsey, D. T., Cronce, L. M., Gerth, J. J., Rodenkirch, B. D., Brunner, J. C., & Gravelle, C. M. (2018) · Weather and Forecasting · doi:10.1175/waf-d-17-0099.1 · Cited by 51
Synoptic-Scale Features Common to Heavy Snowstorms in the Southeast United States
Mote, T. L., Gamble, D. W., Underwood, S. J., & Bentley, M. L. (1997) · Weather and Forecasting · doi:10.1175/1520-0434(1997)012<0154:uorttp>2.0.co;2 · Cited by 47
McGovern, A., Gagne, D. J., Basara, J. B., Hamill, T. M., & Margolin, D. (2015) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-14-00006.1 · Cited by 34
Self-citationAPPLICATION OF STATISTICAL METHODS IN WEATHER PREDICTION
Malone, T. F. (1955) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.41.11.806 · Cited by 26
Combining observations and model data for short-term storm forecasting
Williams, J. K. Y., Ahijevych, D. A., Dettling, S., & Steiner, M. (2008) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.795737 · Cited by 21
Remote detection and diagnosis of thunderstorm turbulence
Williams, J. K. Y., Sharman, R. D., Craig, J., & Blackburn, G. (2008) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.795570 · Cited by 20
Interpretable Discovery in Large Image Data SetsPreprint
Wagstaff, K. L., & Lee, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.08340 · Cited by 9
Severe Hail Prediction within a Spatiotemporal Relational Data Mining Framework
Gagne, D. J., McGovern, A., Brotzge, J. A., & Xue, M. (2013) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2013.121 · Cited by 5
Self-citation
Cited by
Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3060483 · Cited by 1,477
A review of machine learning applications in wildfire science and management
Jain, P., Coogan, S. C. P., Subramanian, S. G., Crowley, M., Taylor, S., & Flannigan, M. D. (2020) · Environmental Reviews · doi:10.1139/er-2020-0019 · Cited by 765
WeatherBench: A Benchmark Data Set for Data‐Driven Weather Forecasting
Rasp, S., Dueben, P. D., Scher, S., Weyn, J. A., Mouatadid, S., & Thuerey, N. (2020) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2020ms002203 · Cited by 663
Physics-informed machine learning: case studies for weather and climate modelling
Kashinath, K., Mustafa, M., Albert, A., Wu, J.-L., Jiang, C., Esmaeilzadeh, S., Azizzadenesheli, K., Wang, R., Chattopadhyay, A., Singh, A., et al. (2021) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2020.0093 · Cited by 662
Heat Waves: Physical Understanding and Scientific Challenges
Barriopedro, D., García‐Herrera, R., Ordóñez, C., Miralles, D. G., & Salcedo‐Sanz, S. (2023) · Reviews of Geophysics · doi:10.1029/2022rg000780 · Cited by 404
Metadata
- Authors
- McGovern, A., Lagerquist, R., John Gagne, D., Jergensen, G. E., Elmore, K. L., Homeyer, C. R., & Smith, T.
- Venue
- Bulletin of the American Meteorological Society
- Publisher
- American Meteorological Society
- Published
- 2019-11
- Volume
- 100
- Issue
- 11
- Pages
- 2175-2199
- ISSN
- 0003-0007 (ISSN-L)
- 1520-0477
- OpenAlex
- W2969309273
- Cited by
- 558
- References
- 79
- FWCI
- 25.79
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.