CoChat

Quantifying uncertainty in deep learning approaches to radio galaxy classification

Open access
Mohan, Devina, Scaife, Anna M M, Porter, Fiona, Walmsley, Mike, Bowles, Micah · 2022 · Monthly Notices of the Royal Astronomical Society

The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.

Abstract

In this work we use variational inference to quantify the degree of uncertainty in deep learning model predictions of radio galaxy classification. We show that the level of model posterior variance for individual test samples is correlated with human uncertainty when labelling radio galaxies. We explore the model performance and uncertainty calibration for different weight priors and suggest that a sparse prior produces more well-calibrated uncertainty estimates. Using the posterior distributions for individual weights, we demonstrate that we can prune 30 per cent of the fully connected layer weights without significant loss of performance by removing the weights with the lowest signal-to-noise ratio. A larger degree of pruning can be achieved using a Fisher information based ranking, but both pruning methods affect the uncertainty calibration for Fanaroff–Riley type I and type II radio galaxies differently. Like other work in this field, we experience a cold posterior effect, whereby the posterior must be down-weighted to achieve good predictive performance. We examine whether adapting the cost function to accommodate model misspecification can compensate for this effect, but find that it does not make a significant difference. We also examine the effect of principled data augmentation and find that this improves upon the baseline but also does not compensate for the observed effect. We interpret this as the cold posterior effect being due to the overly effective curation of our training sample leading to likelihood misspecification, and raise this as a potential issue for Bayesian deep learning approaches to radio galaxy classification in future.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/mnras/stac223

Cites

48 of 48 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 48 references

Research works printed but not linked: 20

  1. Abdar, 2021, Inf. Fusion
  2. Aitchison, 2021, International Conference on Learning Representations
  3. Alquier, 2016, J. Mach. Learn. Res., 17, 8374
  4. Blundell, 2015, 32nd International Conference on International Conference on Machine Learning - Vol. 37, 1613
  5. Gal, 2016, Uncertainty in Deep Learning
  6. Gal, 2016, International Conference on Learning Representations
  7. 2000, A&A, 363, 507
  8. Graves, 2011, Proceedings of the 24th International Conference on Neural Information Processing Systems. NIPS’11, 2348
  9. Guo, 2017, International Conference on Machine Learning, 1321
  10. Guo, 2017, Proceedings of the 34th International Conference on Machine Learning - Vol. 70. ICML’17, 1321
  11. Izmailov, 2021, Proceedings of Machine Learning Research Vol. 139, Proceedings of the 38th International Conference on Machine Learning, 4629
  12. Kingma, 2015, Adv. Neural Inf. Process. Syst., 28, 2575
  13. Krishnan, 2020, Adv. in Neural Info. Proc. Syst., 18237
  14. Kunstner, 2019, Vol. 32, Advances in Neural Information Processing Systems
  15. LeCun, 1989, Advances in Neural Information Processing Systems, 598
  16. Masegosa, 2019, Adv. in Neural Info. Proc. Syst., 5479
  17. Nguyen, 2015, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 427
  18. Peterson, 2019, Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 9617
  19. Shalaeva, 2020, Proceedings of the AAAI Conference on Artificial Intelligence, 5660
  20. Wenzel, 2020, Proceedings of Machine Learning Research Vol. 119, Proceedings of the 37th International Conference on Machine Learning, 10248

Cited by

See all 27

Metadata

Authors
Mohan, D., Scaife, A. M. M., Porter, F., Walmsley, M., & Bowles, M.
Affiliations
University of Manchester (GB)
Corresponding author
Mohan, Devina
Venue
Monthly Notices of the Royal Astronomical Society
Published
2022-01-29
Volume
511
Issue
3
Pages
3722-3740
ISSN
  • 0035-8711 (ISSN-L)
  • 1365-2966
OpenAlex
W4210818292
Cited by
16
References
48
Citations by year
2026: 3 · 2025: 3 · 2024: 1 · 2023: 5 · 2022: 4
FWCI
1.30
Type
journal article
Language
en
Open access
Locations
  • Monthly Notices of the Royal Astronomical Society (publishedVersion, cc-by)
  • Research Explorer (The University of Manchester) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Strathprints: The University of Strathclyde institutional repository (University of Strathclyde) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
  • Alan Turing Institute (EP/V030302/1)
  • STFC
  • IBM (ST/P006795/1)
Topics
  • Physical Sciences › Physics and Astronomy › Astronomy and Astrophysics › Galaxies: Formation, Evolution, Phenomena
  • Physical Sciences › Physics and Astronomy › Astronomy and Astrophysics › Radio Astronomy Observations and Technology
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
radio galaxy classification, uncertainty quantification, variational inference, Bayesian deep learning, uncertainty calibration, weight pruning, signal-to-noise ratio, Fisher information, data augmentation, model misspecification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Quantifying uncertainty in deep learning approaches to radio galaxy classification. https://app.cochat.ai/papers/10.1093/mnras/stac223
The paper
Mohan, D., Scaife, A. M. M., Porter, F., Walmsley, M., & Bowles, M. (2022). Quantifying uncertainty in deep learning approaches to radio galaxy classification. Monthly Notices of the Royal Astronomical Society. https://doi.org/10.1093/mnras/stac223

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.