PreprintCited by 64Galaxies: Formation, Evolution, PhenomenaAdvanced Neural Network ApplicationsAstronomical Observations and Instrumentation

The DOI matches its DataCite record.

Practical Galaxy Morphology Tools from Deep Supervised Representation Learning

Open access

Abstract

Astronomers have typically set out to solve supervised machine learning problems by creating their own representations from scratch. We show that deep learning models trained to answer every Galaxy Zoo DECaLS question learn meaningful semantic representations of galaxies that are useful for new tasks on which the models were never trained. We exploit these representations to outperform several recent approaches at practical tasks crucial for investigating large galaxy samples. The first task is identifying galaxies of similar morphology to a query galaxy. Given a single galaxy assigned a free text tag by humans (e.g. "#diffuse"), we can find galaxies matching that tag for most tags. The second task is identifying the most interesting anomalies to a particular researcher. Our approach is 100% accurate at identifying the most interesting 100 anomalies (as judged by Galaxy Zoo 2 volunteers). The third task is adapting a model to solve a new task using only a small number of newly-labelled galaxies. Models fine-tuned from our representation are better able to identify ring galaxies than models fine-tuned from terrestrial images (ImageNet) or trained from scratch. We solve each task with very few new labels; either one (for the similarity search) or several hundred (for anomaly detection or fine-tuning). This challenges the longstanding view that deep supervised methods require new large labelled datasets for practical use in astronomy. To help the community benefit from our pretrained models, we release our fine-tuning code Zoobot. Zoobot is accessible to researchers with no prior experience in deep learning.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2110.12735

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 112

  1. S Abadi M. et al., 2016, preprint ( arXiv:1603.04467 ) · arXiv
  2. Abd El Aziz M., Selim I. M., Xiong S., 2017, Sci. Rep. , 7, 1 · DOI
  3. Ackermann S., Schawinski K., Zhang C., Weigel A. K., Turp M. D., 2018, MNRAS , 479, 415 · DOI · Paper page
  4. Agarap A. F., 2018, preprint ( arXiv:1803.08375 ) · arXiv
  5. Aggarwal C. C., Hinneburg A., Keim D. A., 2001, in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Springer, New York, p. 420
  6. Albareti F. D. et al., 2017, ApJS , 233, 25 · DOI
  7. Ardizzone E., Di Ges ù V., Maccarone M. C., 1996, Vistas Astron. , 40, 401 · DOI
  8. Athanassoula E., Romero-G ómez M., Masdemont J. J., 2009, MNRAS , 394, 67 · DOI
  9. Austin J. et al., 2021, preprint ( arXiv:2108.07732 ) · arXiv
  10. Barchi P. H. et al., 2020, Astron. Comput. , 30, 100334 · DOI
  11. Baron D., Poznanski D., 2017, MNRAS , 465, 4530 · DOI
  12. Boyajian T. S. et al., 2016, MNRAS , 457, 3988 · DOI · Paper page
  13. Breiman L., 2001, Mach. Learn. , 45, 5 · DOI · Paper page
  14. Brown T. B. et al., 2020, in Larochelle H., Ranzato M., Hadsell R., Balcan M. F., Lin H., eds, Advances in Neural Information Processing Systems, Vol. 33, Curran Associates, Inc., Vancouver, BC, p. 1877
  15. Buncher B., Sharma A. N., Carrasco Kind M., 2021, MNRAS , 503, 777 · DOI
  16. Buta R. J., 2013, Planets, Stars and Stellar Systems: Volume 6: Extragalactic Astronomy and Cosmology. Springer, Netherlands, p.
  17. Buta R. J., 2017, MNRAS , 471, 4027 · DOI
  18. Buta R. J. et al., 2015, ApJS , 217, 32 · DOI
  19. Buta R. J. et al., 2019, MNRAS , 488, 2175 · DOI
  20. Cardamone C. et al., 2009, MNRAS , 399, 1191 · DOI
  21. Caron M., Touvron H., Misra I., J égou H., Mairal J., Bojanowski P., Joulin A., 2021, Proceedings of the IEEE/International Conference on Computer Vision. IEEE, Montreal, QC
  22. Cheng T.-Y., Huertas-Company M., Conselice C. J., Arag ón-Salamanca A., Robertson B. E., Ramachandra N., 2021, MNRAS , 503, 4446 · DOI
  23. Comer ón S. et al., 2014, A&A , 562, 16 · DOI
  24. Csillaghy A., Hinterberger H., Benz A. O., 2000, Inform. Retr. , 3, 229 · DOI
  25. Danka T., Horvath P., 2018, preprint ( arXiv:1805.00979 ) · arXiv
  26. Das S., Wong W.-K., Fern A., Dietterich T. G., Siddiqui M. A., 2017, preprint ( arXiv:1708.09441 ) · arXiv
  27. Dey A. et al., 2019, AJ , 157, 168 · DOI
  28. Dieleman S., Willett K. W., Dambre J., 2015, MNRAS , 450, 1441 · DOI · Paper page
  29. Dobbels W., Krier S., Pirson S., Viaene S., De Geyter G., Salim S., Baes M., 2019, A&A , 624, A102 · DOI
  30. Dom ínguez S ánchez H., Huertas-Company M., Bernardi M., Tuccillo D., Fischer J. L., 2018, MNRAS , 476, 3661 · DOI · Paper page
  31. Dominguez Sanchez H. et al., 2019, MNRAS , 484, 93 · DOI · Paper page
  32. Fedus W., Zoph B., Shazeer N., 2021, preprint ( arXiv:2101.03961 ) · arXiv
  33. Fei-Fei L., Fergus R., Perona P., 2006, IEEE Trans. Pattern Anal. Mach.Intell. , 28, 594 · DOI · Paper page
  34. Fischer J. L., Dom ínguez S ánchez H., Bernardi M., 2019, MNRAS , 483, 2057 · DOI
  35. Fort S., Hu H., Lakshminarayanan B., 2019, preprint ( arXiv:1912.02757 ) · arXiv
  36. Ghosh A., Urry C. M., Wang Z., Schawinski K., Turp D., Powell M. C., 2020, ApJ , 895, 112 · DOI · Paper page
  37. Goodfellow I., Bengio Y., Courville A., 2016, Deep Learning. MIT Press, Boston, MA
  38. Grill J. B. et al., 2020, Advances in Neural Information Processing Systems. Neural Information Processing Systems Foundation, San Diego
  39. Hart R. E. et al., 2016, MNRAS , 461, 3663 · DOI
  40. Hayat M. A., Stein G., Harrington P., Luki ć Z., Mustafa M., 2021, ApJ , 911, L33 · DOI · Paper page
  41. He K., Girshick R., Doll ár P., 2019, Proceedings of the IEEE/International Conference on Computer Vision. Computer Vision Foundation, Seoul, p. 4918
  42. Hendrycks D., Dietterich T., 2019, preprint ( arXiv:1903.12261 ) · arXiv
  43. Henrion M., Mortlock D. J., Hand D. J., Gandy A., 2013, in Hilbe J. M., ed., Astrostatistical Challenges for the New Astronomy. Springer, Berlin, p. 149
  44. Hocking A., Geach J. E., Sun Y., Dav e y N., 2018, MNRAS , 473, 1108 · DOI
  45. Houlsby N., 2014, PhD thesis, Univ. Cambridge
  46. Hunter J. D., 2007, Comput. Sci. Eng. , 9, 99 · DOI · Paper page
  47. Ilyas A., Santurkar S., Tsipras D., Engstrom L., Tran B., Madry A., 2019, in Wallach H., Larochelle H., Beygelzimer A., d'Alch é-Buc F., Fox E., Garnett R., eds, Advances in Neural Information Processing Systems.
  48. Ioffe S., Szegedy C., 2015, Proceedings of the 32nd International Conference on Machine Learning. JMLR Workshop and Conference Proceedings, Lille, p. 448
  49. Jones D. R., Schonlau M., Welch W. J., 1998, J. Global Optim. , 13, 455 · DOI · Paper page
  50. Kant N., Puri R., Yako v enko N., Catanzaro B., 2018, preprint ( arXiv:1812.01207 ) · arXiv
  51. Kaplan J. et al., 2020, preprint ( arXiv:2001.08361 ) MNRAS 513, 1581-1599 (2022) · arXiv
  52. Khalifa N. E., Hamed Taha M., Hassanien A. E., Selim I., 2018, 2018 In- ternational Conference on Computing Sciences and Engineering, ICCSE 2018 -Proceedings.
  53. Khramtsov V., Dobryche v a D. V., Vasylenko M. Y., Akhmetov V. S., 2019, Odessa Astron. Publ. , 32, 21 · DOI
  54. Kong L., Chen L., Chen M., Bhatia P., Callot L., 2020, preprint ( arXiv:2009.07241 ) · arXiv
  55. Laureijs R. et al., 2011, preprint ( arXiv:1110.3193 ) · arXiv
  56. Lavery R. J., Remijan A., Charmandaris V., Hayes R. D., Ring A. A., 2004, ApJ , 612, 679 · DOI
  57. LeCun Y. A., Bengio Y., Hinton G. E., 2015, Nature , 521, 436 · DOI · Paper page
  58. Lintott C. J., 2019, The Crowd and the Cosmos: Adventures in the Zooniverse. Oxford Univ. Press, Oxford
  59. Lintott C. J. et al., 2009, MNRAS , 399, 129 · DOI
  60. Liu F. T., Ting K. M., Zhou Z. H., 2008, Proceedings -IEEE International Conference on Data Mining, ICDM. IEEE Computer Society, Washing- ton, DC, p. 413
  61. Lochner M., Bassett B. A., 2021, Astron. Comput. , 36, 100481 (LB21) · DOI · Paper page
  62. McInnes L., Healy J., Saul N., Großberger L., 2018, J. Open Source Softw. , 3, 861 · DOI · Paper page
  63. McKinney W., 2010, Data Structures for Statistical Computing in Python. Available at: http://conference.scip y.org/proceedings/scip y2010/mckin ney.html
  64. Marmanis D., Datcu M., Esch T., Stilla U., 2016, IEEE Geosci. Remote Sens. Lett. , 13, 105 · DOI
  65. Martinazzo A., Espadoto M., Hirata N. S. T., 2021, 2020 -25th International Conference on Pattern Recognition (ICPR). IEEE, Milan, Italy, p. 4176
  66. Mathis A., Biasi T., Rogers B., Bethge M., Weygandt Mathis M., 2020, Uncer- tainty in Deep Learning (ICML Workshop). Available at: http://www.gats by.ucl.ac.uk/ ∼ balaji/ udl2020/ accepted-papers/ UDL2020-paper-047.pdf
  67. Mockus J. B., Mockus L. J., 1991, J. Optim. Theory Appl. , 70, 157 · DOI
  68. Moiseev A. V., Smirnova K. I., Smirnova A. A., Reshetnikov V. P., 2011, MNRAS , 418, 244 · DOI
  69. Moosavi-Dezfooli S. M., F a wzi A., F a wzi O., Frossard P., 2017, Proceedings -30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. Institute of Electrical and Electronics Engineers Inc., New York, p.
  70. Murphy K. P., 2012, Machine Learning: A Probabilisitic Perspective. MIT Press, Boston, MA
  71. Nair P. B., Abraham R. G., 2010, ApJS , 186, 427 · DOI
  72. Pedregosa F. et al., 2012, J. Mach. Learn. Res. , 12, 2825 · DOI
  73. Pelleg D., Moore A., 2004, Proceedings of the 17th International Conference on Neural Information Processing Systems. MIT Press, Cambridge, MA, p. 1073
  74. Ralph N. O. et al., 2019, PASP , 131, 108011 · DOI
  75. Rasmussen C. E., Williams C. K. I., 2006, Gaussian Processes for Machine Learning. Adaptive Computation and Machine Learning, MIT Press, Cambridge, MA
  76. Recht B., Roelofs R., Schmidt L., Shankar V., 2019, in Chaudhuri K., Salakhutdinov R., eds, Proceedings of the 36th International Conference on Machine Learning, Vol. 97, PMLR, Long Beach, CA, p. 5389
  77. Reis I., Rotman M., Poznanski D., Prochaska J. X., Wolf L., 2021, Astron. Comput. , 34, 100437 · DOI
  78. Ridnik T., Ben-Baruch E., Noy A., Zelnik-Manor L., 2021, in Proceedings of Neural Information Processing Systems. MIT Press, Boston, MA
  79. Ross D. A., Lim J., Lin R. S., Yang M. H., 2008, Int. J. Comput. Vision , 77, 125 · DOI · Paper page
  80. Russako vsk y O. et al., 2015, Int. J. Comput. Vision , 115, 211 · DOI · Paper page
  81. Sandler M., Howard A., Zhu M., Zhmoginov A., Chen L. C., 2018, in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, New York, p. 4510
  82. Sarmiento R., Huertas-Company M., Knapen J. H., S ánchez S. F., S ánchez H. D., Drory N., Falc ón-Barroso J., 2021, ApJ , 921, 177 · DOI · Paper page
  83. Schutter A., Shamir L., 2015, Astron. Comput. , 12, 60 · DOI
  84. Schwarz M. P., 1981, ApJ , 247, 77 · DOI
  85. Shamir L., 2020, MNRAS , 491, 3767 · DOI
  86. Sharma U., Kaplan J., 2020, preprint ( arXiv:2004.10802 ) · arXiv
  87. Siddiqui M. A., Wright R., Fern A., Theriault A., Dietterich T. G., Archer D. W., 2018, in Proceedings of the ACM SIGKDD International Conference on Knowledge Disco v ery and Data Mining. Association for Computing Machinery, p. 2200
  88. Simmons B. et al., 2013, MNRAS , 429, 2199 · DOI
  89. Simonyan K., Zisserman A., 2015, International Conference on Learning Representations. International Conference on Representation Learning, San Diego
  90. Smethurst R. J., Lintott C. J., Bamford S. P., Hart R. E., Kruk S. J., Masters K. L., Nichol R. C., Simmons B. D., 2017, MNRAS , 469, 3670 · DOI
  91. Spindler A., Geach J. E., Smith M. J., 2020, MNRAS , 502, 985 · DOI
  92. Stein G., Harrington P., Blaum J., Medan T., Lukic Z., 2021, preprint ( arXiv:2110.13151 ) · arXiv
  93. Store y-Fisher K., Huertas-Compan y M., Ramachandra N., Lanusse F., Leau- thaud A., Luo Y., Huang S., Prochaska J. X., 2021, MNRAS , 508, 2946 · DOI · Paper page
  94. Struck C., 2010, MNRAS , 403, 1516 · DOI
  95. Szegedy C. et al., 2015, in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Society, Washington, DC
  96. Tan M., Le Q. V., 2019, in 36th International Conference on Machine Learning, ICML 2019. p. 10691
  97. Tang H., Scaife A. M., Leahy J. P., 2019, MNRAS , 488, 3358 · DOI
  98. The Astropy Collaboration, 2018, AJ , 156, 123 · DOI
  99. Timmis I., Shamir L., 2017, ApJS , 231, 2 · DOI
  100. Tschandl P., Sinz C., Kittler H., 2019, Comput. Biol. Med. , 104, 111 · DOI
  101. Van Den Oord A., Kalchbrenner N., Vinyals O., Espeholt L., Graves A., Kavukcuoglu K., 2016, Advances in Neural Information Processing Systems. Neural Information Processing Systems Foundation.
  102. van der Walt S., Sch önberger J. L., Nunez-Iglesias J., Boulogne F ., W arner J. D., Yager N., Gouillart E., Yu T., 2014, PeerJ , 2, e453 · DOI
  103. Variawa M. Z., van Zyl T. L., Wool w ay M., 2020, 2020 IEEE 23rd International Conference on Information Fusion (FUSION), p.
  104. Vaswani A., Shazeer N., Parmar N., Uszkoreit J., Jones L., Gomez A. N., Kaiser L., Polosukhin I., 2017, Advances in Neural Information Processing Systems, Vol. 2017, Curran Associates. Inc., Long Beach, CA, p. 5999
  105. Walmsley M. et al., 2020, MNRAS , 491, 1554 · DOI · Paper page
  106. Walmsley M. et al., 2022, MNRAS , 509, 3966 (W + 22 · DOI
  107. Welsh W. F. et al., 2011, ApJS , 197, 4 · DOI
  108. Willett K. W. et al., 2013, MNRAS , 435, 2835 · DOI · Paper page
  109. Wu C. et al., 2018, MNRAS , 1230, 1211 · DOI
  110. Yang K., Qinami K., Fei-Fei L., Deng J., Russako vsk y O., 2020a, in FAT * 2020 -Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. Association for Computing Machinery, Inc., p. 547
  111. Yang Z., Zhu C., Gmyr R., Zeng M., Huang X., Darve E., 2020b, in Findings of the Association for Computational Linguistics Findings of ACL: EMNLP 2020. Association for Computational Linguistics (ACL), Virtual, p. 1865
  112. Zanisi L. et al., 2021, MNRAS , 501, 4359 · DOI · Paper page

Cited by

Sorted by citations

Metadata

Authors
Walmsley, M., Scaife, A. M. M., Lintott, C., Lochner, M., Etsebeth, V., Géron, T., Dickinson, H., Fortson, L., Kruk, S., Masters, K. L., Mantha, K. B., & Simmons, B. D.
Affiliations
  • University of Manchester (GB)
  • University of Oxford (GB)
  • South African Radio Astronomy Observatory (ZA)
  • University of the Western Cape (ZA)
  • The Open University (GB)
  • University of Minnesota (US)
  • European Space Agency (FR)
  • Haverford College (US)
  • Lancaster University (GB)
Corresponding author
Walmsley, Mike
Venue
arXiv
Publisher
arXiv
Published
2021
Volume
513
Issue
2
Pages
1581-1599
ISSN
0035-8711 (ISSN-L)
OpenAlex
W3209309880
Cited by
64
References
91
Citations by year
2026: 7 · 2025: 24 · 2024: 18 · 2023: 12 · 2022: 3
FWCI
5.60
Type
journal article
Language
en
Open access
Locations
  • Monthly Notices of the Royal Astronomical Society (publishedVersion, cc-by)
  • Research Explorer (The University of Manchester) (publishedVersion, other-oa)
  • Open Research Online (The Open University) (acceptedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Lancaster EPrints (Lancaster University) (submittedVersion)
  • OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) (submittedVersion)
  • UWC Research Repository (University of the Western Cape) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Science Foundation (2014B-0404, 2006894, 0950945)
  • U.S. Department of Energy (DE‐AC02‐05CH1123, AST-0950945, AC02-05CH1123, DE-AC02-)
  • National Aeronautics and Space Administration
  • University of Pennsylvania
  • Ohio State University
  • California Institute of Technology
  • University of Chicago
  • University of Portsmouth
  • Alan Turing Institute (EP/V030302/1)
  • University College London
  • University of Sussex
  • National Research Foundation
  • Deutsche Forschungsgemeinschaft
  • Financiadora de Estudos e Projetos
  • University of Illinois at Urbana-Champaign
  • Office of Science (DE‐AC02‐05CH1123, DE-AC02)
  • Jet Propulsion Laboratory
  • Science and Technology Facilities Council (ST/T000414/1)
  • Division of Astronomical Sciences (AST‐0950945)
  • High Energy Physics (DE-AC02-05CH1123)
  • Integrated Electronics Engineering Center, Binghamton University
  • Argonne National Laboratory (DE-AC02)
  • Fermilab
  • Lawrence Berkeley National Laboratory (DE‐AC02‐05CH1123)
  • SLAC National Accelerator Laboratory
Topics
  • Physical Sciences › Physics and Astronomy › Astronomy and Astrophysics › Galaxies: Formation, Evolution, Phenomena
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Physics and Astronomy › Astronomy and Astrophysics › Astronomical Observations and Instrumentation
Keywords
galaxy morphology, semantic representation, transfer learning, fine-tuning, anomaly detection, similarity search, ring galaxies, Galaxy Zoo, DECaLS, pre-trained models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review