The impact of human expert visual inspection on the discovery of strong gravitational lenses
Open accessAbstract
We investigate the ability of human ‘expert’ classifiers to identify strong gravitational lens candidates in Dark Energy Survey like imaging. We recruited a total of 55 people that completed more than 25 per cent of the project. During the classification task, we present to the participants 1489 images. The sample contains a variety of data including lens simulations, real lenses, non-lens examples, and unlabelled data. We find that experts are extremely good at finding bright, well-resolved Einstein rings, while arcs with g-band signal to noise less than ∼25 or Einstein radii less than ∼1.2 times the seeing are rarely recovered. Very few non-lenses are scored highly. There is substantial variation in the performance of individual classifiers, but they do not appear to depend on the classifier’s experience, confidence or academic position. These variations can be mitigated with a team of 6 or more independent classifiers. Our results give confidence that humans are a reliable pruning step for lens candidates, providing pure and quantifiably complete samples for follow-up studies.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
We have ignored the fallibility of expert humans at lens finding.
“We have ignored the fallibility of expert humans at lens finding (Rojas et al. 2023).”
From Euclid Quick Data Release (Q1): XXVIII. The Strong Lensing Discovery Engine C – Finding lenses with machine learning (2026) · cites this paper as Rojas et al. 2023
Cites
CMU DeepLens: deep learning for automatic image-based galaxy–galaxy strong lens finding
Lanusse, F., Ma, Q., Li, N., Collett, T. E., Li, C.-L., Ravanbakhsh, S., Mandelbaum, R., & Póczos, B. (2018) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stx1665 · Cited by 191 · see citing works
Self-citationFinding strong lenses in CFHTLS using convolutional neural networks
Jacobs, C., Glazebrook, K., Collett, T., More, A., & McCarthy, C. (2017) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stx1492 · Cited by 133 · cited 1× · see citing works
“As astronomy enters the era of billion object surveys, sophisticated methods for discovering strong lenses have been developed (e.g. Lanusse et al. 2018; Avestruz et al. 2019) and applied (Jacobs et al. 2017, 2019b,a; Rojas et al. 2022; Savary et al. 2022; Petrillo et al. 2017, 2019).”
MethodSelf-citationSpace Warps – I. Crowdsourcing the discovery of gravitational lenses
Marshall, P. J., Verma, A., More, A., Davis, C. P., More, S., Kapadia, A., Parrish, M., Snyder, C., Wilcox, J., Baeten, E., et al. (2016) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stv2009 · Cited by 117 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Cañameras, R., Schuldt, S., Shu, Y., Suyu, S. H., Taubenberger, S., Andika, I. T., Bag, S., Inoue, K. T., Jaelani, A. T., Leal-Taixé, L., et al. (2024) · Astronomy & Astrophysics · doi:10.1051/0004-6361/202347072 · Cited by 19 · see citing works
Self-citationEuclid Collaboration, Lines, N. E. P., Collett, T. E., Walmsley, M., Rojas, K., Li, T., Leuzzi, L., Manjón-García, A., Vincken, S. H., Wilde, J., et al. (2026) · Astronomy & Astrophysics · doi:10.1051/0004-6361/202554542 · Cited by 14 · see citing works
Self-citation
- Authors
- Rojas, K., Collett, T. E., Ballard, D., Magee, M. R., Birrer, S., Buckley-Geer, E., Chan, J. H. H., Clément, B., Diego, J. M., Gentile, F., González, J., Joseph, R., Mastache, J., Schuldt, S., Tortora, C., Verdugo, T., Verma, A., Daylan, T., Millon, M., Jackson, N., Dye, S., Melo, A., Mahler, G., Ogando, R. L. C., Courbin, F., Fritz, A., Herle, A., Acevedo Barroso, J. A., Cañameras, R., Cornen, C., Dhanasingham, B., Glazebrook, K., Martinez, M. N., Ryczanowski, D., Savary, E., Góis-Silva, F., Arturo Ureña-López, L., Wiesner, M. P., Wilde, J., Valim Calçada, G., Cabanac, R., Pan, Y., Sierra, I., Despali, G., Cavalcante-Gomes, M. V., Macmillan, C., Maresca, J., Grudskaia, A., O’Donnell, J. H., Paic, E., Niemiec, A., de la Bella, L. F., Bromley, J., Williams, D. M., More, A., & Levine., B. C.
- Affiliations
- University of Portsmouth (GB)
- SLAC National Accelerator Laboratory (US)
- Fermi National Accelerator Laboratory , P.O. Box 500, Batavia, IL 60510, USA
- American Museum of Natural History (US)
- École Polytechnique Fédérale de Lausanne (CH)
- Instituto de Física de Cantabria (CSIC-UC), Avda Los Castros s/n, E-39095 Santander, Spain
- Osservatorio di Astrofisica e Scienza dello Spazio (IT)
- University of Wisconsin–Madison (US)
- Stockholm University (SE)
- Universidad Autónoma de Chiapas (MX)
- Max Planck Institute for Astrophysics (DE)
- Astronomical Observatory of Capodimonte (IT)
- Universidad Nacional Autónoma de México (MX)
- University of Oxford (GB)
- Princeton University (US)
- University of Manchester (GB)
- University of Nottingham (GB)
- Durham University (GB)
- National Observatory (BR)
- OmegaLambdaTec GmbH , Lichtenbergstraße 8, D-85748 Garching, Germany
- Institute of Physics, Laboratory of Astrophysics , Ecole Polytechnique Fédérale de Lausanne (EPFL), Observatoire de Sauverny, CH-1290 Versoix, Switzerland
- University of New Mexico (US)
- Swinburne University of Technology (AU)
- University of Birmingham (GB)
- Departamento de Física, DCI, Campus León, Universidad de Guanajuato , León, Guanajuato 37150, México
- Benedictine University (US)
- The Open University (GB)
- Observatório Nacional , Rua Gal. José Cristino 77, Rio de Janeiro, RJ – 20921-400, Brazil
- Centre National de la Recherche Scientifique (FR)
- University of Chicago (US)
- Heidelberg University (DE)
- Ludwig-Maximilians-Universität München (DE)
- University of California, Santa Cruz (US)
- Institute of Cosmology and Gravitation, University of Portsmouth , Burnaby Rd, Portsmouth PO1 3FX, UK
- Kavli Institute for the Physics and Mathematics of the Universe (JP)
- Department of Physics and Astronomy, Stony Brook University , Stony Brook, NY 11794, USA
- Corresponding author
- Rojas, Karina
- Collett, Thomas E
- ORCID
- Rojas, Karina
- Collett, Thomas E
- Ballard, Daniel
- Magee, Mark R
- Birrer, Simon
- Buckley-Geer, Elizabeth
- Chan, James H H
- Clément, Benjamin
- Diego, José M
- Gentile, Fabrizio
- Joseph, Rémy
- Mastache, Jorge
- Schuldt, Stefan
- Tortora, Crescenzo
- Verdugo, Tomás
- Verma, Aprajita
- Daylan, Tansu
- Millon, Martin
- Jackson, Neal
- Dye, Simon
- Melo, Alejandra
- Mahler, Guillaume
- Ogando, Ricardo L C
- Courbin, Frédéric
- Fritz, Alexander
- Cañameras, Raoul
- Cornen, Claude
- Dhanasingham, Birendra
- Glazebrook, Karl
- Martinez, Michael N
- Ryczanowski, Dan
- Góis-Silva, Filipe
- Wilde, Joshua
- Cabanac, Rémi
- Pan, Yue
- Sierra, Isaac
- Despali, Giulia
- Maresca, Jacob
- O’Donnell, Jackson H
- Paic, Eric
- Niemiec, Anna
- Bromley, Jane
- Williams, Devon M
- More, Anupreeta
- Venue
- Monthly Notices of the Royal Astronomical Society
- Publisher
- Oxford University Press (OUP)
- Published
- 2023-06-07
- Volume
- 523
- Issue
- 3
- Pages
- 4413-4430
- ISSN
- 0035-8711 (ISSN-L)
- 1365-2966
- OpenAlex
- W4315705762
- Cited by
- 22
- References
- 3
- Citations by year
- 2026: 4 · 2025: 6 · 2024: 11 · 2023: 1
- FWCI
- 2.56
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Monthly Notices of the Royal Astronomical Society (publishedVersion, cc-by)
- Repository@Nottingham (University of Nottingham) (publishedVersion, other-oa)
- arXiv (Cornell University) (submittedVersion, cc-by)
- HAL (submittedVersion, cc-by)
- ORBi (University of Liège) (submittedVersion, cc-by)
- Open Research Online (The Open University) (submittedVersion, cc-by)
- OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) (submittedVersion, cc-by)
- Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) (submittedVersion, cc-by)
- DIGITAL.CSIC (Spanish National Research Council (CSIC)) (submittedVersion, cc-by)
- Dipòsit Digital de la Universitat de Barcelona (Universitat de Barcelona) (submittedVersion, public-domain)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Repository@Nottingham (University of Nottingham) (submittedVersion, cc-by)
- Research Explorer (The University of Manchester) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Greater South Information System
- Funders
- STFC
- Swiss National Science Foundation
- European Research Council (787886, 771776)
- Max Planck Society
- Deutsche Forschungsgemeinschaft
- John Templeton Foundation (#62192)
- NASA (80NSSC18K1014)
- CONACYT (A1-S-17899, 286897, 297771, 304001)
- Science and Technology Facilities Council (ST/P006760/1)
- Topics
- Physical Sciences › Physics and Astronomy › Astronomy and Astrophysics › Astrophysical Phenomena and Observations
- Physical Sciences › Physics and Astronomy › Astronomy and Astrophysics › Galaxies: Formation, Evolution, Phenomena
- Physical Sciences › Physics and Astronomy › Astronomy and Astrophysics › Stellar, planetary, and galactic studies
- Keywords
- gravitational lensing, human visual inspection, citizen science, Dark Energy Survey, Einstein ring, classification performance, lens candidate selection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
