The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
TPZ: photometric redshift PDFs and ancillary information by using prediction trees and random forests
Open accessAbstract
With the growth of large photometric surveys, accurately estimating photometric redshifts, preferably as a probability density function (PDF), and fully understanding the implicit systematic uncertainties in this process, has become increasingly important. In this paper, we present a new, publicly available, parallel, machine learning algorithm that generates photometric redshift PDFs by using prediction trees and random forest techniques, which we have named TPZ.1 This new algorithm incorporates measurement errors into the calculation while also dealing efficiently with missing values in the data. In addition, our implementation of this algorithm provides supplementary information regarding the data being analysed, including unbiased estimates of the accuracy of the technique without resorting to a validation data set, identification of poor photometric redshift areas within the parameter space occupied by the spectroscopic training data, a quantification of the relative importance of the variables used to construct the PDF, and a robust identification of outliers. This extra information can be used to optimally target new spectroscopic observations and to improve the overall efficacy of the redshift estimation. We have tested TPZ on galaxy samples drawn from the Sloan Digital Sky Survey (SDSS) main galaxy sample and from the Deep Extragalactic Evolutionary Probe-2 (DEEP2) survey, obtaining excellent results in each case. We also have tested our implementation by participating in the PHAT1 project, which is a blind photometric redshift contest, finding that TPZ performs comparable to if not better than other empirical photometric redshift algorithms. Finally, we discuss the various parameters that control the operation of TPZ, the specific limitations of this approach and an application of photometric redshift PDFs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00058655 · Cited by 16,325
Stellar population synthesis at the resolution of 2003
Bruzual, G., & Charlot, S. (2003) · Monthly Notices of the Royal Astronomical Society · doi:10.1046/j.1365-8711.2003.06897.x · Cited by 10,500
The Sloan Digital Sky Survey: Technical Summary
York, D. G., Adelman, J., Anderson, Jr., Anderson, S. F., Annis, J., Bahcall, N. A., Bakken, J. A., Barkhouser, R., Bastian, S., Berman, E., et al. (2000) · The Astronomical Journal · doi:10.1086/301513 · Cited by 10,366
Self-citationProceedings of the 25th international conference on Machine learning - ICML '08
(2008) · doi:10.1145/1390156 · Cited by 6,680
THE SEVENTH DATA RELEASE OF THE SLOAN DIGITAL SKY SURVEY
Abazajian, K. N., Adelman-McCarthy, J. K., Agüeros, M. A., Allam, S. S., Prieto, C. A., An, D., Anderson, K. S. J., Anderson, S. F., Annis, J., Bahcall, N. A., et al. (2009) · The Astrophysical Journal Supplement Series · doi:10.1088/0067-0049/182/2/543 · Cited by 5,341
Self-citationBoosting the margin: a new explanation for the effectiveness of voting methods
Bartlett, P., Freund, Y., Lee, W. S., & Schapire, R. E. (1998) · The Annals of Statistics · doi:10.1214/aos/1024691352 · Cited by 2,306
The Keck Low-Resolution Imaging Spectrometer
Oke, J. B., Cohen, J. G., Carr, M., Cromer, J., Dingizian, A., Harris, F. H., Labrecque, S., Lucinio, R., Schaal, W., Epps, H., et al. (1995) · Publications of the Astronomical Society of the Pacific · doi:10.1086/133562 · Cited by 2,099
Spectroscopic Target Selection in the Sloan Digital Sky Survey: The Main Galaxy Sample
Strauss, M. A., Weinberg, D. H., Lupton, R. H., Narayanan, V. K., Annis, J., Bernardi, M., Blanton, M., Burles, S., Connolly, A. J., Dalcanton, J., et al. (2002) · The Astronomical Journal · doi:10.1086/342343 · Cited by 1,909
Accurate photometric redshifts for the CFHT legacy survey calibrated using the VIMOS VLT deep survey
Ilbert, O., Arnouts, S., McCracken, H. J., Bolzonella, M., Bertin, E., Le Fèvre, O., Mellier, Y., Zamorani, G., Pellò, R., Iovino, A., et al. (2006) · Astronomy & Astrophysics · doi:10.1051/0004-6361:20065138 · Cited by 1,683
The Great Observatories Origins Deep Survey: Initial Results from Optical and Near-Infrared Imaging
Giavalisco, M., Ferguson, H. C., Koekemoer, A. M., Dickinson, M., Alexander, D. M., Bauer, F. E., Bergeron, J., Biagetti, C., Brandt, W. N., Casertano, S., et al. (2004) · The Astrophysical Journal · doi:10.1086/379232 · Cited by 1,678
Color Separation of Galaxy Types in the Sloan Digital Sky Survey Imaging Data
Strateva, I., Ivezić, Ž., Knapp, G. R., Narayanan, V. K., Strauss, M. A., Gunn, J. E., Lupton, R. H., Schlegel, D., Bahcall, N. A., Brinkmann, J., et al. (2001) · The Astronomical Journal · doi:10.1086/323301 · Cited by 1,599
Self-citationThe Sixth Data Release of the Sloan Digital Sky Survey
Adelman‐McCarthy, J. K., Agüeros, M. A., Allam, S. S., Allende Prieto, C., Anderson, K. S. J., Anderson, S. F., Annis, J., Bahcall, N. A., Bailer‐Jones, C. A. L., Baldry, I. K., et al. (2008) · The Astrophysical Journal Supplement Series · doi:10.1086/524984 · Cited by 1,486
Self-citationTHE EIGHTH DATA RELEASE OF THE SLOAN DIGITAL SKY SURVEY: FIRST DATA FROM SDSS-III
Aihara, H., Allende Prieto, C., An, D., Anderson, S. F., Aubourg, É., Balbinot, E., Beers, T. C., Berlind, A. A., Bickerton, S. J., Bizyaev, D., et al. (2011) · The Astrophysical Journal Supplement Series · doi:10.1088/0067-0049/193/2/29 · Cited by 1,443
Colors and magnitudes predicted for high redshift galaxies
Coleman, G. D., Wu, C.-C., & Weedman, D. W. (1980) · The Astrophysical Journal Supplement Series · doi:10.1086/190674 · Cited by 1,213
Bayesian Photometric Redshift Estimation
Benitez, N. (2000) · The Astrophysical Journal · doi:10.1086/308947 · Cited by 1,186
An empirical evaluation of supervised learning in high dimensions
Caruana, R., Karampatziakis, N., & Yessenalina, A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390169 · Cited by 524
ANN z: Estimating Photometric Redshifts Using Artificial Neural Networks
Collister, A. A., & Lahav, O. (2004) · Publications of the Astronomical Society of the Pacific · doi:10.1086/383254 · Cited by 512
Reid, B. A., Percival, W. J., Eisenstein, D. J., Verde, L., Spergel, D. N., Skibba, R. A., Bahcall, N. A., Budavari, T., Frieman, J. A., Fukugita, M., et al. (2010) · Monthly Notices of the Royal Astronomical Society · doi:10.1111/j.1365-2966.2010.16276.x · Cited by 487
Thick gas discs in faint dwarf galaxies
Roychowdhury, S., Chengalur, J. N., Begum, A., & Karachentsev, I. D. (2010) · Monthly Notices of the Royal Astronomical Society: Letters · doi:10.1111/j.1745-3933.2010.00835.x · Cited by 395
The Mass Assembly Histories of Galaxies of Various Morphologies in the GOODS Fields
Bundy, K., Ellis, R. S., & Conselice, C. J. (2005) · The Astrophysical Journal · doi:10.1086/429549 · Cited by 351
Slicing Through Multicolor Space: Galaxy Redshifts from Broadband Photometry
Connolly, A. J., Csabai, I., Szalay, A. S., Koo, D. C., Kron, R. G., & Munn, J. A. (1995) · The Astronomical Journal · doi:10.1086/117720 · Cited by 332
LOW-RESOLUTION SPECTRAL TEMPLATES FOR ACTIVE GALACTIC NUCLEI AND GALAXIES FROM 0.03 TO 30 μm
Assef, R. J., Kochanek, C. S., Brodwin, M., Cool, R., Forman, W., Gonzalez, A. H., Hickox, R. C., Jones, C., Le Floc'h, E., Moustakas, J., et al. (2010) · The Astrophysical Journal · doi:10.1088/0004-637x/713/2/970 · Cited by 314
The Team Keck Treasury Redshift Survey of the GOODS-North Field
Wirth, G. D., Willmer, C. N. A., Amico, P., Chaffee, F. H., Goodrich, R. W., Kwok, S., Lyke, J. E., Mader, J. A., Tran, H. D., Barger, A. J., et al. (2004) · The Astronomical Journal · doi:10.1086/420999 · Cited by 292
The Zurich Extragalactic Bayesian Redshift Analyzer and its first application: COSMOS
Feldmann, R., Carollo, C. M., Porciani, C., Lilly, S. J., Capak, P., Taniguchi, Y., Fèvre, O. L., Renzini, A., Scoville, N., Ajiki, M., et al. (2006) · Monthly Notices of the Royal Astronomical Society · doi:10.1111/j.1365-2966.2006.10930.x · Cited by 270
PHAT: PHoto-z Accuracy Testing
Hildebrandt, H., Arnouts, S., Capak, P., Moustakas, L. A., Wolf, C., Abdalla, F. B., Assef, R. J., Banerji, M., Benítez, N., Brammer, G. B., et al. (2010) · Astronomy & Astrophysics · doi:10.1051/0004-6361/201014885 · Cited by 258
The Application of Photometric Redshifts to the SDSS Early Data Release
Csabai, I., Budavári, T., Connolly, A. J., Szalay, A. S., Győry, Z., Benítez, N., Annis, J., Brinkmann, J., Eisenstein, D., Fukugita, M., et al. (2003) · The Astronomical Journal · doi:10.1086/345883 · Cited by 217
Estimating the redshift distribution of photometric galaxy samples
Lima, M., Cunha, C. E., Oyaizu, H., Frieman, J., Lin, H., & Sheldon, E. S. (2008) · Monthly Notices of the Royal Astronomical Society · doi:10.1111/j.1365-2966.2008.13510.x · Cited by 201
A Large Sample of Spectroscopic Redshifts in the ACS-GOODS Region of the Hubble Deep Field North
Cowie, L. L., Barger, A. J., Hu, E. M., Capak, P., & Songaila, A. (2004) · The Astronomical Journal · doi:10.1086/420997 · Cited by 184
Ho, S., Cuesta, A. J., Seo, H., de Putter, R., Ross, A. J., White, M. J., Padmanabhan, N., Saito, S., Schlegel, D. J., Schlafly, E. F., et al. (2012) · The Astrophysical Journal · doi:10.1088/0004-637x/761/1/14 · Cited by 177
Optical multicolors - A poor person's Z machine for galaxies
Koo, D. C. (1985) · The Astronomical Journal · doi:10.1086/113748 · Cited by 174
RANDOM FORESTS FOR PHOTOMETRIC REDSHIFTS
Carliles, S., Budavári, T., Heinis, S., Priebe, C., & Szalay, A. S. (2010) · The Astrophysical Journal · doi:10.1088/0004-637x/712/1/511 · Cited by 170
THE CANADA-FRANCE-HAWAII TELESCOPE LEGACY SURVEY: STACKED IMAGES AND CATALOGS
Gwyn, S. D. J. (2012) · The Astronomical Journal · doi:10.1088/0004-6256/143/2/38 · Cited by 162
A Galaxy Photometric Redshift Catalog for the Sloan Digital Sky Survey Data Release 6
Oyaizu, H., Lima, M., Cunha, C. E., Lin, H., Frieman, J., & Sheldon, E. S. (2008) · The Astrophysical Journal · doi:10.1086/523666 · Cited by 161
Precision photometric redshift calibration for galaxy–galaxy weak lensing
Mandelbaum, R., Seljak, U., Hirata, C. M., Bardelli, S., Bolzonella, M., Bongiorno, A., Carollo, C. M., Contini, T., Cunha, C. E., Garilli, B., et al. (2008) · Monthly Notices of the Royal Astronomical Society · doi:10.1111/j.1365-2966.2008.12947.x · Cited by 148
ArborZ: PHOTOMETRIC REDSHIFTS USING BOOSTED DECISION TREES
Gerdes, D. W., Sypniewski, A. J., McKay, T. A., Hao, J., Weis, M. R., Wechsler, R. H., & Busha, M. T. (2010) · The Astrophysical Journal · doi:10.1088/0004-637x/715/2/823 · Cited by 134
Estimating Photometric Redshifts Using Support Vector Machines
Wadadekar, Y. G. (2005) · Publications of the Astronomical Society of the Pacific · doi:10.1086/427710 · Cited by 119
Photometric redshifts of galaxies
Loh, E. D., & Spillar, E. J. (1986) · The Astrophysical Journal · doi:10.1086/164062 · Cited by 115
A comparison of six photometric redshift methods applied to 1.5 million luminous red galaxies
Abdalla, F. B., Banerji, M., Lahav, O., & Rashkov, V. (2011) · Monthly Notices of the Royal Astronomical Society · doi:10.1111/j.1365-2966.2011.19375.x · Cited by 110
Jee, M. J., Tyson, J., Schneider, M. D., Wittman, D., Schmidt, S., & Hilbert, S. (2012) · The Astrophysical Journal · doi:10.1088/0004-637x/765/1/74 · Cited by 99
Geach, J. E. (2011) · Monthly Notices of the Royal Astronomical Society · doi:10.1111/j.1365-2966.2011.19913.x · Cited by 97
Ball, N. M., Brunner, R. J., Myers, A. D., Strand, N. E., Alberts, S. L., & Tcheng, D. (2008) · The Astrophysical Journal · doi:10.1086/589646 · Cited by 94
Self-citationA Near‐Infrared Analysis of the Submillimeter Background and the Cosmic Star‐Formation History
Wang, W.-H., Cowie, L. L., & Barger, A. J. (2006) · The Astrophysical Journal · doi:10.1086/505292 · Cited by 93
Photometric Redshift Error Estimators
Oyaizu, H., Lima, M., Cunha, C. E., Lin, H., & Frieman, J. A. (2008) · The Astrophysical Journal · doi:10.1086/592591 · Cited by 62
Ball, N. M., Brunner, R. J., Myers, A. D., Strand, N. E., Alberts, S. L., Tcheng, D., & Llorà, X. (2007) · The Astrophysical Journal · doi:10.1086/518362 · Cited by 58
Self-citationToward More Precise Photometric Redshifts: Calibration via CCD Photometry
Brunner, R., Connolly, A. J., Szalay, A. S., & Bershady, M. A. (1997) · The Astrophysical Journal · doi:10.1086/310674 · Cited by 56
Self-citationMyers, A. D., White, M. J., & Ball, N. M. (2009) · Monthly Notices of the Royal Astronomical Society · doi:10.1111/j.1365-2966.2009.15432.x · Cited by 56
Way, M. J., Foster, L. V., Gazis, P. R., & Srivastava, A. N. (2009) · The Astrophysical Journal · doi:10.1088/0004-637x/706/1/623 · Cited by 50
Astroinformatics of galaxies and quasars: a new general method for photometric redshifts estimationPreprint
Laurino, O. (2012) · doi:10.6084/m9.figshare.105153.v1 · Cited by 50
Sample variance in photometric redshift calibration: cosmological biases and survey requirements
Cunha, C. E., Dragan Huterer, A., Busha, M. T., & Wechsler, R. H. (2012) · Monthly Notices of the Royal Astronomical Society · doi:10.1111/j.1365-2966.2012.20927.x · Cited by 48
Astroinformatics of galaxies and quasars: a new general method for photometric redshifts estimation
Laurino, O., D’Abrusco, R., Longo, G., & Riccio, G. (2011) · Monthly Notices of the Royal Astronomical Society · doi:10.1111/j.1365-2966.2011.19416.x · Cited by 48
Photometric redshifts with the quasi Newton algorithm (MLPQNA) Results in the PHAT1 contest
Cavuoti, S., Brescia, M., Longo, G., & Mercurio, A. (2012) · Astronomy and Astrophysics · doi:10.1051/0004-6361/201219755 · Cited by 45
Myers, A. D., White, M. J., & Ball, N. M. (2009) · Cited by 45
Photometric redshift estimation using spectral connectivity analysis
Freeman, P. E., Newman, J. A., Lee, A. B., Richards, J. W., & Schafer, C. M. (2009) · Monthly Notices of the Royal Astronomical Society · doi:10.1111/j.1365-2966.2009.15236.x · Cited by 41
Photometric redshifts with the quasi Newton algorithm (MLPQNA) Results in the PHAT1 contest
Cavuoti, S., Brescia, M., Longo, G., & Mercurio, A. (2012) · Springer Link (Chiba Institute of Technology) · doi:10.1051/0004-6361/201219755/pdf · Cited by 40
Can Self-Organizing Maps Accurately Predict Photometric Redshifts?
Way, M. J., & Klose, C. D. (2012) · Publications of the Astronomical Society of the Pacific · doi:10.1086/664796 · Cited by 39
Photometric Redshift Estimation Using Spectral Connectivity Analysis
Freeman, P. E., Newman, J. A., Lee, A. B., Richards, J. W., & Schafer, C. (2009) · Cited by 38
Can Selforganizing Maps Accurately Predict Photometric Redshifts?
Klose, C., & Way, M. J. (2012) · NASA STI Repository (National Aeronautics and Space Administration) · Cited by 31
A reliable cluster detection technique using photometric redshifts: introducing the 2TecX algorithm
van Breukelen, C., & Clewley, L. (2009) · Monthly Notices of the Royal Astronomical Society · doi:10.1111/j.1365-2966.2009.14692.x · Cited by 30
Two Novel Approaches for Photometric Redshift Estimation based on SDSS and 2MASS
Wang, D., Zhang, Y., Liu, C., & Zhao, Y. (2008) · 天文和天体物理学研究:英文版 · doi:10.1088/1009-9271/8/1/13 · Cited by 19
EFFECT OF INCLINATION OF GALAXIES ON PHOTOMETRIC REDSHIFT
Yip, C.-W., Szalay, A. S., Carliles, S., & Budavári, T. (2011) · The Astrophysical Journal · doi:10.1088/0004-637x/730/1/54 · Cited by 15
Effect of Inclination of Galaxies on Photometric RedshiftPreprint
Yip, C.-W., Szalay, A. S., & Carliles, S. (2016) · Cited by 9
Cited by
Machine learning and the physical sciences
Carleo, G., Cirac, I., Cranmer, K., Daudet, L., Schuld, M., Tishby, N., Vogt-Maranto, L., & Zdeborová, L. (2019) · Reviews of Modern Physics · doi:10.1103/revmodphys.91.045002 · Cited by 2,417
The Dark Energy Survey: more than dark energy – an overview
(2016) · Monthly Notices of the Royal Astronomical Society · doi:10.1093/mnras/stw641 · Cited by 1,019
The Sloan Digital Sky Survey Quasar Catalog: Sixteenth Data Release
Lyke, B. W., Higley, A. N., McLane, J. N., Schurhammer, D. P., Myers, A. D., Ross, A. J., Dawson, K., Chabanier, S., Martini, P., Busca, N. G., et al. (2020) · The Astrophysical Journal Supplement Series · doi:10.3847/1538-4365/aba623 · Cited by 537
Photometric redshifts for Hyper Suprime-Cam Subaru Strategic Program Data Release 1
Tanaka, M., Coupon, J., Hsieh, B.-C., Mineo, S., Nishizawa, A. J., Speagle, J., Furusawa, H., Miyazaki, S., & Murayama, H. (2018) · Publications of the Astronomical Society of Japan · doi:10.1093/pasj/psx077 · Cited by 332
Di Valentino, E., Said, J. L., Riess, A., Pollo, A., Poulin, V., Gómez-Valent, A., Weltman, A., Palmese, A., Huang, C. D., Bruck, C. V. D., et al. (2025) · Physics of the Dark Universe · doi:10.1016/j.dark.2025.101965 · Cited by 319
Metadata
- Authors
- Carrasco Kind, M., & Brunner, R. J.
- Venue
- Monthly Notices of the Royal Astronomical Society
- Publisher
- Oxford University Press (OUP)
- Published
- 2013-05-01
- Volume
- 432
- Issue
- 2
- Pages
- 1483-1501
- ISSN
- 0035-8711 (ISSN-L)
- 1365-2966
- OpenAlex
- W2015990127
- Cited by
- 271
- References
- 62
- FWCI
- 11.49
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.