What Do Editors Maximize? Evidence from Four Economics Journals
The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Abstract
We study editorial decisions using anonymized submissions matched to citations at four leading economics journals. We develop a benchmark model in which editors maximize the expected quality of accepted papers and citations are unbiased measures of quality. We then generalize the model to allow different quality thresholds, systematic gaps between citations and quality, and a direct impact of publication on citations. We find that referee recommendations are strong predictors of citations and that editors follow these recommendations closely. We document two deviations from the benchmark model. First, papers by highly published authors receive more citations, conditional on the referees' recommendations and publication status. Second, recommendations of highly published referees are equally predictive of future citations, yet editors give their views significantly more weight.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Double-anonymizing the peer-review process could also help level the playing field for academics from other marginalized groups, giving them a fairer chance to succeed, which, in turn, would promote diverse points of view in journal output and eventually expand the journal peer-reviewer pool to authors coming from such backgrounds (3–5, 19–23).”
From Nobel and novice: Author prominence affects peer review (2022) · cites this paper as [23]
In the citing paper
“Double-anonymizing the peer-review process could also help level the playing field for academics from other marginalized groups, giving them a fairer chance to succeed, which, in turn, would promote diverse points of view in journal output and eventually expand the journal peer-reviewer pool to authors coming from such backgrounds (3–5, 19–23).”
From Nobel and novice: Author prominence affects peer review (2022) · cites this paper as [23]
In the citing paper
“Double-anonymizing the peer-review process could also help level the playing field for academics from other marginalized groups, giving them a fairer chance to succeed, which, in turn, would promote diverse points of view in journal output and eventually expand the journal peer-reviewer pool to authors coming from such backgrounds (3–5, 19–23).”
From Nobel and novice: Author prominence affects peer review (2022) · cites this paper as [23]
Citations
Cites
Filders, R. (1992) · International Journal of Forecasting · doi:10.1016/0169-2070(92)90130-2 · Cited by 16,165
Conditional logit analysis of qualitative choice behavior
McFadden, D. L. (1972) · eScholarship (California Digital Library) · Cited by 14,203
Cragg, J. G., & Zarembka, P. (1975) · The Canadian Journal of Economics · doi:10.2307/134075 · Cited by 8,342
The Matthew Effect in Science: The reward and communication systems of science are considered.
Merton, R. K. (1968) · Science · doi:10.1126/science.159.3810.56 · Cited by 6,720 · cited 1×
“Second, editors may set higher or lower thresholds for certain groups of authors.2 Third, even in the absence of editorial preferences, citations may be biased by differences in citing practices across fields or by a tendency to cite wellknown authors (Merton, 1968).”
(1976) · Annals of Economic and Social Measurement · Cited by 5,035
(1968) · PubMed · Cited by 4,803
Why the impact factor of journals should not be used for evaluating research
Seglen, P. O. (1997) · BMJ · doi:10.1136/bmj.314.7079.497 · Cited by 2,447 · cited 1×
“…in many recent studies), we develop a control function to separate the mechanical effect of R&R status on citations from the signal contained 1See, Seglen (1997) for a discussion of impact factors, and Hamermesh, Johnson, and Weisbrod (1982) and Hilmer, Ransom, and Hilmer (2015) for analysis of…”
MethodControl Function Methods in Applied Econometrics
Wooldridge, J. M. (2015) · Journal of Human Resources · doi:10.3368/jhr.50.2.420 · Cited by 1,633
Alternative methods for evaluating the impact of interventions
Heckman, J. J., & Robb, R. (1985) · Journal of Econometrics · doi:10.1016/0304-4076(85)90139-3 · Cited by 1,024
Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22784 · Cited by 1,000
Racial Bias in Motor Vehicle Searches: Theory and Evidence
Knowles, J., Persico, N., & Todd, P. (2001) · Journal of Political Economy · doi:10.1086/318603 · Cited by 608
Juvenile incarceration, human capital, and future crime: Evidence from randomly assigned judges
Aizer, A., & Doyle, J. J. (2015) · The Quarterly Journal of Economics · doi:10.1093/qje/qjv003 · Cited by 541
The Slowdown of the Economics Publishing Process
Ellison, G. (2002) · Journal of Political Economy · doi:10.1086/341868 · Cited by 510
Maestas, N., Mullen, K. J., & Strand, A. (2013) · American Economic Review · doi:10.1257/aer.103.5.1797 · Cited by 489
Blank, R. M. (2016) · American Economic Review · Cited by 421
How Are the Mighty Fallen: Rejected Classic Articles by Leading Economists
Gans, J. S., & Shepherd, G. B. (1994) · Journal of Economic Perspectives · doi:10.1257/jep.8.1.165 · Cited by 389
Nine Facts about Top Journals in Economics
Card, D., & DellaVigna, S. (2013) · Journal of Economic Literature · doi:10.1257/jel.51.1.144 · Cited by 333
Self-citationDahl, G. B., Kostøl, A. R., & Mogstad, M. (2014) · The Quarterly Journal of Economics · doi:10.1093/qje/qju019 · Cited by 323
Laband, D. N., & Piette, M. J. (1994) · Journal of Political Economy · doi:10.1086/261927 · Cited by 305
Are Referees and Editors in Economics Gender Neutral?*
Card, D., DellaVigna, S., Funk, P., & Iriberri, N. (2020) · The Quarterly Journal of Economics · doi:10.1093/qje/qjz035 · Cited by 279
Self-citationScholarship, Citations and Salaries: Economic Rewards in Economics
Hamermesh, D. S., Johnson, G. E., & Weisbrod, B. A. (1982) · Southern Economic Journal · doi:10.2307/1058497 · Cited by 261
Beyond LATE with a Discrete Instrument
Brinch, C. N., Mogstad, M., & Wiswall, M. (2017) · Journal of Political Economy · doi:10.1086/692712 · Cited by 243
Social Ties and Favoritism in Chinese Science
Fisman, R., Shi, J., Wang, Y., & Xu, R. (2018) · Journal of Political Economy · doi:10.1086/697086 · Cited by 230
Single-blind vs Double-blind Peer Review in the Setting of Author Prestige
Okike, K., Hug, K. T., Kocher, M. S., & Leopold, S. S. (2016) · JAMA · doi:10.1001/jama.2016.11014 · Cited by 192 · cited 1×
“Okike et al. (2016) provide some evidence from a medical journal of favoritism toward prolific authors, a finding different from ours.”
Predicting Experimental Results: Who Knows What?
DellaVigna, S., & Pope, D. (2018) · Journal of Political Economy · doi:10.1086/699976 · Cited by 183
Self-citationNetworks and productivity: Causal evidence from editor rotations
Brogaard, J., Engelberg, J., & Parsons, C. A. (2014) · Journal of Financial Economics · doi:10.1016/j.jfineco.2013.10.006 · Cited by 177
Expertise versus Bias in Evaluation: Evidence from the NIH
Li, D. (2017) · American Economic Journal: Applied Economics · doi:10.1257/app.20150421 · Cited by 169
Facts and Myths about Refereeing
Hamermesh, D. S. (1994) · Journal of Economic Perspectives · doi:10.1257/jep.8.1.153 · Cited by 147
Chetty, R., Saez, E., & Sándor, L. (2014) · Journal of Economic Perspectives · doi:10.1257/jep.28.3.169 · Cited by 116
Predicting Experimental Results: Who Knows What?
DellaVigna, S., & Pope, D. (2016) · National Bureau of Economic Research · doi:10.3386/w22566 · Cited by 116
Self-citationHow Does the Market Use Citation Data? The Hirsch Index in Economics
Ellison, G. (2013) · American Economic Journal Applied Economics · doi:10.1257/app.5.3.63 · Cited by 106
Editorial Favoritism in Economics?
Medoff, M. H. (2003) · Southern Economic Journal · doi:10.2307/3648979 · Cited by 90
Heinemann, M. (2016) · The Thoracic and Cardiovascular Surgeon · doi:10.1055/s-0036-1579579 · Cited by 83
Are Referees and Editors in Economics Gender Neutral?
Card, D., DellaVigna, S., Funk, P., & Iriberri, N. (2019) · National Bureau of Economic Research · doi:10.3386/w25967 · Cited by 79
Self-citationELLISON, G. (2011) · Economic Inquiry · doi:10.1111/j.1465-7295.2010.00261.x · Cited by 74
A Citation-Based Test for Discrimination at Economics and Finance Journals
Smart, S., & Waldfogel, J. (1996) · doi:10.3386/w5460 · Cited by 62
Hilmer, M. J., Ransom, M. R., & Hilmer, C. E. (2015) · Southern Economic Journal · doi:10.1002/soej.12037 · Cited by 55
Welch, I. (2014) · Review of Financial Studies · doi:10.1093/rfs/hhu029 · Cited by 43
The inside scoop: Acceptance and rejection at the journal of international economics
Cherkashin, I., Demidova, S., Imai, S., & Krishna, K. (2008) · Journal of International Economics · doi:10.1016/j.jinteco.2008.10.002 · Cited by 32
A model of the editorial process in academic journals
Bayar, O., & Chemmanur, T. J. (2021) · Research Policy · doi:10.1016/j.respol.2021.104339 · Cited by 20
How Do Editors Select Papers, and How Good are They at Doing It?
Hofmeister, R., & Krapf, M. (2011) · The B E Journal of Economic Analysis & Policy · doi:10.2202/1935-1682.3022 · Cited by 9
Preselection and expert advice
Schulte, E., & Felgenhauer, M. (2016) · International Journal of Game Theory · doi:10.1007/s00182-016-0551-9 · Cited by 5
A Model of the Editorial Process in Academic JournalsPreprint
Bayar, O., & Chemmanur, T. J. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.2022540 · Cited by 5
Desk rejection in an academic publication market model with matching frictionsPreprint
Radu Vranceanu, B., Besancenot, D., & Huynh, K. (2009) · RePEc: Research Papers in Economics · Cited by 4
Cited by
Publishing While Female: are Women Held to Higher Standards? Evidence from Peer Review
Hengel, E. (2022) · The Economic Journal · doi:10.1093/ej/ueac032 · Cited by 239
Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics
Brodeur, A., Cook, N., & Heyes, A. (2020) · American Economic Review · doi:10.1257/aer.20190687 · Cited by 333
Nonreplicable publications are cited more than replicable ones
Serra-Garcia, M., & Gneezy, U. (2021) · Science Advances · doi:10.1126/sciadv.abd1705 · Cited by 177
“It is used to study intellectual history and evaluate the quality of scientific work across a variety of disciplines (12, 13).”
BackgroundNobel and novice: Author prominence affects peer review
Huber, J., Inoua, S., Kerschbamer, R., König-Kersting, C., Palan, S., & Smith, V. L. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2205779119 · Cited by 148
How the publish-or-perish principle divides a science: the case of economists
van Dalen, H. P. (2021) · Scientometrics · doi:10.1007/s11192-020-03786-x · Cited by 128
Metadata
- Authors
- Card, D., & DellaVigna, S.
- Affiliations
- University of California, Berkeley (US)
- ORCID
- Card, David
- Venue
- The Review of Economics and Statistics
- Publisher
- MIT Press
- Published
- 2020-03
- Volume
- 102
- Issue
- 1
- Pages
- 195-217
- ISSN
- 0034-6535 (ISSN-L)
- 1530-9142
- OpenAlex
- W2948030142
- Cited by
- 123
- References
- 43
- Citations by year
- 2026: 14 · 2025: 13 · 2024: 19 · 2023: 18 · 2022: 20 · 2021: 17 · 2020: 11
- FWCI
- 63.14
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The Review of Economics and Statistics (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Keywords
- peer review, referee recommendations, citation impact, paper quality, economics journals, editorial decision-making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.