CoChat
Sign Up
Cited by 2,188Financial Markets and Investment StrategiesFinancial Risk and Volatility ModelingFinancial Analysis and Corporate Governance

The DOI matches its Crossref record; 495 of the 495 references it cites were found in a registry or an index; 3 of the 26 sources its claims cite were checked for support.

… and the Cross-Section of Expected Returns

Harvey, Campbell R., Liu, Yan, Zhu, Heqing·2016·Review of Financial Studies
DOI 10.1093/rfs/hhv059
Open this paper in CoChat — free account

Abstract

Hundreds of papers and factors attempt to explain the cross-section of expected returns. Given this extensive data mining, it does not make sense to use the usual criteria for establishing significance. Which hurdle should be used for current research? Our paper introduces a new multiple testing framework and provides historical cutoffs from the first empirical tests in 1967 to today. A new factor needs to clear a much higher hurdle, with a t-statistic greater than 3.0. We argue that most claimed research findings in financial economics are likely false. (JEL C12, C52, G12)

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

The paper argues that conventional significance thresholds are too lenient for cross-sectional return factors because hundreds of factors have been tested, and it proposes multiple-testing adjustments that imply a much higher hurdle. It concludes that a newly proposed factor should generally have a t-statistic above 3.0, and that many published factors would not survive this standard.

Key findings

  • The authors catalogue 313 articles and 316 factors, including 250 published articles and 63 working papers covering 68 factors; they note that this likely understates the true number of tested factors.
  • Under multiple-testing adjustments, benchmark t-statistics rise well above 1.96: Bonferroni reaches 3.78 by 2012 and 4.00 by 2032; Holm is close to Bonferroni; BHY stabilizes around 3.39 after 2010 at 1% FDR.
  • When allowing for unobserved tests, they estimate about 71% of tried factors are missing; benchmark t-statistics become 4.01 for Bonferroni, 3.96 for Holm, 3.68 for BHY at 1%, and 3.18 for BHY at 5%, leading them to recommend a minimum threshold near 3.0.
  • Of 296 published significant factors, they classify 158 as false discoveries under Bonferroni, 142 under Holm, 132 under BHY at 1%, and 80 under BHY at 5%.

Methods

  • The authors assemble a hand-collected sample of factor-proposing papers from leading finance, economics, and accounting journals, plus selected SSRN working papers, and record reported t-statistics for 316 factors.
  • They classify factors into “common” risk factors and firm “characteristics,” then apply three multiple-testing procedures: Bonferroni and Holm for family-wise error rate control, and Benjamini-Hochberg-Yekutieli for false discovery rate control.
  • To address unpublished or hidden tests, they use a truncated exponential simulation framework to estimate the total number of trials and adjusted benchmarks, and they also estimate a model with correlation among factor returns to derive threshold t-statistics.

Limitations

  • The paper states that not all factor discoveries should be treated equally: theory-motivated factors may deserve a lower hurdle than purely empirical ones, though a t-statistic of 2.0 is still too low.
  • The paper states that its tests are unconditional, so a factor that appears marginal unconditionally could matter in some economic environments and not others.
  • The paper acknowledges that its factor list underrepresents the population because it focuses mainly on top journals, selects only some working papers, often includes only representative variants of similar characteristics, and cannot observe failed tests; it also takes published t-statistics as given and notes weak identification of the correlation parameter.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1093/rfs/hhv059

Cites

495 of 495 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

3 of 26 cited sources checked · 1 supported · 1 flagged · accuracy not yet measured

Show all 56 claims

  • Probably not a substantial claim. 77
  • It may state two separate claims as one. 64
117 of 117

  • The authors state this as an assumption.

    Published t-statistics are econometrically sound with respect to data errors, coding errors, misalignment, heteroscedasticity, autocorrelation, clustering, outliers, and similar issues.

    “Importantly, we take published t-statistics as given.That is, we assume they are econometrically sound with respect to the usual suspects (data errors, coding errors, misalignment, heteroscedasticity, autocorrelation, clustering, outliers, etc.).”

  • The authors state this as an assumption.

    The test results of all tried factors are available (M = R).

    “We now apply the three adjustment methods previously introduced to the observed factor tests, under the assumption that the test results of all tried factors are available.”

  • The authors state this as an assumption.

    For each factor, the corresponding long-short strategy return standard deviation is normalized to σ = 15% per year.

    “For each factor, suppose researchers construct a corresponding long-short trading strategy and normalize the return standard deviation to be σ = 15% per year, which is close to the annual volatility of the market index.”

  • The authors state this as an assumption.

    Strategy returns are jointly normal and have zero serial correlation.

    “Assuming joint normality and zero serial correlation for strategy returns, this t-statistic has a normal distribution”

  • The authors state this as an assumption.

    Each μ_i is an independent draw from a mixture distribution with a point mass at zero and an exponential distribution with mean parameter λ.

    “We assume that each μ i is an independent draw from the following mixture distribution:”

  • The authors state this as an assumption.

    The contemporaneous correlation between two strategies' returns is ρ, and noncontemporaneous correlations are zero.

    “As a starting point, we assume that the contemporaneous correlation between two strategies' returns is ρ.The noncontemporaneous correlations are assumed to be zero.”

  • The authors state this as an assumption.

    M factors are tried, but only factors that exceed a certain t-statistic threshold are published; the threshold is set at 1.96.

    “Finally, to incorporate the impact of hidden tests, we assume that M factors are tried, but only factors that exceed a certain t-statistic threshold are published.We set the threshold t-statistic at 1.96 and focus on the subsample of factors that have a t-statistic larger than 1.96.”

  • The authors state this as an assumption.

    The sample covers a fraction r of t-statistics between 1.96 and 2.57, and all t-statistics above 2.57 are covered.

    “To overcome this missing data problem, we assume that our sample covers a fraction r of t-statistics in between 1.96 and 2.57 and that all t-statistics above 2.57 are covered.”

  • The authors state this as an assumption.

    The estimation framework assumes a balanced panel with M factors and N periods of returns.

    “Our estimation framework assumes a balanced panel with M factors and N periods of returns.”

  • The paper relies on this without support.

    Many factors were tried by empiricists, failed, and never made it to publication or even a working paper.

    “there are surely many factors that were tried by empiricists, failed, and never made it to publication or even a working paper.”

  • The authors state this as an assumption.

    The t-statistics are independent.

    “Assuming independence, we model observed t-statistics as draws from an exponential distribution with mean parameter λ and a known cutoff point of 2.57.”
    “The underlying assumption for the model in Appendix A is the independence among t-statistics, which may not be plausible given our previous discussions on test dependence.”

  • The authors state this as an assumption.

    Observed t-statistics follow an exponential distribution with mean parameter λ and a known cutoff point of 2.57.

    “Assuming independence, we model observed t-statistics as draws from an exponential distribution with mean parameter λ and a known cutoff point of 2.57.”

  • The authors state this as an assumption.

    For the baseline result, all tried factors with t-statistics above 2.57 are observed and those with t-statistics in the range (1.96, 2.57) are ignored.

    “for our baseline result, we assume all tried factors with t-statistics above 2.57 are observed and ignore those with t-statistics in the range of (1.96, 2.57).”

  • The authors state this as an assumption.

    The density curve of the t-statistic distribution is monotonic, so it is easier to find factors with small t-statistics than with large ones.

    “One restriction that we think any of these distributions should satisfy is the monotonicity of the density curve.Intuitively, it should be easier to find factors with small t-statistics than with large ones.”

  • The authors state this as an assumption.

    For the simulation example, the t-statistics for the 700 hypotheses are independent; a false factor's t-statistic follows N(0,1), and a true factor's t-statistic follows N(2,1).

    “we assume that the t-statistics for the 700 hypotheses are independent.Moreover, we assume the t-statistic for a false factor follows a normal distribution with mean of zero and variance of 1.0, that is, N (0,1); for a true factor, we assume its t-statistic follows a normal distribution with mean of 2.0 and variance of 1.0, that is, N (2,1).”

  • The authors state this as an assumption.

    Neglected significant t-statistics follow the empirical distribution of the t-statistic sample.

    “under the assumption that neglected significant t-statistics follow the empirical distribution of our t-statistic sample, by varying r, we can assess how this underrepresentation of our t-statistic sample affects results.”

  • The authors state this as an assumption.

    Our sample covers only half of all factors with t-statistics between 1.96 and 2.57.

    “with the ad hoc assumption that our sample covers only half of all factors with t-statistics between 1.96 and 2.57.”

  • The authors state this as an assumption.

    Factor returns are independent conditional on mean γ_i μ_i and common variance σ^2.

    “factor returns are independent conditional on mean γ i μ i and common variance σ 2”

  • The authors state this as an assumption.

    The first-step population parameters μ_i's are independently generated from a normal distribution.

    “μ i 's are independently generated from a normal distribution”

  • The authors state this as an assumption.

    The first-step population parameters γ_i's are generated from a Bernoulli distribution, with γ_i = 0 with probability p_0.

    “γ i 's are simply generated from a Bernoulli distribution, that is, γ i = 0 with probability p 0”

  • The paper relies on this without support.

    The reported X_i's randomly represent either long/short or short/long strategy returns.

    “The normality assumption for the μ i 's requires the reported X i 's to randomly represent either long/short or short/long strategy returns.”
21 of 21

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • BHY adjusted-p-value formula conflicts with the example

    Minor

    The formula sets the adjusted p-value for the largest ordered p-value to that original p-value. In Table 4, that is 4.66%, below the stated 5% significance level; the recurrence would then make every adjusted p-value no greater than 4.66%, implying ten discoveries rather than the six reported.

  • The parameter r is defined as both observed and missing share

    Minor

    The model description defines r as the fraction of intermediate t-statistics covered by the sample, whereas the Table 5 note defines it as the fraction missing. These opposite definitions change how the panel B assumption should be interpreted.

2 of 2

Citations

Cites

Items 201–400 of 495

Flagged first, then by citations

Research works printed but not linked: 5

  1. Douglas, 1967, Risk in the equity markets: An empirical appraisal of market efficiency, Yale Economic Essays, 9, 3
  2. Greene, 2008, Econometric analysis
  3. Scheffé, 1959, The analysis of variance
  4. Tukey, 1951, Reminder sheets for “Discussion of paper on multiple comparisons by Henry Scheffe.”, 469
  5. Tukey, 1953, The problem of multiple comparisons. Unpublished manuscript, 1

Cited by

Sorted by citations

See all 2,188

Related

See all

Metadata

Authors
Harvey, C. R., Liu, Y., & Zhu, H.
Affiliations
  • National Bureau of Economic Research (US)
  • Texas A&M University (US)
ORCID
Venue
Review of Financial Studies
Publisher
Oxford University Press (OUP)
Published
2015-10-09
Volume
29
Issue
1
Pages
5-68
ISSN
  • 0893-9454 (ISSN-L)
  • 1465-7368
DOI
10.1093/rfs/hhv059
OpenAlex
W4211170237
Cited by
2,119
References
446
Citations by year
2026: 293 · 2025: 204 · 2024: 199 · 2023: 202 · 2022: 192 · 2021: 213 · 2020: 218 · 2019: 183 · 2018: 172 · 2017: 138 · 2016: 64 · 2015: 25
FWCI
254.19
Type
journal article
Language
en
Open access
closed
Locations
Review of Financial Studies (publishedVersion)
Topics
  • Social Sciences › Economics, Econometrics and Finance › Finance › Financial Markets and Investment Strategies
  • Social Sciences › Economics, Econometrics and Finance › Finance › Financial Risk and Volatility Modeling
  • Social Sciences › Business, Management and Accounting › Accounting › Financial Analysis and Corporate Governance
Keywords
cross-section of expected returns, multiple comparisons, false positives, significance threshold, data mining

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review