A Test for Superior Predictive Ability
Open accessAbstract
We propose a new test for superior predictive ability. The new test compares favorably to the reality check (RC) for data snooping, because it is more powerful and less sensitive to poor and irrelevant alternatives. The improvements are achieved by two modifications of the RC. We use a studentized test statistic that reduces the influence of erratic forecasts and invoke a sample-dependent null distribution. The advantages of the new test are confirmed by Monte Carlo experiments and an empirical exercise in which we compare a large number of regression-based forecasts of annual U.S. inflation to a simple random-walk forecast. The random-walk forecast is found to be inferior to regression-based forecasts and, interestingly, the best sample performance is achieved by models that have a Phillips curve structure.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The bootstrap reality check is sensitive to poor and irrelevant alternatives, and including irrelevant alternatives can erode its power to reject the false null hypothesis.
“Hansen (2005) suggests that the reality check is sensitive to poor and irrelevant alternatives.Including irrelevant alternatives can erode the power of reality check to reject the false null hypothesis.”
From Profitability and Data-Snooping Tests of Four Technical Trade Strategies for Cryptocurrency Pair BTC/USDT and ETH/USDT in Cryptocurrency Markets During 2022–2023 (2023) · cites this paper as Hansen (2005)
The study adopts a stepwise test based on White's reality check and methodologies from Hansen (2005), Romano and Wolf (2005), and Hsu, Hsu, and Kuan (2010).
“We thus adopt a stepwise test that is based on a series of methodologies based on White's reality check test, including Hansen (2005); Romano and Wolf (2005); Hsu, Hsu, and Kuan (2010).”
From Profitability and Data-Snooping Tests of Four Technical Trade Strategies for Cryptocurrency Pair BTC/USDT and ETH/USDT in Cryptocurrency Markets During 2022–2023 (2023) · cites this paper as Hansen (2005)
The study uses a stepwise test to control the family-wise error rate and mitigate data snooping bias.
“To address and mitigate the issue of data snooping bias, a stepwise test is specified with a certain Type I error level within a defined period (𝑡 = 1, ⋯ , 𝑇).The main objective of this test is to effectively control the family-wise error, which represents the probability of rejecting at least one correct null hypothesis.”
From Profitability and Data-Snooping Tests of Four Technical Trade Strategies for Cryptocurrency Pair BTC/USDT and ETH/USDT in Cryptocurrency Markets During 2022–2023 (2023) · cites this paper as Hansen (2005)
The problem of sensitivity to irrelevant alternatives can be alleviated by studentizing the test statistic and incorporating an additional sample-dependent null distribution to identify relevant alternatives.
“This problem can be alleviated by studentizing the test statistic and by incorporating an additional sample-dependent null distribution to identify the relevant alternatives.”
From Profitability and Data-Snooping Tests of Four Technical Trade Strategies for Cryptocurrency Pair BTC/USDT and ETH/USDT in Cryptocurrency Markets During 2022–2023 (2023) · cites this paper as Hansen (2005)
Cites
Politis, D. N., & Romano, J. P. (1994) · Journal of the American Statistical Association · doi:10.1080/01621459.1994.10476870 · Cited by 2,649 · see citing works
Data-snooping, technical trading rule performance and the Bootstrap
Sullivan, R., Timmermann, A., & White, H. (1999) · The Journal of Finance · doi:10.1111/0022-1082.00163 · Cited by 1,028 · cited 1× · see citing works
“This was the benchmark used by Sullivan et al. (1999, 2001) who evaluated the significance of technical trading rules and calendar effects in stock returns.”
Method
Flagged first, then by citations
Cited by
Sorted by citations
Forecasting the equity risk premium: the role of technical indicators
Neely, C. J., Rapach, D. E., Tu, J., & Zhou, G. (2014) · Management Science · doi:10.1287/mnsc.2013.1838 · Cited by 1,003 · see citing works
Technical trading revisited: False discoveries, persistence tests, and transaction costs
Bajgrowicz, P., & Scaillet, O. (2012) · Journal of Financial Economics · doi:10.1016/j.jfineco.2012.06.001 · Cited by 253 · see citing works
A novel multiscale nonlinear ensemble leaning paradigm for carbon price forecasting
Zhu, B., Ye, S., Wang, P., He, K., Zhang, T., & Wei, Y.-M. (2018) · Energy Economics · doi:10.1016/j.eneco.2017.12.030 · Cited by 236 · see citing works
Performance of technical analysis in growth and small cap segments of the US equity market
Shynkevich, A. (2012) · Journal of Banking & Finance · doi:10.1016/j.jbankfin.2011.07.001 · Cited by 108 · see citing works
Technical trading: is it still beating the foreign exchange market?
Hsu, P.-H., Taylor, M. P., & Wang, Z. (2016) · Journal of International Economics · doi:10.1016/j.jinteco.2016.03.012 · Cited by 113 · see citing works
FX technical trading rules can be profitable sometimes!
Zarrabi, N., Snaith, S., & Coakley, J. (2017) · International Review of Financial Analysis · doi:10.1016/j.irfa.2016.12.010 · Cited by 32 · see citing works
Performance of technical trading rules: evidence from the crude oil market
Psaradellis, I., Laws, J., Pantelous, A. A., & Sermpinis, G. (2019) · The European Journal of Finance · doi:10.1080/1351847x.2018.1552172 · Cited by 27 · see citing works
Li, W., Wang, X., Wang, Z., & Yang, M. (2023) · Journal of Accounting and Finance · doi:10.33423/jaf.v23i4.6453 · Cited by 2 · see citing works
- Authors
- Hansen, P. R.
- Affiliations
- Stanford University (US)
- Corresponding author
- Hansen, Peter Reinhard
- Venue
- Journal of Business & Economic Statistics
- Publisher
- Informa UK Limited
- Published
- 2005-10
- Volume
- 23
- Issue
- 4
- Pages
- 365-380
- ISSN
- 0735-0015 (ISSN-L)
- 1537-2707
- OpenAlex
- W2050902901
- Cited by
- 1,230
- References
- 2
- Citations by year
- 2026: 92 · 2025: 55 · 2024: 44 · 2023: 46 · 2022: 55 · 2021: 55 · 2020: 76 · 2019: 64 · 2018: 69 · 2017: 75 · 2016: 68 · 2015: 74 · 2014: 54 · 2013: 80 · 2012: 69
- FWCI
- 38.50
- Type
- journal article
- Language
- en
- Open access
- green
- doi.org
- Locations
- Journal of Business and Economic Statistics (publishedVersion)
- Carolina Digital Repository (University of North Carolina at Chapel Hill) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- UNC Libraries
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Forecasting Techniques and Applications
- Social Sciences › Economics, Econometrics and Finance › General Economics, Econometrics and Finance › Monetary Policy and Economic Impact
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Diverse Scientific and Economic Studies
- Keywords
- superior predictive ability, reality check, data snooping, forecast evaluation, inflation forecasting, random walk forecast, Phillips curve
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
