Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index; none of the 6 sources its claims cite is checked for support yet.
Abstract
Another thing I must point out is that you cannot prove a vague theory wrong.[…] Also, if the process of computing the consequences is indefinite, then with a little skill any experimental result can be made to look like the expected consequences.
Overview
CoChat Summary
TL;DR
The paper argues that backtest overfitting makes it easy to manufacture high in-sample Sharpe ratios, while out-of-sample performance can be zero or negative, especially when performance series have memory. It derives a Minimum Backtest Length and uses simulations plus a seasonal-strategy example to show that high in-sample Sharpe ratios can arise from skill-less random data.
Key findings
- With only \(N=10\) independent strategy configurations, the expected maximum in-sample Sharpe ratio is 1.57 even though all strategies have expected out-of-sample Sharpe ratio of zero; with \(N=128\) trials, the expected maximum Sharpe ratio is above 2.6.
- If only five years of data are available, no more than 45 independent model configurations should be tried to keep the expected maximum in-sample Sharpe ratio at 1 while expected out-of-sample Sharpe ratio is 0; after only seven independent configurations, the expected maximum in-sample Sharpe ratio is 1 for a two-year backtest, with expected out-of-sample Sharpe ratio 0.
- Adding a global constraint or serial dependence creates compensation effects: recentering makes out-of-sample performance negative even for a trendless process, with the p-value for the IS-performance relation equal to 0; for an AR(1) process with \(\phi=0.995\), the half-life is about 138 observations and the p-value for the IS-performance relation is again 0.
- In a synthetic seasonal-strategy example using 1,000 daily prices and 8,800 parameter combinations, the optimal rule has an annualized Sharpe ratio of 1.27 and a PSR-Stat of 2.83, implying less than 1 percent probability that the true Sharpe ratio is below 0, despite no true seasonal effect.
Methods
- Theoretical derivation: assumes IID Normal excess returns and uses the Sharpe ratio, extreme-value/Gumbel approximations, and propositions to derive the expected maximum Sharpe ratio and Minimum Backtest Length (MinBTL).
- Monte Carlo simulations: generates Gaussian random walks of size \(T=1000\), splits them into 500 in-sample and 500 out-of-sample observations, and studies selection of the best in-sample Sharpe ratio; also runs recentered and first-order autoregressive (\(\phi=0.995\)) variants.
- Practical application: performs a grid search over Entry day (1–22), Holding period (1–20), Stop loss (0–10), and Side (\(\pm1\)), forming 8,800 parameter combinations on a 1,000-day random-walk price series, and reports the optimal Sharpe ratio and PSR-Stat.
Limitations
- The paper states that MinBTL is a necessary but not sufficient condition to avoid overfitting, and that Proposition 1 assumes independent trials, which is a conservative estimate; if trials are not independent, the number of independent trials could be derived using a dimension-reduction procedure such as Principal Component Analysis.
- The paper states that the hold-out method cannot assess the representativeness of a backtest because it does not account for the number of trials attempted, and it leaves a more precise measure of backtest overfitting to Bailey et al. [1].
- Evident from the text: the simulations and practical example rely on synthetic IID Normal/Gaussian random walks and specific parameter choices rather than real market data, so the numerical illustrations are model-based demonstrations rather than empirical estimates.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures








Verification
Claims the paper makes
0 Valid · 8 Not valid · 6 Not checked · 68 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Figure 6 shows that adding a single global constraint causes the OOS performance to be negative even though the underlying process was trendless.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This illustrates the fact that, in absence of compensation effects, overfitting the IS performance ( x -axis) has no bearing on the OOS performance ( y -axis), which remains around zero.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Proposition 1. Given a sample of IID random variables, xn Z , n 1 ; : : : ; N , where Z is the CDF of the Standard Normal distribution, the expected maximum of that sample, E max N E max φ xn γ , can be approximated for a large N as”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
- Rests on 3 assumptions not shown here.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Theorem 2. The Minimum Backtest Length (MinBTL, in years) needed to avoid selecting a strategy with an IS Sharpe ratio of E max N among N independent strategies with an expected OOS Sharpe ratio of zero is (6)”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 3 assumptions not shown here.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This has allowed us to determine the Minimum Backtest Length (MinBTL) needed to avoid selecting a strategy with a given IS Sharpe ratio among N trials with an expected OOS Sharpe ratio of zero.”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
45 more claims
Create a free account to see all 82 claims and every source verdict.
- It may state two separate claims as one. 21
- Probably not a substantial claim. 16
- The quote only partly states this claim. 1
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“For example, we could explore all nodes that span the set φ 1 ; : : : ; 22 γ for Entry day , the set φ 1 ; : : : ; 20 γ for Holding period , the set φ 0 ; : : : ; 10 γ for Stop loss , and φ 1 ; 1 γ for Side . The parameter combinations involved form a four-dimensional mesh of 8,800 elements.”
Reported 8800 · recomputed 9680
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Is Backtest Overfitting a Fraud? · p. 9
“Consider an investment manager who emails his stock market forecast for the next month to 2 n x prospective investors, where x and n are positive integers.”
§ Is Backtest Overfitting a Fraud? · p. 9
“The investment manager is hiding the fact that for every one of the x successful witnesses, he has tried 2 n unsuccessful ones”
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702
Embrechts, P., Klüppelberg, C., & Mikosch, T. (1997) · doi:10.1007/978-3-642-33483-2 · Cited by 4,402
Extreme Values, Regular Variation and Point Processes
Resnick, S. I. (1987) · Springer Series in Operations Research and Financial Engineering · doi:10.1007/978-0-387-75953-1 · Cited by 3,158
Hawkins, D. M. (2004) · Journal of Chemical Information and Computer Sciences · doi:10.1021/ci0342472 · Cited by 2,522
Feynman, R. (2017) · The MIT Press eBooks · doi:10.7551/mitpress/11068.001.0001 · Cited by 1,881
A Reality Check for Data Snooping
White, H. (2000) · Econometrica · doi:10.1111/1468-0262.00152 · Cited by 1,851
Rules for Ordering Uncertain Prospects
Hadar, J., & Russell, W. R. (1969) · American Economic Review · Cited by 1,720
Stepwise Multiple Testing as Formalized Data Snooping
Romano, J. P., & Wolf, M. (2005) · Econometrica · doi:10.1111/j.1468-0262.2005.00615.x · Cited by 1,019
Weiss, S. M., & Kulikowski, C. A. (1991) · Morgan Kaufmann Publishers Inc. eBooks · Cited by 898
The Statistics of Sharpe Ratios
Lo, A. W. (2002) · Financial Analysts Journal · doi:10.2469/faj.v58.n4.2453 · Cited by 843
Research works printed but not linked: 4
- [3][3] , Drawdown-based stop-outs and the triple penance rule, 2013, working paper. Available at http://ssrn.com/abstract=2201302 . -
- [9][9] C. Harvey and Y. Liu , Backtesting, working paper, SSRN, 2013, http://ssrn.com/abstract=2345489 . -
- [10][10] C. Harvey, Y. Liu , and H. Zhu , : : : and the crosssection of expected returns, working paper, SSRN, 2013, http://ssrn.com/abstract=2249314 . -
- [12][12] Y. Hirsch , Don't Sell Stocks on Monday , Penguin Books, 1st edition, 1987. -
Cited by
Building Diversified Portfolios that Outperform Out of Sample
López de Prado, M. (2016) · The Journal of Portfolio Management · doi:10.3905/jpm.2016.42.4.059 · Cited by 293
“See Bailey et al. [2014] for a broader discussion of in-sample vs.”
BackgroundSelf-citationImproving random forest predictions in small datasets from two-phase sampling designs
Han, S., Williamson, B. D., & Fong, Y. (2021) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-021-01688-3 · Cited by 184
The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting, and Non-Normality
Bailey, D. H., & de Prado, M. L. (2014) · The Journal of Portfolio Management · doi:10.3905/jpm.2014.40.5.094 · Cited by 351
Self-citationTen Applications of Financial Machine LearningPreprint
de Prado, M. L. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3365271 · Cited by 101
Self-citationHarvey, C. R., & Liu, Y. (2015) · The Journal of Portfolio Management · doi:10.3905/jpm.2015.42.1.013 · Cited by 114
Buczynski, W., Fabio Cuzzolin, A., & Sahakian, B. J. (2021) · International Journal of Data Science and Analytics · doi:10.1007/s41060-021-00245-5 · Cited by 66
“The concept of “backtest overfitting” was introduced to the investment realm by acclaimed mathematician David Bailey et al. in 2014 [31].”
BackgroundEmpirical asset pricing via machine learning: evidence from the European stock market
Drobetz, W., & Otto, T. (2021) · Journal of Asset Management · doi:10.1057/s41260-021-00237-x · Cited by 61
“If information from the out-of-sample period is used to fit the models, consciously or not (Schorfheide and Wolpin 2012), this might lead to overstated out-of-sample predictive performance (Bailey et al. 2014, 2017; Harvey and Liu 2014, 2015; Harvey et al. 2016).”
MethodA Hierarchy of Limitations in Machine Learning
Malik, M. M. (2020) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.2002.05193 · Cited by 51
The 10 Reasons Most Machine Learning Funds Fail
de Prado, M. L. (2018) · The Journal of Portfolio Management · doi:10.3905/jpm.2018.44.6.120 · Cited by 60
Self-citationDetection of False Investment Strategies Using Unsupervised Learning Methods
de Prado, M. L., & Lewis, M. (2018) · Quantitative Finance · doi:10.1080/14697688.2019.1622311 · Cited by 59
“Bailey and López de Prado (2012, 2014) and Bailey et al. (2014) introduced the False Strategy theorem (see Section 6.1), and demonstrated how a SR estimate can be used to reject false discoveries under non-Normal returns while controlling for SBuMT. Critically, this theorem required the estimation of two meta-research…”
InfluentialSelf-citationHsu, J. C., Kalesnik, V. M., & Kose, E. (2019) · Financial Analysts Journal · doi:10.1080/0015198x.2019.1567194 · Cited by 66
Deep Reinforcement Learning in Agent Based Financial Market Simulation
Maeda, I., deGraw, D., Kitano, M., Matsushima, H., Sakaji, H., Izumi, K., & Kato, A. (2020) · Journal of risk and financial management · doi:10.3390/jrfm13040071 · Cited by 56
“Financial practitioners have traditionally been limited to training models with past data and do not have many options for improving their predictive models (Bailey et al. 2014).”
A novel backtesting methodology for clustering in mean-variance portfolio optimization
Tayali, S. T. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106454 · Cited by 36
Singular Point Probability Improve LSTM Network Performance for Long-term Traffic Flow Prediction
Liu, B., Cheng, J., Cai, K., Shi, P., & Tang, X. (2017) · Communications in computer and information science · doi:10.1007/978-981-10-6893-5_24 · Cited by 36
The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting and Non-NormalityPreprint
Bailey, D. H., & de Prado, M. L. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.2460551 · Cited by 35
Self-citationWiecki, T. V., Campbell, A. T., Lent, J., & Stauth, J. (2016) · The Journal of Investing · doi:10.3905/joi.2016.25.3.069 · Cited by 43
The Evolution of Reinforcement Learning in Quantitative Finance: A Survey
Pippas, N., Ludvig, E. A., & Turkay, Ç. (2025) · ACM Computing Surveys · doi:10.1145/3733714 · Cited by 69
Pagliaro, A. (2025) · Electronics · doi:10.3390/electronics14091721 · Cited by 64
Will Your Factor Deliver? An Examination of Factor Robustness and Implementation Costs
Beck, N., Hsu, J. C., Kalesnik, V. M., & Kostka, H. (2016) · Financial Analysts Journal · doi:10.2469/faj.v72.n5.6 · Cited by 31
“Although several researchers (Bailey et al. 2014, 2015; McLean and Pontiff 2015) have failed to replicate many factor strategies in-sample, interest in such strategies fortunately had waned long before researchers reminded us about them.”
BackgroundMethodInfluentialBeyond Econometrics: A Roadmap Towards Financial Machine LearningPreprint
de Prado, M. L. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3365282 · Cited by 46
Self-citationA Data Science Solution to the Multiple-Testing Crisis in Financial Research
de Prado, M. L. (2019) · The Journal of Financial Data Science · doi:10.3905/jfds.2019.1.099 · Cited by 30
Self-citationCausal Factor Investing: Can Factor Investing Become Scientific?
de Prado, M. L. (2023) · Cambridge University Press eBooks · doi:10.1017/9781009397315 · Cited by 44
Self-citationBeing Honest in Backtest Reporting: A Template for Disclosing Multiple Tests
Fabozzi, F. J., & de Prado, M. L. (2018) · The Journal of Portfolio Management · doi:10.3905/jpm.2018.45.1.141 · Cited by 26
“Hedge fund strategies are characterized by returns distributions with negative skewness and positive excess kurtosis (see Brooks and Kat 2002; Ingersoll et al. 2007). Exhibit 2 repeats these calculations for the non-normal case where ( ˆ 3, ˆ 10 3 4 γ = − γ = ). These results indicate that ignoring the non-normality o…”
BackgroundInfluentialSelf-citationQuantifying Backtest Overfitting in Alternative Beta Strategies
Suhonen, A., Lennkh, M., & Perez, F. (2016) · The Journal of Portfolio Management · doi:10.3905/jpm.2017.43.2.090 · Cited by 29
“the JOurnal Of POrtfOliO ManageMent 97 Winter 2017 We next test a hypothesis following Bailey et al. [2014] that increasing the complexity of a strategy may increase the risk of backtest overfitting, because having a larger number of parameters makes it easier to fit an algorithm to specific historical data.”
BackgroundMethodThe probability of back-test over-fittingPreprint
Bailey, D. H., Borwein, J., de Prado, M. M. L., & Zhu, Q. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2326253 · Cited by 114
“max_=max([gauss(dist_base[0],dist_base[1]) for i in range(trials)])”
BackgroundMethodInfluentialSelf-citationAn application of deep reinforcement learning to algorithmic trading
Théate, T., & Ernst, D. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.114632 · Cited by 224
“Indeed, Bailey et al. (2014) claims that many scientific publications in finance suffer from a lack of a proper scientific approach, instead getting closer to pseudo-mathematics and financial charlatanism than rigorous sciences.”
BackgroundThe Mixed vs the Integrated Approach to Style Investing: Much Ado About Nothing?
Leippold, M., & Rueegg, R. (2017) · European Financial Management · doi:10.1111/eufm.12139 · Cited by 29
Estimating Stock Market Betas via Machine Learning
Drobetz, W., Hollstein, F., Otto, T., & Prokopczuk, M. (2024) · Journal of Financial and Quantitative Analysis · doi:10.1017/s0022109024000036 · Cited by 38
“…from the out-of-sample period is consciously or unconsciously used to fit the models (Schorfheide and Wolpin (2012)), this can lead to exaggerated out-of-sample predictive performance (Bailey, Borwein, de Prado, and Zhu (2014), (2017), Harvey and Liu (2014), (2015), and Harvey et al. (2016)).”
A Framework for Assessing Factors and Implementing Smart Beta Strategies
Hsu, J. C., Kalesnik, V. M., & Viswanathan, V. (2015) · The Journal of Index Investing · doi:10.3905/jii.2015.6.1.089 · Cited by 21
Improving risk-adjusted performance in high frequency trading using interval type-2 fuzzy logic
Vella, V., & Ng, W. L. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.01.056 · Cited by 26
“The size of the time series provides a sufficiently large historical dataset which reduces the possibility of over fitting or produce spurious results during back-testing (Bailey et al., 2014).”
BackgroundInfluentialThe Future of Empirical Finance
de Prado, M. L. (2015) · The Journal of Portfolio Management · doi:10.3905/jpm.2015.41.4.140 · Cited by 33
Self-citationEvaluating Trading StrategiesPreprint
Harvey, C. R., & Liu, Y. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2474755 · Cited by 21
Facilitating Reproducibility in Scientific Computing: Principles and Practice
Bailey, D. H., Borwein, J. M., & Stodden, V. (2016) · John Wiley & Sons, Inc. eBooks · doi:10.1002/9781118865064.ch9 · Cited by 30
“As two of us wrote in [13], “Our silence is consent, making us accomplices”
BackgroundInfluentialSelf-citationHeterogeneous tail generalized COMFORT modeling via Cholesky decomposition
Näf, J., Paolella, M. S., & Polak, P. (2019) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2019.02.004 · Cited by 17
Machine Learning for Asset Managers
de Prado, M. L. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108883658 · Cited by 191
Self-citationA Multi-Step Procedure for Asset Allocation in Case of Limited Resources
Georgieva, P., Popchev, I. P., & Stoyanov, S. N. (2015) · Cybernetics and Information Technologies · doi:10.1515/cait-2015-0040 · Cited by 18
INVITED EDITORIAL COMMENT: Mathematics and Economics: A Reality Check
de Prado, M. L. (2016) · The Journal of Portfolio Management · doi:10.3905/jpm.2016.43.1.005 · Cited by 16
Self-citationBacktestingPreprint
Harvey, C. R., & Liu, Y. (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2345489 · Cited by 21
Enhancing Markowitz's portfolio selection paradigm with machine learning
de Prado, M. L., Simonian, J., Fabozzi, F. A., & Fabozzi, F. J. (2024) · Annals of Operations Research · doi:10.1007/s10479-024-06257-1 · Cited by 29
“In the realm of ML, overfitting often arises from incorporating too many features, overly extensive training on the data, the intensive training of the model itself, and usage of the test set multiple times (see Bailey et al., 2014, for a discussion).”
Self-citationArtificial Intelligence for Financial Services
Kaswan, K. S., Dhatterwal, J. S., Kumar, N., & Lal, S. (2023) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-80455-562-020231006 · Cited by 18
Assaf, A., Demir, E., & Mokni, K. (2024) · Finance research letters · doi:10.1016/j.frl.2024.105260 · Cited by 15
Vella, V., & Ng, W. L. (2014) · Intelligent systems in accounting, finance and management/Intelligent systems in accounting, finance & management · doi:10.1002/isaf.1359 · Cited by 20
A framework for online investment decisions
Paskaramoorthy, A., Gebbie, T. J., & van Zyl, T. L. (2020) · Investment Analysts Journal · doi:10.1080/10293523.2020.1806460 · Cited by 24
“These tools are slowly gaining appreciation in the financial community, primarily due to efforts of Bailey and Lopez De Prado (see [28], [10]).”
BackgroundThe False Strategy Theorem: A Financial Application of Experimental Mathematics
de Prado, M. L., & Bailey, D. H. (2021) · American Mathematical Monthly · doi:10.1080/00029890.2021.1965068 · Cited by 13
Self-citationSaadatmand, M., Daim, T. U., Mena, C., Yalçın, H., Bolatan, G. İ. S., & Chatterjee, M. (2024) · IEEE Transactions on Engineering Management · doi:10.1109/tem.2024.3522313 · Cited by 17
A financial fraud detection indicator for investors: an IDeA
Bernard, P., de Freitas, N. E. M., & Maillet, B. B. (2019) · Annals of Operations Research · doi:10.1007/s10479-019-03360-6 · Cited by 14
LIU, Q., Khadidos, A. O., & Wan, P. (2021) · Fractals · doi:10.1142/s0218348x22400692 · Cited by 11
Unbiased Objective Estimation in Predictive Optimization.
Ito, S., Yabe, A., & Fujimaki, R. (2018) · International Conference on Machine Learning · Cited by 12
Arian, H. R., Mobarekeh, D. N., & Seco, L. (2024) · Knowledge-Based Systems · doi:10.1016/j.knosys.2024.112477 · Cited by 10
Deep reinforcement learning with positional context for intraday trading
Goluža, S., Kovačević, T., Bauman, T., & Kostanjčar, Z. (2024) · Evolving Systems · doi:10.1007/s12530-024-09593-6 · Cited by 9
“Additionally, to construct features for our state space, the agent requires a one-hour lookback window of intraday data, hence the agent does not trade in the first hour of the liquid trading hours.”
Sentiment and the Performance of Technical Indicators
Feng, S. H., Wang, N., & Zychowicz, E. J. (2017) · The Journal of Portfolio Management · doi:10.3905/jpm.2017.43.3.112 · Cited by 10
Koshiyama, A., & Firoozye, N. B. (2018) · The Journal of Financial Data Science · doi:10.3905/jfds.2019.1.013 · Cited by 16
Estimating Stock Market Betas via Machine LearningPreprint
Drobetz, W., Hollstein, F., Otto, T., & Prokopczuk, M. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3933048 · Cited by 9
Stock Trading Via Feedback Control: an Extremum Seeking Approach
Formentin, S., Previdi, F., Maroni, G., et al. (2018) · 2006 14th Mediterranean Conference on Control and Automation · doi:10.1109/med.2018.8442799 · Cited by 9
“Backtest overfitting [2] is avoided tuning the parameters of the methods on in-sample price data and testing them on out-of sample price data from the same stocks but coming from a different period of time, preventing ambiguity with respect to generalization properties.”
MethodVliet, B. V. (2019) · International Journal of Innovation Management · doi:10.1142/s1363919620500292 · Cited by 14
Better Investing Through Factors, Regimes and Sensitivity AnalysisPreprint
Homescu, C. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2557236 · Cited by 7
Mango, F., Murè, P., Cardi, M., Paccione, C., & Bittucci, L. (2023) · Corporate Ownership and Control · doi:10.22495/cocv20i3siart9 · Cited by 10
Predicting price trends combining kinetic energy and deep reinforcement learning
Ghotbi, M., & Zahedi, M. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122994 · Cited by 7
de Prado, M. L., & Fabozzi, F. J. (2017) · The Journal of Portfolio Management · doi:10.3905/jpm.2017.44.1.001 · Cited by 7
Self-citationFama–French Factor Timing: The Long-Only Integrated Approach
Leippold, M., & Rueegg, R. (2019) · European Financial Management · doi:10.1111/eufm.12285 · Cited by 12
COBra: Copula-Based Portfolio Optimization
Paolella, M. S., & Polak, P. (2017) · Studies in computational intelligence · doi:10.1007/978-3-319-70942-0_3 · Cited by 7
Determining Optimal Trading Rules Without BacktestingPreprint
Carr, P., & de Prado, M. L. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2658641 · Cited by 7
“Bailey et al. [2013] provide a rigorous definition of backtest overfitting, which can be applied to our study of trading rules as follows.”
BackgroundSelf-citationForking Paths in Empirical StudiesPreprint
Coqueret, G. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.3999379 · Cited by 12
Volatility Targeting: It’s Complicated!
Mylnikov, G. (2021) · The Journal of Portfolio Management · doi:10.3905/jpm.2021.1.258 · Cited by 6
International high-frequency arbitrage for cross-listed stocks
Poutré, C., Dionne, G., & Yergeau, G. (2023) · International Review of Financial Analysis · doi:10.1016/j.irfa.2023.102777 · Cited by 6
de Prado, M. L. (2017) · The Journal of Portfolio Management · doi:10.3905/jpm.2017.43.4.005 · Cited by 6
Self-citationCausal Factor Investing: Can Factor Investing Become Scientific?Preprint
de Prado, M. L. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4205613 · Cited by 17
Self-citationHierarchical risk parity: Efficient implementation and real world analysis
Deković, D., & Posedel, P. (2025) · Future Generation Computer Systems · doi:10.1016/j.future.2025.107744 · Cited by 5
Detection of Black Regions in the Forex Market By Analyzing High-Frequency Intraday Data
Weeraddana, N. R., Silva, T., & Jayathilake, P. W. D. C. (2018) · International Conference on Advances in ICT for Emerging Regions · doi:10.1109/icter.2018.8615500 · Cited by 5
Learning the dynamics of technical trading strategies
Murphy, N., & Gebbie, T. (2021) · Quantitative Finance · doi:10.1080/14697688.2020.1869292 · Cited by 9
“We can then implement the CSCV procedure on the matrix of profits and losses resulting from the trials to recover a PBO estimate.”
BackgroundInfluentialDynamic scaling of distributed data-flows under uncertainty
Jamieson, S. J. (2020) · ACM International Conference on Distributed and Event-Based Systems (DEBS) · doi:10.1145/3401025.3406444 · Cited by 5
“an acceptably low out-of-sample performance degradation [5].”
How “Backtest Overfitting” in Finance Leads to False Discoveries
Bailey, D. H., & de Prado, M. L. (2021) · Significance · doi:10.1111/1740-9713.01588 · Cited by 11
Self-citationOptimal Trend Following Rules in Two-State Regime-Switching Models
Zakamulin, V., & Giner, J. (2022) · Journal of Asset Management · doi:10.1057/s41260-024-00357-0 · Cited by 5
Bailey, D. H., & Borwein, J. M. (2015) · Springer eBooks · doi:10.1007/978-3-319-21473-3_3 · Cited by 5
Self-citationBeating the market? A mathematical puzzle for market efficiency
Baumann, M. H. (2021) · Decisions in Economics and Finance · doi:10.1007/s10203-021-00361-8 · Cited by 5
“To overcome the joint hypotheses problem, the data-dredging problem, and the overfitting problem (Bailey et al., 2014)—i.”
BackgroundHow Hard Is It to Pick the Right Model?
Aparicio, D., & de Prado, M. L. (2017) · Algorithmic Finance · doi:10.3233/af-180231 · Cited by 12
“A growing variety of methods address the multiple testing problem (White (2000), Benjamini & Hochberg (1995), Benjamini & Yekutieli (2001), Storey (2002), Romano & Wolf (2005), Romano et al. (2010); see also Bailey et al. (2014) and Harvey et al. (2016) for recent methodologies in finance).”
Self-citationStatistical Overfitting and Backtest PerformancePreprint
Bailey, D. H., Ger, S., de Prado, M. L., et al. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2507040 · Cited by 5
Self-citationWhat is the Optimal Significance Level for Investment Strategies?Preprint
de Prado, M. L., & Lewis, M. J. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3193697 · Cited by 5
Self-citationFinancial Time Series Modelling Using Fractal Interpolation Functions
Manousopoulos, P., Drakopoulos, V., & Polyzos, E. (2023) · AppliedMath · doi:10.3390/appliedmath3030027 · Cited by 4
“Such approaches can provide greater predicting accuracy but are often prone to overfitting, thus resulting in data-specific findings ([17]).”
BackgroundMicro Data analytics: a test for analytical procedures
Santosuosso, P. (2021) · Meditari Accountancy Research · doi:10.1108/medar-02-2020-0767 · Cited by 6
Hamori, S., & Takiguchi, T. (2020) · MDPI eBooks · doi:10.3390/books978-3-03936-225-7 · Cited by 7
Issues in Applying Financial Econometrics to Factor-Based Modeling in Investment Management
Engle, R. F., Focardi, S. M., & Fabozzi, F. J. (2016) · The Journal of Portfolio Management · doi:10.3905/jpm.2016.42.5.094 · Cited by 4
The Topology of Macro Financial Flows: An Application of Stochastic Flow DiagramsPreprint
Calkin, N. J., & de Prado, M. L. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2379319 · Cited by 4
Self-citationUnderstanding the Complexity of Financial Systems of SystemsPreprint
Freedman, R. S. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2512307 · Cited by 4
A Long Short-term Traffic Flow Prediction Method Optimized by Cluster ComputingPreprint
Liu, B., Cheng, J., Liu, Q., et al. (2018) · Preprints.org · doi:10.20944/preprints201808.0163.v1 · Cited by 4
Network Diversification for a Robust Portfolio AllocationPreprint
Jaeger, M., & Marinelli, D. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4068889 · Cited by 4
An early-warning risk signals framework to capture systematic risk in financial markets
Ciciretti, V., Nandy, M., Pallotta, A., Lodh, S., Senyo, P. K., & Kartašova, J. (2025) · Quantitative Finance · doi:10.1080/14697688.2025.2482637 · Cited by 8
Sokolovsky, A., & Arnaboldi, L. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.119836 · Cited by 5
Should You Use GARCH Models for Forecasting Volatility? A Comparison to GRU Neural Networks
Pallotta, A., & Ciciretti, V. (2023) · Studies in Nonlinear Dynamics and Econometrics · doi:10.1515/snde-2022-0025 · Cited by 6
Kernel-Based Aggregating Learning System for Online Portfolio Optimization
Wang, X., Sun, T., & Liu, Z. (2020) · Mathematical Problems in Engineering · doi:10.1155/2020/6595329 · Cited by 3
“Following the theorem in [35], the Minimum Backtest Length (MinBTL, in years) is needed to avoid selecting a strategy with a given in-sample SR among N trials with an expected out-of-sample SR of zero.”
MethodMironov, T., Motaylenko, L., Andreev, D., et al. (2021) · Environment Technology Resources Proceedings of the International Scientific and Practical Conference · doi:10.17770/etr2021vol2.6629 · Cited by 3
“…various studies indicate a high probability of overfitting when a strategy’s model is constructed and optimized based on the results of a backtest [2], the testing of trading algorithms on historical data is still the main tool for trading strategy development and can often be seen both in…”
BackgroundData Driven Dimensionality Reduction to Improve Modeling Performance✱
Chung, J., de Prado, M. L., Simon, H. D., & Wu, K. J. (2023) · International Conference on Scientific and Statistical Database Management · doi:10.1145/3603719.3603744 · Cited by 4
Self-citationModeling and Trading the VIX and VSTOXX with Futures, Options and the VXXPreprint
Donninger, C. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2609414 · Cited by 3
Stock Portfolio Design and Backtest OverfittingPreprint
Bailey, D. H., Borwein, J. M., & de Prado, M. L. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2739335 · Cited by 3
“For a detailed discussion of backtest overfitting, we refer the reader to [1].”
BackgroundSelf-citationSystematic Testing of Systematic Trading StrategiesPreprint
Perumal, K., & Flint, E. J. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3132229 · Cited by 3
A Flexible Regime Switching Model for Asset ReturnsPreprint
Paolella, M. S., Polak, P., & Walker, P. S. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3389305 · Cited by 3
Futuretesting Quantitative StrategiesPreprint
Bloch, D. A. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4647103 · Cited by 3
Why Has Factor Investing Failed?: The Role of Specification ErrorsPreprint
de Prado, M. L., & Zoonekynd, V. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4697929 · Cited by 11
Self-citationThe good, the bad, and latency: exploratory trading on Bybit and Binance
Albers, J., Cucuringu, M., Howison, S., et al. (2025) · Quantitative Finance · doi:10.1080/14697688.2025.2515933 · Cited by 2
“Several lines of work have examined common pitfalls of backtesting: Bailey et al. (2014, 2017) warn against the acceptance of strategies based solely on their in-sample performance, and propose a combinatorial approach to quantify and mitigate the risk of spurious in-sample performance leading to…”
Atwa, A. E. M., Kholief, M., & Sedky, A. (2024) · International Conference on Computer Theory and Applications (ICCTA) · doi:10.1109/iccta64612.2024.10974778 · Cited by 2
“This phenomenon, known as survivorship bias, has been widely discussed in the literature [3, 4, 8, 9, 10, 11].”
Metadata
- Authors
- Bailey, D. H., Borwein, J. M., López de Prado, M., & Zhu, Q. J.
- Affiliations
- Lawrence Berkeley National Laboratory (US)
- Western Michigan University (US)
- Venue
- Notices of the American Mathematical Society
- Publisher
- American Mathematical Society (AMS)
- Published
- 2014-05-01
- Volume
- 61
- Issue
- 5
- Pages
- 458
- ISSN
- 0002-9920 (ISSN-L)
- 1088-9477
- OpenAlex
- W3125595766
- Cited by
- 221
- References
- 25
- Citations by year
- 2026: 43 · 2025: 18 · 2024: 12 · 2023: 16 · 2022: 6 · 2021: 23 · 2020: 15 · 2019: 16 · 2018: 16 · 2017: 11 · 2016: 15 · 2015: 20 · 2014: 9
- FWCI
- 33.55
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Notices of the American Mathematical Society (publishedVersion)
- NOVA (University of Newcastle Australia) (submittedVersion)
- Funders
- American Mathematical Society
- University of Oxford
- Eidgenössische Technische Hochschule Zürich
- University of California, Irvine
- Lawrence Berkeley National Laboratory
- Topics
- Social Sciences › Economics, Econometrics and Finance › Finance › Financial Markets and Investment Strategies
- Social Sciences › Decision Sciences › Management Science and Operations Research › Stock Market Forecasting Methods
- Social Sciences › Economics, Econometrics and Finance › Finance › Stochastic processes and financial applications
- Keywords
- backtest overfitting, out-of-sample performance, financial forecasting, model selection, time series forecasting
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.