THE RESEARCH
The event horizon: the tests a result must pass before it counts as edge
The event horizon is the line nothing escapes. Here it is the set of tests a result must pass before it counts as edge and not luck. Most strategies do not cross it, and that is the point.
The five tests
- A luck test, corrected: every strategy’s luck test is corrected for every other one tested at the same time (Holm, 1979).
- A higher bar for many tries: with hundreds of factors tried, a t-statistic near 3, not 2, is the honest bar (Harvey, Liu and Zhu, 2016).
- The deflated Sharpe ratio: a backtest’s Sharpe shrinks for the number of variants tried (Bailey and López de Prado, 2014).
- Held-out data and forward trades: the lab keeps a held-out half, then the rule must hold up on forward trades it never saw (Bailey, Borwein, López de Prado and Zhu, 2017).
- Edges fade once published: published anomalies kept roughly 42% of their return after publication, so survival is graded hard (McLean and Pontiff, 2016).
One more: predict, don’t describe
Some effects explain the bar they are measured on and vanish the moment you have to act on them. A rule from a paper only stays in the factory if its edge survives being entered a session late. For how ideas are graded before any of this, see where edge is likely.
FAQ
Why correct the luck test for other strategies?
Test thousands of strategies and some will look great by chance. Each strategy’s luck test is corrected for every other one tested at the same time (Holm, 1979), so a lucky one does not get through just because many were tried.
Why does a backtest need a held-out half?
A rule tuned on all of history will fit noise. Keeping half the data back, then requiring forward trades the rule never saw, is the main defence against backtest overfitting (Bailey, Borwein, López de Prado and Zhu, 2017).
What is a deflated Sharpe ratio?
A Sharpe ratio shrunk for how many variants were tried before the best one was picked (Bailey and López de Prado, 2014). The more you tried, the more it is shrunk.