Building Honest Quant Validation From First Principles

Table of contents

1. Why validation, why now. The base rate of "validated" retail strategies surviving 6 months in production, and what makes it that low.
2. The five-family stack. Statistical robustness, look-ahead audits, execution realism, regime stratification, forward-paper — and why the order matters.
3. Family A — statistical robustness. Purged combinatorial CV with embargo, Deflated Sharpe, SPA, PBO, synthetic-null. Math + pseudocode + worked example.
4. Family B — look-ahead audits. The five contamination patterns (L1–L5), the time-shuffle test, the corpus-rebuild method.
5. Family C — execution realism. Orderbook-depth fill simulator, adverse-selection model, fee/slippage stress, capacity-curve construction.
6. Family D — regime stratification. Partition schemes, what to require, the cross-universe OOS transfer test.
7. Family E — forward-paper as the gold standard. Pre-registration, 14-day SPAN gate, champion-challenger A/B.
8. The corpus-contamination audit. How to rebuild a contaminated corpus from raw tape, fingerprint it, and ship the test suite.
9. A case study: 43 → 0. Walking the full Empire fleet through the battery. Where the funnel narrowed, and why.
10. Inverse RL as a leak-clean alternative to behavior cloning. Why predicting the winning side 80% of the time still loses money.
11. Appendix A. Pseudocode for every method.
12. Appendix B. The 24-class edge taxonomy.
13. Appendix C. Citations, links, and corpora.

Who this is for

• · Retail quants building strategies on prediction markets, options, or microstructure.
• · Research engineers at prop shops who want an explicit, auditable validation stack.
• · Educators / authors writing curricula around real (rather than aspirational) quant validation.
• · Risk teams evaluating an outside strategy submission.