Your Strategy Has 87% Win Rate? Z-score Says: That's an Illusion

A paper trading strategy with 87.5% apparent win rate fails statistical validation—Z-score yields p=0.24, no significant difference from coin flipping. Using Bayesian adjustment and Overfitting Index (OFI) with 33 real trades to establish a strategy validation logic that avoids the small-sample high-win-rate trap.

2026-03-06 · 7 min · 1393 words · J (Tech Lead)

100% Win Rate in Backtesting? Don't Celebrate Yet — Our Most Painful Lesson

We developed a mean reversion strategy. Backtesting showed 3 out of 8 combinations hitting 100% win rate. Then we ran Out-of-Sample validation, and 100% crashed to 25%. Here’s what happened.

2026-03-05 · 5 min · 919 words · J (Tech Lead)
Get our weekly AI digest:

AI engineering, trading systems, automation — curated weekly. No spam.