Backtesting Methodology

1. Strategy Detection

Our strategy engine uses pure price and volume data to detect swing trading setups. No fundamental data (earnings, revenue) is used in the detection phase. Each of the 7 strategies in the engine has its own detector function that scans OHLCV bars for specific technical patterns: VCP consolidations, trend pullbacks, RSI mean reversion, bear flags, cup-and-handle formations, and more.

A strategy "fires" when its detector finds a valid pattern and the confluence score (a weighted combination of technical confirmations) exceeds the minimum threshold. Each signal gets a grade (A+ through C) based on the composite score.

2. Walk-Forward Testing

We use a walk-forward approach, not a static backtest. For each bar in the history:

1. The strategy engine sees only data up to and including that bar (no future data)
2. If a BUY signal fires, the entry price is the next day's open (not the signal bar's close)
3. Forward returns are measured 5 and 10 trading days after entry
4. Stop and target hits are checked bar-by-bar within the 10-day window

This prevents lookahead bias — the most common flaw in retail backtesting. The engine never sees tomorrow's data when making today's decision.

3. Regime Approximation

In production, our market regime detector uses a 5-state model incorporating ADX, SPY trend, market breadth, and VIX levels. In backtesting, we use a simplified 2-state approximation:

• TRENDING_UP: SPY closes above its 200-day simple moving average
• TRENDING_DOWN: SPY closes below its 200-day SMA

This is less granular than production but avoids introducing breadth/VIX data dependencies into the backtest that would be hard to source historically. The regime affects which strategies are eligible to fire (each strategy declares valid regimes).

4. Survivorship Bias

This is the biggest limitation. Backtests run on currently listed symbols only. Stocks that were delisted, went bankrupt, or were acquired during the backtest period are excluded. Since these tend to be losers, actual historical results would likely be worse than what the backtest shows.

We cannot eliminate this bias without a point-in-time universe dataset (which we don't have). When interpreting results, assume the real win rate is several percentage points lower than reported.

5. Entry Model

All backtest entries use next-day open after the signal fires. No slippage model is applied. In reality:

• You may not get filled at the exact open price
• Spreads, commissions, and market impact are not modeled
• Gap opens (especially after earnings) can cause entries far from the expected price

For liquid large-cap stocks, next-day-open is a reasonable approximation. For small-caps and low-volume names, actual fills will be worse.

6. Relative Strength Ranking

In production, relative strength is computed as a universe-ranked percentile (0-100) across all ~2,000 stocks. In backtesting, we use a raw 6-month return ratio vs SPY, linearly mapped to 0-100.

This means a stock's RS rank in the backtest doesn't reflect how it ranked relative to the full universe at that point in time — just how it performed vs SPY over the prior 6 months. Strategies that rely heavily on RS rank (like VCP Breakout) may show different selectivity in backtest vs production.

7. Sample Size Warning

Results with fewer than 30 signals are not statistically significant. A strategy showing 80% win rate on 8 signals means very little — the confidence interval is enormous.

We flag small sample sizes in the UI, but it's your responsibility to not over-interpret limited data. More signals = more confidence. Try longer time periods or broader strategy selections for meaningful statistics.

8. What the Numbers Don't Tell You

• How the strategy performs in market regimes not covered in the test period
• Whether you would have had the discipline to take every signal
• Position sizing impact — a 60% win rate with 0.5R average is still a losing system if you size wrong
• Correlation between signals — multiple signals in the same sector at the same time concentrate risk
• The emotional cost of drawdowns that look small in a table but feel huge in real-time