Backtesting Honestly: Why Most Backtests Lie to You – Spotting Biases and Fake Fills
Backtesting Honestly: Why Most Backtests Lie to You
Most retail traders run a backtest, see a rosy equity curve, and assume the strategy will work in live markets – but the truth is that many backtests contain hidden flaws that inflate performance. The most common culprits are lookahead bias, phantom fills, midpoint fills and improper spread handling. Understanding these pitfalls is the first step to building a trustworthy performance record.
What is lookahead bias and how does it creep into a backtest?
Lookahead bias occurs when a test uses information that would not have been available at the moment a trade was executed. In practice this means the algorithm “cheats” by seeing future price data, order‑book depth or indicator values before they actually occur.
- Data‑time mismatch: loading daily close prices into a strategy that makes intraday entries creates a timing error – the close of the next bar is already known when the trade is placed.
- Indicator leakage: calculating a moving average that includes the current bar’s price and then using it to decide whether to buy on that same bar gives the system foresight.
- Event‑driven signals: using a news‑feed timestamp that is later than the price timestamp can cause the model to react to news that, in real time, would have arrived later.
To avoid lookahead bias, always align the timestamp of every data point with the exact moment the market would have provided it. In code, this typically means using the previous bar’s values for any decision made on the current bar.
Phantom fills: When your backtest thinks you were filled but you weren’t
A phantom fill is a simulated trade that would never have been executed because the price never actually reached the order level. This happens when a backtester assumes market orders fill at the bar’s closing price regardless of the spread or volatility within the bar.
- Identify the bar’s high‑low range.
- Check whether the order price lies inside that range.
- If it does, assume a fill; if not, discard the trade.
Many simple backtest engines skip step 2, treating every market order as a guaranteed fill. The result is an inflated win rate and a misleading profit factor. A more honest approach is to simulate order execution at the exact price where the market first touched the order level, or to reject the trade if the level was never touched.
Midpoint fills and why they mask real execution costs
Midpoint fills assume that every trade occurs at the exact middle of the bid‑ask spread. While convenient for quick calculations, this assumption hides two important realities:
- Slippage: in fast markets the execution price can be several ticks away from the midpoint, especially for larger orders.
- Spread widening: during volatile periods the spread can widen dramatically, turning a theoretical midpoint fill into a costly loss.
To model execution more realistically, use the actual bid or ask price that would have been hit based on order type:
| Order type | Typical fill price |
|---|---|
| Market buy | Current ask |
| Market sell | Current bid |
| Limit buy | Price ≤ bid when order hits |
| Limit sell | Price ≥ ask when order hits |
Applying these rules forces the backtest to respect the true cost of crossing the spread and reduces the illusion of a perfect fill.
Spread handling: The silent profit killer
Every market has a bid‑ask spread – the difference between the price at which you can sell (bid) and the price at which you can buy (ask). Ignoring the spread is equivalent to assuming you can buy and sell at the same price, which is impossible.
Two common mistakes are:
- Using the same price series for both entry and exit, effectively removing the spread.
- Applying a static spread value that does not change with market conditions, under‑estimating costs during high‑volatility periods.
Honest backtesting should:
- Store separate bid and ask series (or derive them from a known spread).
- Apply the appropriate side of the spread for each trade (ask for buys, bid for sells).
- Update the spread dynamically if your data source provides it, or use a volatility‑based proxy.
When you factor in realistic spreads, many strategies that looked profitable on paper become marginal or even unprofitable, revealing the true edge – or lack thereof.
Putting it all together: A checklist for an honest backtest
Below is a practical checklist you can run before trusting any backtest result. Each item targets one of the biases discussed above.
- Use bar‑by‑bar data with timestamps that match the decision horizon.
- Calculate indicators on the previous bar only; never on the current bar’s incomplete data.
- For every market order, verify that the order price lies within the bar’s high‑low range before assuming a fill.
- Execute trades at the actual bid or ask price, not at the midpoint.
- Maintain separate bid and ask series, and apply the correct side of the spread for each trade.
- Record each simulated fill in a trade journal – an automatic journal such as the one in Tim Edge helps you spot patterns of phantom fills and spread leakage.
- Run the same backtest on out‑of‑sample data to confirm that performance is not a product of over‑fitting.
Following this checklist forces the backtest to behave like a live trading engine, exposing the true risk‑adjusted return.
The bottom line
Backtests that ignore lookahead bias, phantom fills, midpoint fills and realistic spread handling give a false sense of security. By aligning data timestamps, validating fill conditions, using true bid/ask prices and dynamically modelling spreads, you can turn a deceptive backtest into a reliable decision‑making tool. Honest backtesting is the only path to discovering whether a strategy truly has an edge before you risk real capital.