Curve-fitting in general is the process of finding the (mathematical) description which best matches a given set of data. When its not applied to trading strategies, it can be a very useful way of drawing conclusions from experimental data.
When applied to trading strategies, curve-fitting can produce over-optimized, over-optimistic results. In any set of price data, there is some “magic” combination of indicators and parameters that catches most every move and shows outstanding results. Unfortunately, that magic formula is the result of chance and is different for every data set. That means that future results probably won’t come close to the numbers generated with the full benefit of hindsight.
There’s a fine line here. On the one hand, we want to use backtesting to see how trading strategies performed in the past with an eye to picking the best one to trade. On the other hand, we don’t want to trade a fantasy strategy that has little chance of working in the future.
I’m using the term curve-fitting as the negative connotation of over-optimization and data-mining as the positive connotation of selecting the best of many strategies via backtesting.
Here are three things I do to help avoid the pitfalls of curve-fitting:
- Out-of sample testing, e.g. test and compare results across multiple time periods.
- Select parameters which fall in the middle of a range of good parameters. Avoid the outlier settings that produce much better results than their neighbors.
- Forward-test new trading strategies in live trading with small amounts before committing to full size trades.
See Technical Traders Guide to Computer Analysis of the Futures Marketsfor more against curve-fitting.
(Backtesting Blog is an Amazon Associate.)
Last updated 11/11/08.