Curve-fitting in general is the process of finding the (mathematical) description which best matches a given set of data.    When its not applied to trading strategies, it can be a very useful way of drawing conclusions from experimental data.

 When applied to trading strategies, curve-fitting can produce over-optimized, over-optimistic results.   In any set of price data, there is some “magic”  combination of indicators and parameters that catches most every move and shows outstanding results.    Unfortunately, that magic formula is the result of chance and is different for every data set.   That means that future results probably won’t come close to the numbers generated with the full benefit of hindsight.

Extra Insight: 

There’s a fine line here.   On the one hand, we want to use backtesting to see how trading strategies performed in the past with an eye to picking the best one to trade.    On the other hand, we don’t want to trade a fantasy strategy that has little chance of working in the future.

I’m using the term curve-fitting as the negative connotation of over-optimization and data-mining as the positive connotation of selecting the best of many strategies via backtesting. 

Here are three things I do to help avoid the pitfalls of curve-fitting:

• Out-of sample testing, e.g. test and compare results across multiple time periods.
• Select parameters which fall in the middle of a range of good parameters.   Avoid the outlier settings that produce much better results than their neighbors.
• Forward-test new trading strategies in live trading with small amounts before committing to full size trades.

See Technical Traders Guide to Computer Analysis of the Futures Marketsfor more against curve-fitting.

(Backtesting Blog is an Amazon Associate.)

Last updated 11/11/08.

Tags: backtesting, curve-fitting, data, forward-testing, indicator, mining, probability, sizing, strategy, trading

A Drawdown is a dip in account value from its highest point.

Wikipedia has a very rigorous definition here.

Extra Insight:

An open drawdown is calculated by taking the current market value of both open and closed positions.

Two tricky things about drawdowns:  knowing how much you can handle, and estimating how much you might be in for.  That’s the key to pick a trading system (strategy and sizing) that doesn’t risk more than you can afford to lose.

During backtesting, keeping track of adverse excursions, or how far winning trades go in the wrong direction can give some insight into potential drawdowns of a trading strategy.  

To be absolutely sure to avoid devastating drawdowns its necessary to limit the amount of money in play.

Each person must define what level is devastating for themselves however 50% loss is often called “blowing up”.

Last updated 11/11/08.

Tags: backtesting, drawdown, loss, risk, sizing, system, trading

Forward-testing, means trading a strategy live with very small size to see how well the strategy (and the trader!) perform in real life.   

Forward-testing is typically done after backtesting to make sure the trading strategy is not the over-optimized result of curve-fitting or data mining.  It also gives a chance to try out the mechanics of entering, tracking and exiting trades.

Extra Insight:

I’ve heard varying advice on the size of trades for forward-testing ranging from smallest possible size - think 1 share - up to just enough to engage the trader’s emotions.

For more insight into this topic, Design, Testing, and Optimization of Trading Systemscomes highly recommended.

(Backtesting Blog is an Amazon Associate.)

Updated: 11/12/08.

Tags: backtesting, curve-fitting, data, exit, forward-testing, sizing, strategy, trading

Risk management is crucial because all trading strategies lose sometimes.    By limiting risk, a trader has a chance to survive long enough to find a way to thrive.   The topic is way too big for my little glossary but its here because I want to encourage traders to think about risk.

Extra Insight: 
Funny Carrie Underwood parody reminder to manage risk:

Backtesting trading strategies provides insight into the risk of various trading strategies.   Understanding risk is the first step towards managing it.   A complete understanding includes the knowledge that there’s always the risk of something completely unknown and unforeseen happening.

Different people have different risk tolerance and even the same person will view various risk differently.  That means you have to decide for yourself what to risk.

Here are a few “rules of thumb”:

• Don’t risk more than you can afford to lose.
• Limit the risk on each position.  1% of your trading account risked per stock is a middle-of-the-road estimate.  By risked, I mean the amount lost if the stop is hit.  No stop?  Then limit the size of the whole position to a small fraction of your account.
• Limit the total exposure of your account.   What happens if all your stops get hit?   What happens if the market gaps past all your stops.   Make sure you can handle it.
• “Blowing up” means losing 50% of account value.   Don’t let that happen!

Here’s a link to the book advertised in the video: Mastering the Trade (McGraw-Hill Trader’s Edge).

(Backtesting Blog is an Amazon Associate.)

Updated 11/12/08.

Tags: backtesting, long, management, risk, sizing, stop, strategy, trading

Size Matters

Sizing is the rule for deciding how many shares or contracts to buy.

Extra Insight:

Sizing is critical to risk management, worthwhile returns, and also making comparisions between backtesting runs.   For the backtesting runs, I use a very common and straightforward sizing:

• If there is no stop loss for the strategy under test, my backtesting trade size is 1000 shares (and the amount at risk is the total amount of the trade).
• If there is a stop loss, my backtesting trade size is the nominal risk amount of $1000 divided by the distance from the expected entry price to the stop price.   If its a next-day market order then today’s close serves as the expected entry price.   This way, the risk amount is constant for every trade but the trade size varies in both dollar amount and number of shares.

(Backtesting Blog is an Amazon Associate.)

Updated 11/13/08.

Tags: backtesting, expectancy, loss, management, risk, sizing, stop, strategy, trade size, trading

TradeStation Strategy

A Trading Strategy is the collection of rules about when to enter and exit trades as well as the size of each trade.

Extra Insight:

Sometimes people say “trading system” instead and I do it too.   I really think trading system is larger than just the entry/exit strategies and includes things like record keeping, etc.

TradeStation is a software tool for analysis and backtesting with facilities for creating custom trading strategies.    The image at the top of this article is a screen shot of a very simple entry strategy that buys whenever the stock meets minimum volume requirements.    This obviously is not a tradeable strategy but is something I use as a baseline for comparison.

Updated 11/13/08.

Tags: backtesting, entry, exit, sizing, strategy, system, TradeStation, trading

Trading System refers to the whole set of rules, practices, and habits that make up the process of trading.    This includes market selection, portfolio selection, when to trade, which trading strategies to use when, entry signals, exit signals,  sizing, record-keeping, risk management.   The whole enchillada.

Extra Insight:

Even though I often use the terms interchangably, I think Trading System is bigger than Trading Strategy.

A Trading System is said to be either mechanical, discretionary, or a mixture of the two. 

Most mechanical systems are run by a computer, but they need not be.  A person could conceivably make manual calculations and monitor trades according to rigid rules.   Even in a fully automated mechanical system, the human element is present — someone must decide which system, when to turn it on, how to keep the computers running, etc.   However, backtesting is an obvious step in the development of a mechanical trading system. 

For discretionary traders, modern trading also relies on computers acting according to fixed rules.  For example, many people, wheither they consider themselves traders or investors, fundamental or technical, consult stock charts populated with their favorite analysis techniques and indicators.   Backtesting can inform the judgement of a discretionary trader by outlining the potential performance of various strategies and indicators.

Ed Seykota often says that a trader’s system is really the set of emotions he/she is unwilling to feel.  (See Sat, 17 July 2004 in his Trading Tribe FAQ).   I feel like dodging by saying the emotional side is beyond the scope of this blog.  

Now that I think about it, its not so hard to backtest a general example with software.    For example, the Rational Choice book cites a few studies that prove our human tendency for loss aversion.  To codify that, write a system with: 

• no stops in order to avoid the pain of taking a known loss,
• close targets to avoid the pain of giving profits back, and
• quick file deletion to avoid the pain of knowing its unprofitable. 

(Backtesting Blog is an Amazon Associate.)

Updated 11/13/08.

Tags: backtesting, behavioral finance, discretionary s, Ed Seykota, entry, exit, indicator, loss, mechanical s, Rational Choice, risk, signal, sizing, strategy, system, technical, trading