Telechart for Forward-Testing Trading Strategies

Last week I wrote an article about a situation that often stymies traders who are serious about learning a discretionary strategy.   During forward-testing, many tools inadvertantly give clues as to what the next bar will do, rendering the effort useless.

One tool, however, presents a clean view of the chart at all times.   That tool is Telechart by Worden Brothers.   

See more about Telechart – click here

Here’s an example.  Can you tell from the charting (not the price action) what the next bar will do?

telechart_stock_chart_forward_walk_1

I don’t think so.   Click here to see the chart with one more bar.

Forward-testing – done correctly – gives a trader the chance to see a trading strategy in action does bar by bar.  Even though it lacks the emotional component of live trading, its a necessary step to learning discretionary trading.  It does take real effort though.

If you’re serious enough to do forward-testing, then you owe it to yourself to set up an environment without “cheats” that cheat yourself out of the real experience.  Telechart  is the one tool in my kit that helps me do that.

Forward-Testing Spoilers

forward-testing spoiler

Many of the tools I’ve tried subtly spoil the surprise when used for forward-testing.

What I mean by forward-testing is taking a chart, scrolling back in time – without peeking! – and then stepping forward one bar at a time.  The purpose of the exercise is to try out a discretionary trading strategy in a safe environment.   Granted it is not the same as live trading but it does give one a way to see how  the decision-making  might work, if not feel.

The trouble is that many tools give away the secret of the next bar with subtle and not-so-subtle hints about what the next bar is going to do.   These type of signs you wouldn’t really find at the right edge of a real chart, they are artifacts of display in the middle of the price history.  

Take this example at the top of this article.   Can you guess what happens with the next bar without applying any trading strategy, just looking at how the chart is rendered?

Next post will explain this example and show you a tool that’s not a spoiler.

Curve-Fitting Definition

 Curve-fitting in general is the process of finding the (mathematical) description which best matches a given set of data.    When its not applied to trading strategies, it can be a very useful way of drawing conclusions from experimental data.

 When applied to trading strategies, curve-fitting can produce over-optimized, over-optimistic results.   In any set of price data, there is some “magic”  combination of indicators and parameters that catches most every move and shows outstanding results.    Unfortunately, that magic formula is the result of chance and is different for every data set.   That means that future results probably won’t come close to the numbers generated with the full benefit of hindsight.

Extra Insight: 

There’s a fine line here.   On the one hand, we want to use backtesting to see how trading strategies performed in the past with an eye to picking the best one to trade.    On the other hand, we don’t want to trade a fantasy strategy that has little chance of working in the future.

I’m using the term curve-fitting as the negative connotation of over-optimization and data-mining as the positive connotation of selecting the best of many strategies via backtesting. 

Here are three things I do to help avoid the pitfalls of curve-fitting:

  • Out-of sample testing, e.g. test and compare results across multiple time periods.
  • Select parameters which fall in the middle of a range of good parameters.   Avoid the outlier settings that produce much better results than their neighbors.
  • Forward-test new trading strategies in live trading with small amounts before committing to full size trades.

See Technical Traders Guide to Computer Analysis of the Futures Marketsfor more against curve-fitting.

(Backtesting Blog is an Amazon Associate.)

Last updated 11/11/08.

Data Mining Definition

Data-mining is the process of selecting the best of many strategies via backtesting.   Each strategy is tested across the same stocks and time periods.  Results are compared, and the best strategy from (proper) backtesting is likely to be the best strategy to trade live.   

Live trading performance numbers will vary as the market will not behave exactly like the historical price data used in the backtest, as well as other factors.  

Extra Insight:

Care must be taken to design the process to pick a robust strategy that will work under real-life market conditions going forward.   Limiting the number of degrees of freedom, e.g. the number of parameters, is one helpful tactic.    Another is to use advanced mathematics to gauge the statistical significance of the results.

David Aronson’s Evidence-Based Technical Analysis is an excellent reference for taking a scientific approach to data mining.  It also contains a chapter on estimating the statistical significance of trading strategies selected by data-mining.    Aronson says that data mining can identify the best strategy but, because results will have an upward “data-mining bias”, they should not be used to estimate the performance of that strategy.   Thus only relative comparisons between strategies are possible.

I backtested a baseline strategy and use it as a reference for comparison.

(Backtesting Blog is an Amazon Associate.)

Last updated 11/11/08.

Forward-Testing Definition

Forward-testing, means trading a strategy live with very small size to see how well the strategy (and the trader!) perform in real life.   

Forward-testing is typically done after backtesting to make sure the trading strategy is not the over-optimized result of curve-fitting or data mining.  It also gives a chance to try out the mechanics of entering, tracking and exiting trades.

Extra Insight:

I’ve heard varying advice on the size of trades for forward-testing ranging from smallest possible size – think 1 share – up to just enough to engage the trader’s emotions.

For more insight into this topic, Design, Testing, and Optimization of Trading Systemscomes highly recommended.

(Backtesting Blog is an Amazon Associate.)

Updated: 11/12/08.