Before starting the current round of major backtesting, I evaluated several tools to decide which to use.    This article shares the highlights of that endeavor and the main reasons for the outcome.

TradeStation was my incumbent.  By 2007 when I made my last evaluation, I’d had a couple years of experience with it as a charting tool and a backtesting engine.    I’d also used TradeStation Radarscreen to scan the market for opportunities, but found it awkward and slow.   The backtesting had proved reliable but limited to running one stock at a time.     Then TradeStation came out with two critical enhancements:  a means to read in outside data for historical prices, and hooks to automatically process more than one stock per run.

Another favorite tool is Telechartwhich has been my top-down market analysis tool since 2005.    It doesn’t have a backtesting engine.   However, in early 2007 the Worden Bros who make Telechart had just come out with the Blocks Backscanner.    I tried it out extensively and really liked the super support as well as the flexibility.   Its strength is scanning through a huge list of stocks in incredibly fast run time.    But as a very young tool, it didn’t yet have all the features I wanted like independent data sources.  Plus, it was so new  in 2007 that I often felt like a beta tester which is exciting but not what I was looking for to prove out trading strategies.    

I didn’t get past reading the specs on other backtesting tools.    Trading Blox seemed to locked into their own strategies, plus a very high price tag.  I’d previously been exposed MetaStock, struggled with it back in 2004, and was not keen on revisiting it.   I had heard good things about Wealth-Lab but didn’t want to get locked into Fidelity, and didn’t see all the features I wanted there either.

So basically my choice came down to the new Blocks or my old standby TradeStation.   I came to the conclusion that the optimal way to proceed was to rely on each tool’s strengths.    So I use TradeStation for backtesting because I can set up the large-scale, controlled software environment for it.   This gives me a way to prove out a strategy which I do only once per strategy.

Once a strategy is proven and I’m ready to trade it, I usually want to scan the market for opportunities to apply it.   I do this daily, after the market closes, using Blocks 2.0 because it is so very fast at scanning the market.

I recently attended a class on Worden Stockfinder 4.0.  It looks promising –  the Blocks program grown up and renamed.  I’ll review my impressions in another post.

(Backtestingblog is a Worden affiliate, meaning I may be compensated if you buy their product.  Blocks, Stockfinder, Telechart are trademarks of Worden Brothers Inc.   TradeStation is a trademark of TradeStation. TradingBlox is a trademark of Trading Blox. Wealth-Lab is a trademark of Fidelity. MetaStock is a trademark of Equis.)

Tags: backtesting, data, Stockfinder, strategy, Telechart, TradeStation, trading, worden

Cleaning data for backtesting is not easy but its very necessary to get meaningful results.    Mis-adjusted price splits can skew the price data and mislead the unwary backtester into thinking they’re found the holy grail when the strategy merely happens to catch the good side of a bad gap.

Here’s the steps to screen out dirty data and produce a clean dataset:

1. Pick at least 3 candidate data vendors.

2. Format the data for comparison.

3. Write a program to do a smart comparison and run it on the 3 candidate data sets.

4. Analyze the mis-compares to see which set is in error.   if 2 of 3 sets agree, assume that’s the correct value and the outlier is wrong.

5. Send feedback to the data vendors so they can fix the errors.

6. Select the set of historical price data to use for backtesting and lock it down to prevent changes during the backtesting.

7. Feed the golden price data to the backtesting engine.

This process took me several weeks of work but was worth it to get accurate results.  There’s little point of going to the work of backtesting if the underlying data is riddled with errors.

Read on for details if you are going to attempt this on your own or if you just want to see what preparations go into serious backtesting. (more…)

Tags: backtesting, data, google, long, mining, msn, strategy, Telechart, TradeStation, trading, worden, yahoo finance

The Chart Time Scale refers to the amount of time covered by each bar in a price chart. For example, here is a collection of three charts with different time scales for AAPL.

Weekly Chart

The bars on the weekly chart open with Monday’s opening price and close with Friday’s closing price.

Daily Chart

The bars on the daily chart open with each day’s opening price and close with that day’s closing price.   This is the most frequently shown chart time scale.

15 Minute Chart .Daily Chart

 On a 15-minute chart, each bar is 15 minutes wide. The first bar of the day opens with the market open at 9:30am and closes 15 minutes later at 9:45. The next bar opens at 9:45am and closes at 10:00am. This continues throughout the day with 26 bars showing the trading for the 6.5 hour regular session of each day.

Extra Insight:

Since I am using End-of-Day data for backtesting, the corresponding chart time scale is daily.

Different vendors calculate the weekly charts their own ways. TradeStation, shown above, always starts the week on Monday open and ends on Friday close. I believe this is the usual way of doing it. Telechart, however, closes the weekly bar as of the most recent close and uses the open from 5 days ago as the open of the weekly bar. So a weekly chart on a Wednesday in Telechart shows Thursday – Wednesday “weeks” for every bar on the chart all the way back.

Last updated 11/11/08.

Tags: backtesting, data, Telechart, TradeStation