With a rally coming together over the past week, I want to review two distinct and important elements of the buying process.   Before I jump in, let me say that I don’t claim to know if this is The Time to buy or not.  I do want to give you two steps to consider when making that decision for yourself.   They are:

• Screening for candidate stocks
• Signaling the time to buy (and sell)

This first, stock screens, is more time invariant - a fancy way of saying the list doesn’t change much day to day. In fact, you may not want to change you stock list much year to year as there is a lot to be said for getting to know a handful of stocks inside and out.   If you’re like me, you itch to scan a huge number of stocks to see offer up the best opportunities.   Of course, you can do both.  I have my screeners delivering new stocks every day and I also have my favorites that I’ve gone back to so many times over the years that I am part of the reason support and resistance works because I know from memory what is a low buying price and when to take the money and run.    On the other hand, a decidedly Bad Idea is obsessively returning to a stock on which you lost thinking that it “owes you”.  This is where an objective plan can really save you.

Sit down at a quiet time and think through the things you want in your candidate stocks.    I consider three categories:  practical matters, fundamentals, and technical screens.   Other folks might want to consider social values, or what their friends say, or worldwide demographics.    I have a hard time making the last three objective myself.

On the bare-bones practical side:

•             Volume is a very measurable criterion that is almost pointless to test.   You need enough volume to get in and out.  Period.  I like to see at least 500,000 shares traded per day.
•             Price is another area where it’s very easy to be objective - at least at the extremes.   If you want to be a penny stock trader - great!  Go be the best.   But most people choose $1, $5, or $10 and say it’s as low as they go.  
•             Your risk limits drive a price/volatility limit.   If you risk only 2% of your account per trade (and that’s being generous), you can’t get into any stock with an average true range bigger than that amount.

Do fundamentals matter or is the sum total of available information reflected in the price?   You will have to make your own decision on that, dear reader, because the fundamental data I had at my disposal last I looked was not clean enough to make a good determination.    Even so, I do use fundamentals as a tie-breaker for the times when I get too many signals at once.   Then I’ll take the signals on the stocks with fundamentals (and story) to my liking.  A better way is to refine the objective rules to come up with just the right number of stocks to fit your account.

Technical stock screens are an important component of your plan.  They can and should be tested thoroughly.   First define what you want, say stocks in an uptrend, or beaten down stocks, or stocks trading in a nice range, whichever.  Then figure out which tool from technical analysis delivers the best candidates for your needs.  This means know how well the various technical analysis tools have performed.  This applies whether you are picking the short list of stocks you will grow with for the next couple years, or setting up a daily screen.   If it’s done right, your technical stock screener can give a solid list of candidates.   Remember, the point is not to go out and buy all of these right away, but instead to stalk them for the right time to make your move.

 For example, a moving average can function as a stock screener, where stocks above the moving average are said to be in an up-trend and stocks below the moving average are said to be in a downtrend.   Many experts recommend buying only stocks which are above their moving average, although some will say you can capture more of the move by getting in well below the averages.  Of course the advice varies on which length of moving average and what you use may depend on whether you have a short- or long-term outlook.    Click here to order your copy of the BackTesting Report on moving average stock screens and find out which moving averages have potential and which have not proven out as a stock screener.

 This post talks about screens stocks to build a candidate list that is bigger than you can use at any one time.   The examples — price, volume, above/below moving averages — showed objective, testable tactics that can identify out a candidate list.  But the stock screener doesn’t pinpoint the time to buy.   For that we need a more precise entry signal which I’ll discuss in the next article.

Tags: backtesting, entry strategy, moving average, risk, signal, Strategy Development

Why Moving Averages

As a trader or investor, the only reason to investigate moving averages is to gain knowledge to increase profits. Like many other technical indicators, moving averages are meant to help us objectively tell the market status at any given time. This helps us see through the emotions of the day and make rational decisions, which we’re told will lead to greater profits and/or fewer losses over the long run. Moving averages (MAs) smooth the series of prices for a stock. MAs are most often used to identify the trend of market direction, and are classed as a trend-following indicator. This doesn’t mean that MAs are only for long-term investors - short term traders use them also. Moving averages can be used to screen stocks for good candidates, signal buying opportunities, and offer sell signals.

Why Backtest - A Story

The goal of backtesting is to find out if moving averages really do lead to better results and what are the most promising ways to apply MAs. Let me tell you a short story. While I was putting together the results for one of the moving average BackTesting Report issues, I happened to visit a friend. At her house, I came across some reading material from a well-advertised discount stock broker. In it was an article that advising its customers to use a particular moving average length applied in a certain way to get the best results. I had my comprehensive tests right in front of me and I can tell you that broker’s method did not get the best results although they did mention a MA length that is useful in other ways. I had in my hand test results that showed that the way that broker applied the moving average had a win rate worse than the baseline when tested on 7147 stocks over 14 years of stock market data. Clearly the broker wasn’t running that kind of testing. It’s up to the customers - us! - to fend for ourselves and find out what works versus what doesn’t.

How to Calculate MAs

When backtesting moving averages, the first decision is how to calculate the moving average. Do you want a simple moving average (SMA)? Or something designed to track price better such as an exponential moving average (EMA)? You might consider an experiment to compare the win rates of the two different averages. I did just that a couple years ago, and while I don’t have the results to publish, I came away with the notion that it didn’t make a big difference whether I chose SMA or EMA — just pick one and use it consistently. So for this project, I choose to use simple moving averages because I see them mentioned in commentary most often. To actually do the calculation, I relied on the built-in function which came with TradeStation. (The choice of backtesting engine is another decision which is general enough to write about in another post.)

How to Use MAs

Next you need to pin down how exactly you want to apply moving averages. How will you interpret the relationship between price and moving average? What rules will you use to decide when to buy and sell? You don’t have to read long about stocks before coming across a bullish reference to a stock trading above its 200-day moving average or its 50-day moving average, or even the 10- or 20-day MA. Or advice about buying stocks as they cross their 50-day or 200-day moving average. These are important rules to test in the backtesting engine. And then there’s the moving average crossover - a classic method of technical analysis. That makes three distinct ways of using moving averages to test.

Going more in-depth, some trading texts talk about the slope of a moving average. If you hark back to algebra and consider the MA as a line, to find its slope you would pick two points on the line and apply the usual formula ((x2-x1)/(y2-y1)). This brings up the question of how far apart to pick the two points which can make a difference to results. Really, since the MA is being used to identify the trend, we just want to know if it is sloping up or down. Then we can simplify the whole calculation by noticing that if the price is above the moving average, it must be pulling the average up, and a price below the MA pulls it down. Thus another reason to test the efficacy of price above the moving average.

Parameter settings

Once you decide on how to use the MAs, you need to pick a selection of various lengths to test. Beware of over-optimizing. Somewhere out there is a guy with backtesting results showing 3895% gain or whatever using just the right moving average. Too bad he doesn’t know what MA will produce those results in the future. That said, you need to try more than one length to make sure that your results aren’t a fluke. Stick with defaults settings or the ones you hear about most in the media. Finding the one perfect parameter setting is not going to make you rich. Finding a cluster of good, robust settings just might do you a great deal of good though.

As a practical matter when backtesting allow enough data lag before measuring. All tests must begin measuring at the same place for apples-to-apples comparison among different MA lengths. For example, if you’re testing a 200-day moving average, it will take the first 200-days of data to calculate the first point of that moving average. That means that the first day you could possibly have a signal is 200-days into the data set. To make a fair comparison with, say, the 10-day moving average, you need to make sure not to count any signals from the 10-day moving average before the 200-day is ready to go. Fortunately TradeStation has a way to set the “Maximum number of bars study will reference” in “Properties for All” strategies which forces the backtesting engine to wait that long before tabulating data.

More Profit from Buying or Selling?

Moving average rules, and in particular moving average crossover rules, are often discussed as a reversal system. This means that one signal, say the MAs crossing upwards is a buy signal and then its opposite, say MA lines crossing down, is not only a sell signal but also the trigger to go short. Theoretically, that’s just fine but many people are not interested in shorting the market. They are looking for techniques to help them buy and maybe sell. Even a person who regularly sells and sells short might use different techniques for buying and selling. For these reasons, it’s wise to test the buy signals separately from the sell signals.

This poses a dilemma because it’s hard to evaluate a buy signal in isolation. One way to do this is to use timed exits - that is, exit the trade or sell the stock after a certain amount of time elapses. I chose to run each backtest three times with three different times exits because different people have different styles and different needs. To produce backtesting results useful to swing traders, I exit after 2 days. To model position traders, 20 days. To meet the needs of active investors, backtesting holds each position for 200 days. This gives a way to isolate the buy signals and find out just how useful the moving average is to stock buyers of various temperaments.

Need to Define Goodness

One more very important thing to consider if you are backtesting moving averages to find out how well they do in the stock market: How will you know what is good? You need objective criteria for success. That means identifying the key statistics such as win rate, expectancy, hypothetical equity gains, etc. It also means setting standards for acceptable performance in each of these areas.

An example illustrates why this is important and why it’s not as easy as it first appears. Say your tests show a win rate of 55% for a particular indicator. That may might not be so good if, say, 62% of all stocks went up during the same period of time. Or if only 25% of stocks rose during that time period, your 55% win rate would be spectacular. What is good depends on how it compares to baseline market performance under the same conditions.

You can download a free copy of the BackTesting Report Baseline issue by clicking here.

Test Set

For a meaningful backtest, you need to have enough data to make a statistically valid comparison. At the minimum, that means 30 trades. Even if you are trading just one instrument - just one stock or just one currency pair - I think it’s important to test your trading strategy on many different instruments to prove its robustness. I went over the top with an extremely large test set — 7147 stocks over 14 years — to make sure my results would apply in a wide variety of market conditions.

You can get your copy of my backtesting reports on moving average buy signals by clicking here.

Tags: backtesting, baseline, data, day, exit, expectancy, goals, indicator, linkedin, long, loss, moving average, selling, short, signal, statistics, strategy, system, technical, TradeStation, trading, trend following

Read remarks from Market Wizard Ed Seykota on back-testing and this blog about back-testing by following the links embedded in this article.

(Backtesting Blog is an Amazon Associate.)

Last updated 02/04/09.

Tags: back-testing, backtesting, Ed Seykota

When people say they want recent data for backtesting, I wonder two things:

1. How do they test longer term strategies using only recent data?   Anything involving a 200 day moving average, for example, needs nearly a year of data just to calculate the average.   Then it needs multiple years of data to get various signals and see how they play out.

2. How did they do in 2008 relying on anything resembling the recent past of 2003-2007.   It seems to me that you need to go back to 2001, at the very least, to find data that represents a crash.

When I selected time periods for backtesting, I looked for a nice long segment that included sideways, up, and down markets.   I ultimately decided on May 1994- April 2004 as the core time period for my testing.    This includes the tech bubble and the bubble bursting in a double-dip recession.  1994 showed less volatility, and the 1998 crisis broke up the monotony of the rising market.

Added to the core period is the next segment for out-of-sample testing.   I read somewhere (sorry, forgot where) that the out-of-sample period should be one-third the length of the initial period.   May 2004- April 2007 fit that bill timewise. The market action during this period was flat and upward biased.

Yes, I could detrend the data but I’ve found that approach very unintuitive.   If I am testing trend-following strategies, it doesn’t make sense to me to remove the trend.  Instead, I do a baseline test using a simple strategy and use it for comparison to supposedly smarter strategies.    This approach, and the fact that I apply it during varying market conditions, provides a way to see if a strategy really adds value.    A strategy demonstrates “alpha” by doing better than the baseline.

Another note about my time periods.   I began my efforts with this data set in May 2007.   Hence the time periods run May - April of each year.

My third time period, May 2007 - April 2008 came about because a) it was there, and b) this period is now one-third the previous period, making it the out-of-sample for the out-of-sample data.   As it turns out, the market action was again very different, allowing further stress-testing of the strategies.   Here again, I think it would be a mistake to focus solely on the carnage of the recent past.   Dawn has so far always followed the dark of night.

Tags: backtesting, baseline, data, long, signal, strategy, time period, Volatility

Cleaning data for backtesting is not easy but its very necessary to get meaningful results.    Mis-adjusted price splits can skew the price data and mislead the unwary backtester into thinking they’re found the holy grail when the strategy merely happens to catch the good side of a bad gap.

Here’s the steps to screen out dirty data and produce a clean dataset:

1. Pick at least 3 candidate data vendors.

2. Format the data for comparison.

3. Write a program to do a smart comparison and run it on the 3 candidate data sets.

4. Analyze the mis-compares to see which set is in error.   if 2 of 3 sets agree, assume that’s the correct value and the outlier is wrong.

5. Send feedback to the data vendors so they can fix the errors.

6. Select the set of historical price data to use for backtesting and lock it down to prevent changes during the backtesting.

7. Feed the golden price data to the backtesting engine.

This process took me several weeks of work but was worth it to get accurate results.  There’s little point of going to the work of backtesting if the underlying data is riddled with errors.

Read on for details if you are going to attempt this on your own or if you just want to see what preparations go into serious backtesting. (more…)

Tags: backtesting, data, google, long, mining, msn, strategy, Telechart, TradeStation, trading, worden, yahoo finance

As a private trader, I like to scan the whole US stock market for opportunities each day.  I want a backtesting stock list that is representative of my trading style.  However, most other backtesting publications are limited to the stocks in the S&P500 or even the SP-500 index itself.   For my backtesting, I choose a much wider selection of stocks.   

The foundation of my stock list is the top 7500 stocks on NYSE, AMEX, NASDAQ in 1994.   This list is somewhat representative of the world markets too by including ADRs (international stocks cross-listed on the US exchanges.)   

Unlike most readily available backtesting data, I included stocks that have since been delisted - either failed, merged, taken private.  I bought the premium-priced delisted historical data, as well as the regular stock data, from CSI Data.   For example, Lehman Bros (LEH) is in my test set even though they have been delisted from the major exchanges and relegated to the pink sheets with TWA airlines.  In the process, both these tickers disappeared from most historical data sets which leaves the remaining data with a strong survivorship bias.  Using delisted data more accurately models the markets.

I spent several weeks validating which reduced the test set of stock tickers by screening out dirty data.   Of course, this also increased the accuracy.

The list further shrinks as time moves forward and the delisted stocks drop off.    I did not purchase new listings because the test set is still more than big enough to make a statistically valid sample.   Sometimes new listings appear on their own when tickers get “recycled”  – new companies adopt the ticker of delisted companies.    At times, especially in the longer test periods, there are actually two different companies using the same ticker (they start with different tickers, one dies, the other changes to the dead company’s ticker and that ticker is applied to all historical data for that stock).  These cases are denoted by appending “DUP” to the ticker for one of the companies. 

My stock lists are available here, by test period:

May 1994 - April 2004

May 2004 - April 2007

May 2007 - April 2008

Tags: backtesting, data, S&P500

I use the term Active Investor to identify people who manage their market positions.  An active investor not only investigates a stock before buying, but also considers when and why they might sell.   

This distinguishes the active investor from the passive investor.  The latter often simply invests in a market index, and whatever they buy, they set out to keep for the long term.   Hence the moniker “buy and hold”.

Extra Insight:

In an effort to match up to trader types in my backtesting, I estimated that an active investor sticks with a position for about a year.

Tags: backtesting, long

The ATR Trailing Stop is one way to limit losses and protect profits. A stop loss order is set a multiple of the Average True Range (ATR) away from the current stock price. As the price moves in the trade’s favor, the stop rachets along with, always calculated from a better closing prices and never from worse closing prices.    This mostly keeps from giving ground once its protected by the stop, except in the case of increasing volatility as measured by the ATR.

Click here for Back Test Performance of Trailing Stops 

Chuck LeBeau popularized the method of trailing a stop loss order a few ATRs below the recent high price for a long trade. This method became known as the Chandelier Stop.    LeBeau’s Book covers other aspects of ATRs.   The best description of the Chandelier exit is in Come Into My Trading Room: A Complete Guide to Trading by Alexander Elder.

The ATR Trailing Stop is also known as a volatility stop.

Extra Insight:

In backtesting, the ATR Trailing Stop reflects each stock’s unique daily price range.  Hence it can fit each stock better than a dollar trailing stop or even a percentage trailing stop.

As with all trailing stops, the ATR trail never exits at the extreme of a movement. Hence it always gives back some of the profits.

The ATR stop amount can be subtracted from either the high, the close, or the low of the day.    Each variation gives slightly different results.    The important concept is to match the stop distance to the stock’s volatility and to move it along with improving prices.

ATR stops are not offered by brokers, to my knowledge.   They are also tedious to calculate by hand.   The only realistic way to use an ATR stop is with software support.   Programmable software packages such as TradeStation can be programmed to display (and backtest!) an ATR stop.   

Click here for BackTesting Reports on Trailing Stops

(Backtesting Blog is an Amazon Associate.)

Last updated 11/11/08.

Tags: Alexander Elder, ATR, backtesting, chandelier Chuck LeBeau, exit, long, stop, stop loss, strategy, TradeStation, trading, Volatility

Avg Hold is also known as the average holding time.  

The holding time is the length of time that the trade is open.   The average is taken across all the trades which resulted from backtesting a particular strategy.

The average holding time is measured on the same scale as the chart time scale.    Since I am backtesting on daily charts, the avg hold is measured in days.

Extra Insight:

When backtesting a strategy, its useful to know the average holding time of all the trades made by that strategy.   This gives an idea of how long the funds are needed - on average — for each trade.   It also gives some insight into the type of trader that might like the strategy.

All else being equal, the trading strategy with the smallest average hold time is best.   Not all things are equal in practice though.   People may seek out longer or shorter hold times based on their temperment, tax situation, global market views, margin, and a host of other factors.   The average hold time gives insight into which strategies will fit these other requirements.  I also use it to classify different strategies and compare those with similar hold times.

Last updated 11/11/08.

Tags: avg hold, backtesting, productivity

Backtesting is the process of checking a trading strategy by applying the strategy to historical price data to generate hypothetical trades, then analyzing the results to assess the performance of the trading strategy.

The backtesting engine is the most glorified piece of the operation however it would not be successful without clean data, objective trading strateiges, statiscally sound analysis, and someone to do perform all the tests.

Although most traders concur that backtesting is useful, many traders don’t do it because of the time, expense, and expertise required.    

Backtesting provides a model of trading.   Significant differences with live trading exist such as: commissions, slippage or skid, after hours trading, dirty data, software bugs, human error.  Backtesting results don’t guarantee future performance.    

Backtesting is a screening tool that is really good at eliminating under-performing trading strategies.   If a trading strategy didn’t perform well in the past, it’s unlikely to make money in the future.     

If a set of rules does perform well in backtesting - that’s great but not the end of the game.  Traders ultimately must decide if a given strategy is suitable for their own use.

Last updated 11/11/08.

Tags: backtesting, data, skid, slippage, trading