How About Doing What Works?

marketclubminute3

Adam Hewison’s  Minute 3 video (click here to watch) suggests traders choose between technical and fundamental information for their trading decisions.

I have to ask:  How about using what works?

We have the technology now to check out how well each of the different types of data performed in the past.  You can look at the track record of experts writing newsletters, back test technical indicators, back test fundamental numbers like Earnings Per Share (EPS), Price/Earnings, and most of the key statistics of a company.   Services tell how well seasonal predictions correlated with actuals, and even programs to data-mine for dates a market “always” moves.

With all that at our fingertips, we’re in a great position to estimate the quality of data source, and cherry-pick the data sources that demonstrated their effectiveness — or at least avoid the ones that are complete hooey.

However, our human brains are wired for stories.   Both fundamental analysists and technical analysists can weave compelling stories.  Unfortunately, at times the juiciest stories are not based on the strongest data.

Perhaps the key question is:  Do you want to make your decisions based on objective data with a known track record, or do you want to remain a sucker for a good story?  Adam Hewison is right in that you need to decide what type of information you will use to trade.

Stock Buy Signals

Here is part 2 about stock entry strategy or the buying process.   The previous article talked about stock screening, which is the background investigation to select a pool of candidate stocks to buy when the time is right.   The trigger or market timing signal is the topic of this article.   

 Why You Need to Time Your Entry

Once you have a universe of candidates, you need an entry signal or trigger.  Stocks can sit around looking good enough to buy for a long time, and you need a discrete event to say “Buy Now”.  Hard experience has taught me that “when I have time to complete research” and “when I feel excited about stocks” are not the best entry conditions.    In retrospect, it was usually a price extreme that got me pumped enough to research stocks and hit the buying point.   I’ve found that exercising the judgement to pick a better entry point can be more financially rewarding than just jumping in.     Personally, I suspect that even a random entry point would be better than emotion-driven buying, and backtesting can help identify strategies that do better than random.

 How To Time Your Entry

I see three broad categories that can be used as in entry signal: news events, clock or calendar events, and price events, especially as indicated by objective technical analysis.   Let’s compare them.

 News Events

If you’re new to the stock market, reacting to news events may seem the most natural thing in the world.  However, a little experience shows that the market anticipates and prices in news before it happens.  This is called discounting.  As an example, remember the recent situation with Steve Jobs and Apple.   It follows the saying, “Buy the rumor, sell the news”, only in reverse because bad news is what moves the market lately.   Here’s what happened:  Amid rumors of Jobs’ recurring illness, the price of AAPL declined, all the while Apple insisted Jobs was healthy.   Then Jobs announced that he was taking a medical leave of absence.   If the rumor of illness prompted a decline, then one might think that the news of his departure would tank the stock – he has had an unquestionable impact on the company, after all.    What actually happened, though, is that AAPL traded down to a new 52 week low in after-hours trading on January 14, the day of Jobs’ departure.  The following day, the price opened low, but regained most of it to close at near the high of the day.  Price bounced around the lows for 3 days, and then began an ascent that ended 3 weeks and 30% later.   The market had already priced in the news and the reaction went in the opposite direction, as it often does.    The upshot of this example is that it is difficult, if not impossible, to form an objective strategy around the news because the news may be priced into the market and always must be subjectively interpreted.

 Calendar Events

The second type of entry signals, clock and calendar events, are more objective than the news, but that’s not saying they’re 100% reliable.   Some of the people who use this category of signals are

  • day-traders who never hold overnight
  • pro traders who only hold overnight
  • investors following the adage to “sell in May and go away”
  • small-cap investors who show up in December
  • commodity traders following the seasonal fundamentals
  • and those folks who mine the charts looking for the dates when a stock almost always seems to go a certain way

Some of the calendar-driven moves truly are driven by the calendar. Others are due to coincidence, while still others are illusion.  Backtesting – either automatically or by manually checking the charts – can weed out the pretenders by determining which have been profitable in the past, and that is a useful first step.    I think you owe it to yourself to take it one step further and look for a plausible cause for the move rather than betting good money on a pattern that came about by chance.

Technical Indicator Signals

 The same can be said of technical indicator signals – you need to understand why they work — plus you need to make sure they are objective.   Aronson’s book makes a good case for using objective indicators rather than relying on subjective information for trading decisions.   A signal is objective if there is no “wiggle room” in describing it, if any two people always see it the same way (not like pattern recognition) and/or you could program it into a computer.  Elder’s first book gives good descriptions of technical indicators grounded in crowd behavior.  

 You can also think through the implications of the strategy.   For example, consider the trend-following strategy of buying when price hits a new high.   A new high doesn’t guarantee that the price will keep going, but all runaway stocks had to make new highs along the way.   A good thing to know is how many stocks making new highs go on to make a profit for investors holding for, say, one year.   Backtesting is one good way to estimate this info.   Sign up for email alerts to find out when new highs will be featured in BackTesting Report.             

Backtesting can also help us overcome our human tendency to become overconfident in a signal because we can easily spot on a chart the times that the signals worked and all too easily overlook the false signals.   A false signal is where the signal comes but the stock price doesn’t go in the expected direction long enough for the trader to profit.  It’s expensive to learn about false signals and our little foibles of human cognition in live trading.

The previous article used the example of price above the moving average to illustrate a potential stock screen.  A corresponding signal using moving averages is price crossing the moving average, or moving averages crossing each other.    They offer objective, discrete events to replace emotional guesswork with rational decision-making.  To find out more, check out the BackTesting Report MA Buy Signal package.

Updated on 3/17/09 to add: (BacktestingBlog is an Amazon Associate. )

Updated on 3/19/09 to add: (Author has a position in stocks mentioned in this article. )

BackTesting Moving Averages

Why Moving Averages

As a trader or investor, the only reason to investigate moving averages is to gain knowledge to increase profits. Like many other technical indicators, moving averages are meant to help us objectively tell the market status at any given time. This helps us see through the emotions of the day and make rational decisions, which we’re told will lead to greater profits and/or fewer losses over the long run. Moving averages (MAs) smooth the series of prices for a stock. MAs are most often used to identify the trend of market direction, and are classed as a trend-following indicator. This doesn’t mean that MAs are only for long-term investors – short term traders use them also. Moving averages can be used to screen stocks for good candidates, signal buying opportunities, and offer sell signals.

Why Backtest – A Story

The goal of backtesting is to find out if moving averages really do lead to better results and what are the most promising ways to apply MAs. Let me tell you a short story. While I was putting together the results for one of the moving average BackTesting Report issues, I happened to visit a friend. At her house, I came across some reading material from a well-advertised discount stock broker. In it was an article that advising its customers to use a particular moving average length applied in a certain way to get the best results. I had my comprehensive tests right in front of me and I can tell you that broker’s method did not get the best results although they did mention a MA length that is useful in other ways. I had in my hand test results that showed that the way that broker applied the moving average had a win rate worse than the baseline when tested on 7147 stocks over 14 years of stock market data. Clearly the broker wasn’t running that kind of testing. It’s up to the customers – us! – to fend for ourselves and find out what works versus what doesn’t.

How to Calculate MAs

When backtesting moving averages, the first decision is how to calculate the moving average. Do you want a simple moving average (SMA)? Or something designed to track price better such as an exponential moving average (EMA)? You might consider an experiment to compare the win rates of the two different averages. I did just that a couple years ago, and while I don’t have the results to publish, I came away with the notion that it didn’t make a big difference whether I chose SMA or EMA — just pick one and use it consistently. So for this project, I choose to use simple moving averages because I see them mentioned in commentary most often. To actually do the calculation, I relied on the built-in function which came with TradeStation. (The choice of backtesting engine is another decision which is general enough to write about in another post.)

How to Use MAs

Next you need to pin down how exactly you want to apply moving averages. How will you interpret the relationship between price and moving average? What rules will you use to decide when to buy and sell? You don’t have to read long about stocks before coming across a bullish reference to a stock trading above its 200-day moving average or its 50-day moving average, or even the 10- or 20-day MA. Or advice about buying stocks as they cross their 50-day or 200-day moving average. These are important rules to test in the backtesting engine. And then there’s the moving average crossover – a classic method of technical analysis. That makes three distinct ways of using moving averages to test.

Going more in-depth, some trading texts talk about the slope of a moving average. If you hark back to algebra and consider the MA as a line, to find its slope you would pick two points on the line and apply the usual formula ((x2-x1)/(y2-y1)). This brings up the question of how far apart to pick the two points which can make a difference to results. Really, since the MA is being used to identify the trend, we just want to know if it is sloping up or down. Then we can simplify the whole calculation by noticing that if the price is above the moving average, it must be pulling the average up, and a price below the MA pulls it down. Thus another reason to test the efficacy of price above the moving average.

Parameter settings

Once you decide on how to use the MAs, you need to pick a selection of various lengths to test. Beware of over-optimizing. Somewhere out there is a guy with backtesting results showing 3895% gain or whatever using just the right moving average. Too bad he doesn’t know what MA will produce those results in the future. That said, you need to try more than one length to make sure that your results aren’t a fluke. Stick with defaults settings or the ones you hear about most in the media. Finding the one perfect parameter setting is not going to make you rich. Finding a cluster of good, robust settings just might do you a great deal of good though.

As a practical matter when backtesting allow enough data lag before measuring. All tests must begin measuring at the same place for apples-to-apples comparison among different MA lengths. For example, if you’re testing a 200-day moving average, it will take the first 200-days of data to calculate the first point of that moving average. That means that the first day you could possibly have a signal is 200-days into the data set. To make a fair comparison with, say, the 10-day moving average, you need to make sure not to count any signals from the 10-day moving average before the 200-day is ready to go. Fortunately TradeStation has a way to set the “Maximum number of bars study will reference” in “Properties for All” strategies which forces the backtesting engine to wait that long before tabulating data.

More Profit from Buying or Selling?

Moving average rules, and in particular moving average crossover rules, are often discussed as a reversal system. This means that one signal, say the MAs crossing upwards is a buy signal and then its opposite, say MA lines crossing down, is not only a sell signal but also the trigger to go short. Theoretically, that’s just fine but many people are not interested in shorting the market. They are looking for techniques to help them buy and maybe sell. Even a person who regularly sells and sells short might use different techniques for buying and selling. For these reasons, it’s wise to test the buy signals separately from the sell signals.

This poses a dilemma because it’s hard to evaluate a buy signal in isolation. One way to do this is to use timed exits – that is, exit the trade or sell the stock after a certain amount of time elapses. I chose to run each backtest three times with three different times exits because different people have different styles and different needs. To produce backtesting results useful to swing traders, I exit after 2 days. To model position traders, 20 days. To meet the needs of active investors, backtesting holds each position for 200 days. This gives a way to isolate the buy signals and find out just how useful the moving average is to stock buyers of various temperaments.

Need to Define Goodness

One more very important thing to consider if you are backtesting moving averages to find out how well they do in the stock market: How will you know what is good? You need objective criteria for success. That means identifying the key statistics such as win rate, expectancy, hypothetical equity gains, etc. It also means setting standards for acceptable performance in each of these areas.

An example illustrates why this is important and why it’s not as easy as it first appears. Say your tests show a win rate of 55% for a particular indicator. That may might not be so good if, say, 62% of all stocks went up during the same period of time. Or if only 25% of stocks rose during that time period, your 55% win rate would be spectacular. What is good depends on how it compares to baseline market performance under the same conditions.

You can download a free copy of the BackTesting Report Baseline issue by clicking here.

Test Set

For a meaningful backtest, you need to have enough data to make a statistically valid comparison. At the minimum, that means 30 trades. Even if you are trading just one instrument – just one stock or just one currency pair – I think it’s important to test your trading strategy on many different instruments to prove its robustness. I went over the top with an extremely large test set — 7147 stocks over 14 years — to make sure my results would apply in a wide variety of market conditions.

You can get your copy of my backtesting reports on moving average buy signals by clicking here.

Indicator Definition

Price and Other Indicators
Price and Other Indicators

An Indicator is an abstraction of historical price data which is used to gain insight into the stock or market behavior.   Examples of popular technical indicators are Moving Averages, Bollinger Bands, MACD lines and histogram, RSI, Stochastic Oscillator.

Stockcharts.com has a good comprehensive definition of technical indicators.

Extra Insight:

Indicators describe past market action in a way that can be calculated by computer.   We can objectively define trading signals in terms of the indicators.  For example, we could define a buy signal as the stock price increasing above a moving average.

Its also possible to use indicators subjectively, but that greatly diminishes their value (IMHO).

Backtesting checks market action subsequent to an indicator’s signal.  Backtesting a large number of stocks over a large time period gives insight into how the indicator performed in the past.   Backtesting can only be done with objective rules for evaluating an indicator.

Even an indicator that tested well in the past may not perform well in the future — there are no guarantees.

An objectively defined indicator can help a trader make crisp decisions and trade by a system of rules rather than be ruled by emotion.

Updated: 11/12/08.

Monte Carlo Simulation Definition

Monte Carlo Simulation is a method of stress-testing a trading strategy.   The general idea is to use random data to construct a larger sample space built according to the same results distribution as the original sample.   This more clearly shows the effects of chance on potential outcomes and gives a broader set of data to make decisions.

Monte Carlo methods may be applied at different places in the trading strategy development progress.

One way to apply Monte Carlo methods to backtesting results is to randomly re-sample trades.  Start with the distribution of results for a backtest.  Rather than go  trade to see what happens next, we can run simulated trades.  Tens of thousands of simulated trades.  The result of each simulated trade is generated randomly according to the actual distribution found in the backtesting run.  Then plot the results distribution of all the Monte Carlo simulations to see the broad range of possible outcomes for the trading strategy.

Monte Carlo simulation may also be used to assess the statistical significance of backtesting results.  The process is advocated in Evidence-Based Technical Analysis: Applying the Scientific Method and Statistical Inference to Trading Signalsand described in detail in this paper by Dr. Timothy Masters.  

Extra Insight:

Rather than try to digest the raw results of 100,000+ trades, set boundaries on potential outcomes and use the Monte Carlo method to assess the likelihood of a trading strategy producing those results.   For example, if we define a catastrophic loss as 50% of account value, we can keep track of the number times that happens in 10,000 runs of 1,000 trades each, for example.    That’s one estimate of the probability that the trading strategy will “blow up” in the future.

Of course, the market in the future may not follow the same probability distribution as our initial sample!   Also, we backtest stocks one at the time but a portfolio holds multiple stocks which may move together so the method described above doesn’t exactly model real life.    It is a useful approximation, however.  

For a more comprehensive definition see Wikipedia for Monte Carlo Method and Monte Carlo applied to finance.    For motivation in very accessible terms see Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets

(Backtesting Blog is an Amazon Associate.)

Updated 11/12/08.

Trading System Definition

Trading System refers to the whole set of rules, practices, and habits that make up the process of trading.    This includes market selection, portfolio selection, when to trade, which trading strategies to use when, entry signals, exit signalssizing, record-keeping, risk management.   The whole enchillada.

Extra Insight:

Even though I often use the terms interchangably, I think Trading System is bigger than Trading Strategy.

A Trading System is said to be either mechanical, discretionary, or a mixture of the two. 

Most mechanical systems are run by a computer, but they need not be.  A person could conceivably make manual calculations and monitor trades according to rigid rules.   Even in a fully automated mechanical system, the human element is present — someone must decide which system, when to turn it on, how to keep the computers running, etc.   However, backtesting is an obvious step in the development of a mechanical trading system. 

For discretionary traders, modern trading also relies on computers acting according to fixed rules.  For example, many people, wheither they consider themselves traders or investors, fundamental or technical, consult stock charts populated with their favorite analysis techniques and indicators.   Backtesting can inform the judgement of a discretionary trader by outlining the potential performance of various strategies and indicators.

Ed Seykota often says that a trader’s system is really the set of emotions he/she is unwilling to feel.  (See Sat, 17 July 2004 in his Trading Tribe FAQ).   I feel like dodging by saying the emotional side is beyond the scope of this blog.  

Now that I think about it, its not so hard to backtest a general example with software.    For example, the Rational Choice book cites a few studies that prove our human tendency for loss aversion.  To codify that, write a system with: 

  • no stops in order to avoid the pain of taking a known loss,
  • close targets to avoid the pain of giving profits back, and
  • quick file deletion to avoid the pain of knowing its unprofitable. 

(Backtesting Blog is an Amazon Associate.)

Updated 11/13/08.