Price and Other Indicators

An Indicator is an abstraction of historical price data which is used to gain insight into the stock or market behavior.   Examples of popular technical indicators are Moving Averages, Bollinger Bands, MACD lines and histogram, RSI, Stochastic Oscillator.

Stockcharts.com has a good comprehensive definition of technical indicators.

Extra Insight:

Indicators describe past market action in a way that can be calculated by computer.   We can objectively define trading signals in terms of the indicators.  For example, we could define a buy signal as the stock price increasing above a moving average.

Its also possible to use indicators subjectively, but that greatly diminishes their value (IMHO).

Backtesting checks market action subsequent to an indicator’s signal.  Backtesting a large number of stocks over a large time period gives insight into how the indicator performed in the past.   Backtesting can only be done with objective rules for evaluating an indicator.

Even an indicator that tested well in the past may not perform well in the future — there are no guarantees.

An objectively defined indicator can help a trader make crisp decisions and trade by a system of rules rather than be ruled by emotion.

Updated: 11/12/08.

Tags: backtesting, data, indicator, macdh, rsi, stochastic, system, technical, trading

By Instrumentation I mean adding my own special instructions into the code for the trading strategies to produce useful information for analysis. 

The commericially available backtesting engines provide limited data and what they do provide is not in a convenient format to process for thousands of runs.

Tags: backtesting, data, instrumentation, trading

Here’s the scoop straight from the SEC on limit orders. 

Extra Insight:

Limit orders are one way to reduce slippage because you specify the exact price you are willing to accept.

Backtesting with end-of-day data models limit orders reasonably well for liquid markets.    The backtesting engine will execute the whole order if the stock traded at limit the price during the day.  This is what normally happens in live trading.    Sometimes in live trading though, the full limit order doesn’t execute, but only a smaller number of shares are traded at the limit price.   This can happen if the limit order is larger than the market can bear at the time.

A small private trader is unlikely to get a partial fill in a large liquid market.   By sticking to high volume stocks, its possible to understand more about the differences between market orders and limit orders via backtesting.

There are Limit-on-Close (LOC) and Limit-on-Open (LOO) orders which corresponds to Market-on-Close and Market-on-Open orders, respectively.   I don’t use them in backtesting though.

Updated: 11/12/08.

Tags: backtesting, data, limit order, market-on-close, market-on-open, trading

A Market on Close (MOC) order is entered before the market closes and the transaction takes place at the day’s closing price.  

The US stock exchanges process these orders.  Check with your broker for exact instructions on how to enter them.

Extra Insight:

Due to using historical end-of-day data, a ”this bar at close” order in backtesting behaves similar to a Market-on-Close order because it takes today’s closing price.   A key difference is that the backtest actually “sees” the closing price before placing the order.   I wish I could do that in real life!

I use the MOC or “this bar at close” order in backtesting only for timed exits because the decision to exit in this case doesn’t depend on the closing price, only the number of days in the trade.

For large orders in thinly traded markets, a live market order might move the live market, resulting in a different closing price than would have occurred without the order – an effect that’s not modeled with historical end-of-day data.

Read a professional’s report here that live MOC orders often execute at the published closing prices.

A small private trader is unlikely to move a large liquid market.   By sticking to high volume stocks, its not only possible to backtest market orders, its also possible to understand more about the differences between market orders and limit orders via backtesting.

Updated: 11/12/08.

Tags: backtesting, data, exit, market order, market-on-close, trading

A Market on Open (MOO) order is entered before the market opens and the transaction takes place at the day’s opening price.  

The US stock exchanges process these orders.  Check with your broker for exact instructions on how to enter them.

Extra Insight:

Due to using historical end-of-day data, a next day market order in backtesting behaves most like a Market-on-Open order because it takes the opening price.   

For large orders in thinly traded markets, a live market order might move the live market, resulting in a different opening price than would have occurred without the order – an effect that’s not modeled with historical end-of-day data.

Several traders report that live MOO orders don’t always execute at the published open prices.   Read a trader complain of worse prices here, and a professional find better prices here.

Updated 11/12/08.

Tags: backtesting, data, market order, market-on-open, trading

Here’s the scoop straight from the SEC on market orders. 

Extra Insight:

Backtesting with end-of-day data differs from live trading with respect to market orders.   No way can an order in backtesting move the market — it is assumed to execute at the historical price.  One way to account for this discrepency is to specify a slippage assumption that the backtesting engine applies to each trade.

Another approach is to reduce slippage by trading in liquid markets — I look for a volume of 500,000 shares to trade.

A small private trader is unlikely to move a large liquid market.   By sticking to high volume stocks, its not only possible to backtest market orders, its also possible to understand more about the differences between market orders and limit orders via backtesting.

Updated: 11/13/08.

Tags: backtesting, data, market order, slippage, trading

Monte Carlo Simulation is a method of stress-testing a trading strategy.   The general idea is to use random data to construct a larger sample space built according to the same results distribution as the original sample.   This more clearly shows the effects of chance on potential outcomes and gives a broader set of data to make decisions.

Monte Carlo methods may be applied at different places in the trading strategy development progress.

One way to apply Monte Carlo methods to backtesting results is to randomly re-sample trades.  Start with the distribution of results for a backtest.  Rather than go  trade to see what happens next, we can run simulated trades.  Tens of thousands of simulated trades.  The result of each simulated trade is generated randomly according to the actual distribution found in the backtesting run.  Then plot the results distribution of all the Monte Carlo simulations to see the broad range of possible outcomes for the trading strategy.

Monte Carlo simulation may also be used to assess the statistical significance of backtesting results.  The process is advocated in Evidence-Based Technical Analysis: Applying the Scientific Method and Statistical Inference to Trading Signalsand described in detail in this paper by Dr. Timothy Masters.  

Extra Insight:

Rather than try to digest the raw results of 100,000+ trades, set boundaries on potential outcomes and use the Monte Carlo method to assess the likelihood of a trading strategy producing those results.   For example, if we define a catastrophic loss as 50% of account value, we can keep track of the number times that happens in 10,000 runs of 1,000 trades each, for example.    That’s one estimate of the probability that the trading strategy will “blow up” in the future.

Of course, the market in the future may not follow the same probability distribution as our initial sample!   Also, we backtest stocks one at the time but a portfolio holds multiple stocks which may move together so the method described above doesn’t exactly model real life.    It is a useful approximation, however.  

For a more comprehensive definition see Wikipedia for Monte Carlo Method and Monte Carlo applied to finance.    For motivation in very accessible terms see Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets

(Backtesting Blog is an Amazon Associate.)

Updated 11/12/08.

Tags: backtesting, data, loss, monte carlo simulation, probability, statistics, strategy, technical, trading

My trading strategies follow this Naming Convention:

[Direction]_Entry_TestPeriod_[Dataset]_Exit

where:

• Direction is either L for buying long or S for selling short.   Direction is optional and if missing defaults to L.
• Entry indicates the entry strategy used.
• TestPeriod is the abbreviated years of the test data.   The data runs from May to April.  So 0407 means May 1, 2004 to April 30, 2007.
• Dataset indicates the data vendor.   It is optional and defaults to CSI Data if not used.
• Exit indicates the exit strategy used.

If one of the above field’s parameters are varied during the test, the exact settings for the run are shown next to it.   If settings are not given, then the commonly used settings apply.

For example, L_All_9404_CSI_Timed_200day

• Trades Long (enters by buying stock)
• Enter always
• Spans the time period  May 1, 1994 - April 30, 2004
• Runs on CSI Data
• Exits on a specific time setting of 200 days

Another example, MACDH_0407_ATR3

• Trades Long (enters by buying stock)
• Enter when MACDH ticks up, settings 12, 26, 9
• Spans the time period  May 1, 2004 - April 30, 2007
• Runs on CSI Data
• Exits on a trailing ATR stop of 3

Updated 11/12/08.

Tags: ATR, backtesting, data, exit, indicator, long, macdh, stop, strategy, trading

Number of Trades or # Trades is simply the number of trades taken by the backtesting engine for a particular trading strategy during a particular time period.

Extra Insight:

I have two reasons for prominently recording the number of trades:

1. The opportunity to trade determines the opportunity to profit and also determines the requirements for capital.
2. Comparing the number of trades between backtesting runs is a good consistency check.

Updated 11/12/08.

Tags: backtesting, strategy, trading

Out-of-sample testing is a way to guard against curve-fitting.   Its a good practice because we don’t know how the market will go in the future. When we ultimately trade our strategy it will be on live data as it evolves, not on the historical price data used for backtesting.

Here’s how out-of-sample testing works:  First a backtest is performed on a given test period.    Then the same backtest is run on a new test period — a different sample of data, hence the name.     If the parameters or settings were over-optimized in the first backtest, its unlikely that they will perform well in the second time period.   

For example, its possible to tweak the parameters on just the right indicators to make over 1000% gains in backtesting.    But when we run those same settings in another period, it might actually lose money.   If it is custom fit to one set of data, it won’t work as well in a different set of data.  Much better to find that out with an additional backtesting run rather than live trading!

Extra Insight:

With two different time periods, the results are almost always going to be at least a little different.   

The most challenging situation is if the original sample is a bull market and the out-of-sample is a bearish period (or vice versa).

My backtesting reports are broken into distinctly different samples for exactly this reason.

To be completely effective, the out-of-sample data should only be used once.   Each backtest should have its own out-of-sample data because if it is used frequently, the out-of-sample data too easily becomes in-sample data.  Using Monte Carlo method is better in this respect.    See Evidence-Based Technical Analysis: Applying the Scientific Method and Statistical Inference to Trading Signalsfor more information.

(Backtesting Blog is an Amazon Associate.)

Updated 11/12/08.

Tags: backtesting, data, indicator, mining, out-of-sample testing, strategy, trading