A Market on Open (MOO) order is entered before the market opens and the transaction takes place at the day’s opening price.  

The US stock exchanges process these orders.  Check with your broker for exact instructions on how to enter them.

Extra Insight:

Due to using historical end-of-day data, a next day market order in backtesting behaves most like a Market-on-Open order because it takes the opening price.   

For large orders in thinly traded markets, a live market order might move the live market, resulting in a different opening price than would have occurred without the order – an effect that’s not modeled with historical end-of-day data.

Several traders report that live MOO orders don’t always execute at the published open prices.   Read a trader complain of worse prices here, and a professional find better prices here.

Updated 11/12/08.

Tags: backtesting, data, market order, market-on-open, trading

Here’s the scoop straight from the SEC on market orders. 

Extra Insight:

Backtesting with end-of-day data differs from live trading with respect to market orders.   No way can an order in backtesting move the market — it is assumed to execute at the historical price.  One way to account for this discrepency is to specify a slippage assumption that the backtesting engine applies to each trade.

Another approach is to reduce slippage by trading in liquid markets — I look for a volume of 500,000 shares to trade.

A small private trader is unlikely to move a large liquid market.   By sticking to high volume stocks, its not only possible to backtest market orders, its also possible to understand more about the differences between market orders and limit orders via backtesting.

Updated: 11/13/08.

Tags: backtesting, data, market order, slippage, trading

Monte Carlo Simulation is a method of stress-testing a trading strategy.   The general idea is to use random data to construct a larger sample space built according to the same results distribution as the original sample.   This more clearly shows the effects of chance on potential outcomes and gives a broader set of data to make decisions.

Monte Carlo methods may be applied at different places in the trading strategy development progress.

One way to apply Monte Carlo methods to backtesting results is to randomly re-sample trades.  Start with the distribution of results for a backtest.  Rather than go  trade to see what happens next, we can run simulated trades.  Tens of thousands of simulated trades.  The result of each simulated trade is generated randomly according to the actual distribution found in the backtesting run.  Then plot the results distribution of all the Monte Carlo simulations to see the broad range of possible outcomes for the trading strategy.

Monte Carlo simulation may also be used to assess the statistical significance of backtesting results.  The process is advocated in Evidence-Based Technical Analysis: Applying the Scientific Method and Statistical Inference to Trading Signalsand described in detail in this paper by Dr. Timothy Masters.  

Extra Insight:

Rather than try to digest the raw results of 100,000+ trades, set boundaries on potential outcomes and use the Monte Carlo method to assess the likelihood of a trading strategy producing those results.   For example, if we define a catastrophic loss as 50% of account value, we can keep track of the number times that happens in 10,000 runs of 1,000 trades each, for example.    That’s one estimate of the probability that the trading strategy will “blow up” in the future.

Of course, the market in the future may not follow the same probability distribution as our initial sample!   Also, we backtest stocks one at the time but a portfolio holds multiple stocks which may move together so the method described above doesn’t exactly model real life.    It is a useful approximation, however.  

For a more comprehensive definition see Wikipedia for Monte Carlo Method and Monte Carlo applied to finance.    For motivation in very accessible terms see Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets

(Backtesting Blog is an Amazon Associate.)

Updated 11/12/08.

Tags: backtesting, data, loss, monte carlo simulation, probability, statistics, strategy, technical, trading

My trading strategies follow this Naming Convention:

[Direction]_Entry_TestPeriod_[Dataset]_Exit

where:

• Direction is either L for buying long or S for selling short.   Direction is optional and if missing defaults to L.
• Entry indicates the entry strategy used.
• TestPeriod is the abbreviated years of the test data.   The data runs from May to April.  So 0407 means May 1, 2004 to April 30, 2007.
• Dataset indicates the data vendor.   It is optional and defaults to CSI Data if not used.
• Exit indicates the exit strategy used.

If one of the above field’s parameters are varied during the test, the exact settings for the run are shown next to it.   If settings are not given, then the commonly used settings apply.

For example, L_All_9404_CSI_Timed_200day

• Trades Long (enters by buying stock)
• Enter always
• Spans the time period  May 1, 1994 – April 30, 2004
• Runs on CSI Data
• Exits on a specific time setting of 200 days

Another example, MACDH_0407_ATR3

• Trades Long (enters by buying stock)
• Enter when MACDH ticks up, settings 12, 26, 9
• Spans the time period  May 1, 2004 – April 30, 2007
• Runs on CSI Data
• Exits on a trailing ATR stop of 3

Updated 11/12/08.

Tags: ATR, backtesting, data, exit, indicator, long, macdh, stop, strategy, trading

Number of Trades or # Trades is simply the number of trades taken by the backtesting engine for a particular trading strategy during a particular time period.

Extra Insight:

I have two reasons for prominently recording the number of trades:

1. The opportunity to trade determines the opportunity to profit and also determines the requirements for capital.
2. Comparing the number of trades between backtesting runs is a good consistency check.

Updated 11/12/08.

Tags: backtesting, strategy, trading

The OrderTypes field in the settings report lists the types of orders used for the backtesting runs.

Updated 11/12/08.

Tags: backtesting, order types

Out-of-sample testing is a way to guard against curve-fitting.   Its a good practice because we don’t know how the market will go in the future. When we ultimately trade our strategy it will be on live data as it evolves, not on the historical price data used for backtesting.

Here’s how out-of-sample testing works:  First a backtest is performed on a given test period.    Then the same backtest is run on a new test period — a different sample of data, hence the name.     If the parameters or settings were over-optimized in the first backtest, its unlikely that they will perform well in the second time period.   

For example, its possible to tweak the parameters on just the right indicators to make over 1000% gains in backtesting.    But when we run those same settings in another period, it might actually lose money.   If it is custom fit to one set of data, it won’t work as well in a different set of data.  Much better to find that out with an additional backtesting run rather than live trading!

Extra Insight:

With two different time periods, the results are almost always going to be at least a little different.   

The most challenging situation is if the original sample is a bull market and the out-of-sample is a bearish period (or vice versa).

My backtesting reports are broken into distinctly different samples for exactly this reason.

To be completely effective, the out-of-sample data should only be used once.   Each backtest should have its own out-of-sample data because if it is used frequently, the out-of-sample data too easily becomes in-sample data.  Using Monte Carlo method is better in this respect.    See Evidence-Based Technical Analysis: Applying the Scientific Method and Statistical Inference to Trading Signalsfor more information.

(Backtesting Blog is an Amazon Associate.)

Updated 11/12/08.

Tags: backtesting, data, indicator, mining, out-of-sample testing, strategy, trading

The Percentage Trailing Stop is one way to limit losses and protect profits.  A stop loss order is set a given percentage away from the current price.    As the price moves in the trader’s favor, the stop rachets along with, never giving ground once its protected by the stop.   For example, after buying long, a trader may set a trailing stop 7% below the current price.   As the price moves up, the trader moves up the stop but never moves it down when the price goes down.   Eventually the price does retrace the 7%, the stop is hit, and the trade exits.

Extra Insight:

In backtesting, the same percentage value is applied to all stocks.   This is not ideal because each stock has a different daily price range — some will routinely move 3% in a day while others barely budge.    The percentage trailing stop adapts to the individual stock better than the dollar trailing stop but not as well as the ATR trailing stop.  

As with all trailing stops, the percentage trail never exits at the extreme of a movement.   Hence it always gives back some of the profits.

Click here for BackTesting Reports on Trailing Stops

(Backtesting Blog is an Amazon Associate.)

Updated 11/12/08.

Tags: exit, long, loss, stop, trading, trailing backtesting

 A Position Trader seeks intermediate-term opportunites.   Like a swing trader, the position trader is ready with a quick exit strategy but more willing to stay with a trade for several weeks or more.   

Extra Insight:  

For backtesting, I use timed 20 day exits to approximate position trading.

(Backtesting Blog is an Amazon Associate.)

Updated 11/17/08.

Tags: backtesting, exit, position trader, strategy, trading

The Probability section of my backtesting report breaks down into two mathematical constructs: expectancy and standard deviation.  The intent is to give insight into the relative performance of a trading strategy over a large number of trades, and to help traders select profitable strategies (positive expectancy).  

Extra Insight: 

The most desirable case is a high expectancy with a low standard deviation.

Like the toss of a coin, a strategy may vary considerably from the reported results for a small sample of trades.

For a full definition of probability theory, get a textbook on the topic.

(Backtesting Blog is an Amazon Associate.)

Updated on 11/12/08.

Tags: expectancy, probability, standard deviation