Not all backtest frameworks are equal. Are you clear how yours works?

Whenever I see a backtest that looks spectacular, I am sceptical, and my first instinct is to understand what I am actually looking at. A backtest, which is a simulation of how a trading strategy would have performed on historical data, produces a curve and a handful of numbers, and those numbers can mean quite different things depending on how the test was set up and which tool calculated them.

Before I judge whether a result is any good, I ask two questions: how was each trade sized, and how is the return being calculated? A backtest can mislead in many other ways, from looking ahead at data it could not have known to fitting noise, but these two are the most basic, and they are what this piece is about. The S&P 500 backtests that circulate online are a good place to see why both matter.

The S&P backtest everyone shares

A common example is a strategy on the S&P 500, traded through the SPY exchange-traded fund or the S&P futures contract, where the backtest simply buys one share or one contract on every signal. It looks like a consistent test, because every trade is the same size, but it is not.

SPY closed between about $137 and $156 during 2007, and between about $496 and $690 during 2025. Buying one unit at the end of that period means holding between three and five times the market exposure you held at the start. The same is true of the futures contract, whose value moves with the index. As the index drifts upwards, the strategy is quietly buying more and risking more on every trade. The later years dominate the equity curve, the early drawdowns look small, and the result is no longer a fair picture of the strategy. Part of what the curve is measuring is simply the level of the index.

The S&P makes this worse, because it has a long upward drift. Almost any long strategy on it benefits from that drift, and one-unit sizing amplifies it. You think you are being consistent, but you are not.

The simplest test is to compare the strategy with buying and holding the same instrument, sized the same way, with dividends counted in both or in neither. If the strategy does not clearly beat that, it is adding little beyond the return of the market itself.

The point of a backtest

The question I want a backtest to answer is this: if I had invested a given amount of capital at any point on this curve, what return, what drawdown and what risk should I have expected? A drawdown is the fall from a previous peak in the account to the low that follows it, and it is the number that tells you how painful a strategy is to hold.

To answer that question for every period, the backtest has to treat every period the same way. The size of each trade, and the risk it carries, has to mean the same thing in 2008 as it does in 2024. If it does not, you cannot compare one period with another, and you cannot ask what would happen if you started trading real money in any one of them. This is the same principle as in why your backtest and live trading should agree: a result is only useful if it is consistent.

Fixed base capital is the reference

The simple way to achieve that is to run the backtest on a fixed base capital and to derive each trade’s size from that base. For a single instrument with a base of $100,000, that could mean always buying $100,000 of notional exposure, where notional is the full market value of the position. That keeps the exposure constant, but not the risk: $100,000 of SPY in late 2008 was far riskier than $100,000 in a calm year like 2017. To keep the risk constant as well, you can risk a fixed share of the base on each trade and scale the position to how volatile the instrument is at the time. Either way, the equity is not compounded: profits are not reinvested into larger trades.

This produces a consistent, simple arithmetic return series. Every period carries the same exposure, or the same risk, depending on which you chose, so you can pick any starting point on the curve and read off the return and the drawdown you would have experienced on your capital. It is also the right basis for comparing one run with another, because two versions of a strategy are being measured on exactly the same footing.

Drawdowns are measured in the same way. If the trades are sized from a $100,000 base, a $30,000 fall is a 30% drawdown, even if the account has grown to $400,000 by then, because the trades are no bigger than they were at the start. If the trades had been sized from the current equity instead, they would be four times larger, and the same market move would have cost around $120,000, which is again 30%. The figure that misleads is the one in between: a fixed-base loss of $30,000 divided by the $400,000 account, as many tools do, shows as 7.5% and understates the drawdown four times over.

With futures, two details matter. Size and measure from the price of the contract actually traded at the time, not from a back-adjusted continuous series, whose historical prices have been shifted to join the contracts together. And because contracts are whole units, choose a base large enough that rounding does not distort the sizing, or use the smaller micro contracts.

Compounding is a separate decision

In real life you do want to compound, and that is a separate question with its own risks.

If you are trading your own money, you can size each new trade from your equity at that moment, which gives you something close to daily compounding. If you manage money for investors, you also need to consider how that choice affects the monthly returns they see. In either case, how you compound is a set of choices, and each one should be tested by simulation before real money depends on it. Sizing from equity means the largest trades, and the largest losses in dollars, come straight after the best run. Stepping size up quickly but bringing it down slowly lets leverage creep upwards. Sizing from the peak rather than from current equity keeps trades large while the account is falling. None of these is right or wrong in itself, but each one changes the risk you are taking, so continuous compounding should not be switched on until those effects have been simulated and understood.

Once compounding is involved, the numbers need to be read differently. A compounded curve drawn on an ordinary linear scale bends upwards so steeply that the early years look flat and the recent years look dramatic. On a logarithmic scale, where equal percentage moves take up equal height, the same curve becomes readable, a 20% drawdown looks the same size whether it happened in the first year or the last, and different periods can be compared. As a rule, read a fixed base curve on a linear scale and a compounded one on a log scale. Annual returns should then be stated as a compound annual growth rate, or CAGR, which is the constant yearly rate that would turn the starting capital into the ending capital. Even so, everything should be traceable back to the fixed base capital result, which remains the reference.

Understand the metric you are reading

The same backtest can be described with very different numbers, all of them correct. Suppose $100,000 grows to $600,000 over 20 years.

• Total growth is 500%.
• The simple average annual return is 25% a year: the total profit divided by the starting capital and by the number of years.
• The compound annual growth rate is about 9.4% a year.

“25% a year” and “9.4% a year” describe the same account. They answer different questions, and comparing a simple figure from one backtest with a compounded figure from another tells you nothing. Before trusting a headline return, ask which of these it is, and whether the chart beside it is drawn on a linear or a logarithmic scale.

Understand the tool you are using

Whatever tool you use, it is worth learning exactly how it calculates, either from its documentation or from simple tests of your own. A quick check is to run a trivial strategy that always buys the same notional amount, then compare the tool’s return and drawdown figures with your own calculation. Most tools try to simulate real trading, so the account’s equity rises and falls with each trade, and returns and drawdowns are measured against that moving equity. That is perfectly reasonable, but you then need to know what its average annual return means and how it is calculated, and how it measures drawdowns when growth in equity is already built into the result. Many tools measure drawdown against the running equity whatever the sizing, which can make a fixed-base strategy look far safer than it is. If equity growth is built in, the trade size should usually be set relative to that equity, for example a fixed percentage of it; otherwise the results mix two different things.

The usual sizing options are a fixed number of contracts, a fixed notional amount, or a fixed percentage of equity. MultiCharts, which I use for research, offers a fixed number of shares or a fixed cash amount in its settings, and any other approach has to be written into your own strategy code. This is typical, because most tools are built to be simple for the majority of their users, and simplicity is a good thing. At the more demanding end, though, accuracy means looking deeper and making the tool work the way you need it to.

The lesson

The lesson I would pass on to anyone starting out is that, for every new trade, you need to understand how much risk is being taken relative to the capital at that point in time, and to know whether that capital is a fixed base or the live equity that reflects compounding. Once you know that, and exactly how each figure you are reading was calculated, whether it is the return, the drawdown or any other metric, you can judge a backtest for what it really is and compare it with another on equal terms, whether it is your own or someone else’s.