How to Test Your Trading Strategy Without Fooling Yourself

The Backtest That Lies to You

Almost everyone who asks me how to test your trading strategy has already done the test. They have a rule set, they have scrolled back through the chart, and they have a result they are pleased with. What they want from me is confirmation.

I am not going to give it, and not because I doubt the work. I doubt the procedure. Scrolling back through a chart and counting wins feels like evidence, and under the right conditions it produces a number that means almost nothing at all. The problem is not that people test badly. It is that the ordinary way of testing has a specific mathematical flaw, and once you see it you cannot unsee it.

So let us do this properly. Not with encouragement, with arithmetic.

How to Test Your Trading Strategy When Luck Is Also Applying for the Job

Start with the flaw, because everything else follows from it.

Suppose you build a strategy with no edge whatsoever. A coin flip. It wins exactly 50 percent of the time and every trade is independent. You test it over 50 trades and ask whether it cleared a 60 percent win rate.

A single coin flip strategy clears that bar 10.1 percent of the time. Roughly one attempt in ten. That is already higher than most people expect, but on its own it is manageable.

Now be honest about what you actually did. You did not test one variant. You tried a different stop distance. You moved the moving average from 20 to 50. You added a session filter, then removed it. You tested it on gold, then on the index. Each of those is a separate attempt, and the question is no longer "did my strategy beat the bar" but "did any of my attempts beat the bar".

That is a completely different question, and the arithmetic is brutal.

Twenty variants, none of which has any edge at all, and 88.2 percent of the time at least one of them looks like a winner. Fifty variants and it is 99.5 percent, a near certainty. The strategies never improved. The only thing that changed was how many times you asked.

You did not find an edge. You found the luckiest member of a group you never counted.

I computed these in Python from the binomial distribution, assuming a true 50 percent win rate and independent trades, and you can recalculate them with different thresholds in a few lines. The assumptions are deliberately generous to the trader: real trading has costs, which makes the picture worse rather than better.

This is not a fringe concern. It has a name in the literature, and four mathematicians put it plainly in the Notices of the American Mathematical Society: it takes a relatively small number of trials to identify an investment strategy with a spuriously high backtested performance. Bailey, Borwein, López de Prado and Zhu were writing about professional fund research, where the same failure destroys real money at scale. The retail version is the same mistake with a smaller budget.

The Second Trap: A Real Edge That Looks Broken

The first trap makes nothing look like something. The second does the reverse, and it is the reason people abandon strategies that were working.

Take a strategy with a genuine edge: it truly wins 55 percent of the time at one to one. That is a good, realistic, unspectacular edge. Now ask how often a sample of it will look like a losing strategy, showing under 50 percent wins.

I simulated 200,000 samples at each length:

• After 20 trades, it looks like a loser 24.8 percent of the time.
• After 50 trades, 19.8 percent.
• After 100 trades, 13.5 percent.
• After 200 trades, 6.8 percent.
• After 500 trades, 1.1 percent.

Read the first line again. One time in four, a genuinely profitable strategy will show you a losing record after twenty trades. If your rule is "give it twenty trades and see", you will throw away good work regularly, and you will never know you did it.

Put the two traps together and you get the cycle I watch traders repeat for years. Test many variants, adopt the one that looks best, run it live for twenty trades, watch it disappoint, discard it, and start testing again. Both ends of that loop are driven by sample size, and neither has anything to do with skill.

What Actually Makes a Test Worth Believing

Four things, and none of them are complicated. They are just tedious, which is why they get skipped.

Count your attempts and write the number down. Before you start, decide how many variants you will try, and record every one you test including the ones you abandon. The count is the single most important number in the whole exercise, and it is the one nobody keeps. A result from three attempts and the identical result from fifty attempts are not the same evidence.

Hold data back before you look at it. Split your history in two, build on the first part, and do not touch the second until the rules are final and written. If the strategy only works on the half it was built from, you have described the past rather than found an edge. This one check catches more self deception than any other, and it costs nothing but discipline.

Judge the size of the sample, not just the result. The table above is your guide. Twenty trades tells you almost nothing in either direction. A hundred starts to be informative. If you cannot generate a few hundred trades in a test, you must hold your conclusion loosely and say so out loud.

Include the costs from the first trade. Every backtest that ignores spread and swap is testing a strategy that does not exist. Costs fall hardest on high frequency rules, which are exactly the rules that produce large sample sizes, so the correction is biggest precisely where you were most confident.

Forward Testing Is the Only Test With Teeth

Everything above is still history. History is cooperative: it sits still, it does not requote you, and you already know how it ends even when you are trying not to.

Forward testing removes that. You write the rules down, then you apply them to bars that have not printed yet, and you record the result whether or not you like it. On a demo account it costs nothing except the thing traders are least willing to spend, which is time.

Two rules make forward testing honest. Write the rules before the test, in full, so there is nothing left to interpret in the moment. And log every trade the rules produced, including the ones you decided to skip, because the trades you skip are where your real strategy hides. A record of what your rules said versus what you did is the most useful document you will ever keep, and it is the foundation of consistency.

When the forward test disagrees with the backtest, believe the forward test. It is the only one that was not built with the answer already visible.

What This Changes About How You Work

The practical shift is small and it changes everything. You stop asking "does this strategy win" and start asking "how much evidence do I actually have, and how many times did I ask".

That reframe kills the search for the perfect setup, because you can now see that the search itself manufactures false positives. It also makes you patient with a strategy in a rough patch, because you know what a real edge looks like over a short sample. And it protects capital, which is the point of all of it: an untested rule set sized properly can survive being wrong, while a well tested one sized carelessly still cannot. Testing sits alongside position sizing, never instead of it.

None of this makes trading certain. It makes you honest about how uncertain it is, which turns out to be the part that keeps accounts alive.

Frequently Asked Questions

A Word on Risk, and How to Use This

Everything above is general education about evidence and probability in leveraged markets. Nothing here is financial advice, and no entry, stop or target discussed should be treated as a signal. Trading gold and other leveraged products carries a high risk of losing money quickly.

Cut to the bone: how to test your trading strategy is really a question about counting. Count your attempts, count your sample, and hold data back before you look at it. Do that and you will discard fewer good strategies and adopt far fewer bad ones.

If you want the risk-first companion to this way of working, it is the Black Gold Market Blueprint, a plain walk-through of protecting an account before trying to grow one. It is free, there is nothing to join, and there is no promised return anywhere in it.

Grab the Blueprint here, and for the foundation underneath all of it, start with how to protect your capital when gold gets volatile.

Protect. Master. Grow.

Risk disclaimer: This article is for educational purposes only and is not financial advice, an offer, or a recommendation to buy or sell any instrument. Trading gold, CFDs and leveraged products carries a high risk of rapid loss. No entry, stop or target discussed should be treated as a signal. External figures are linked to their source, and calculated figures are shown with their assumptions so you can check them yourself.