What over 11,000 strategy tests taught us
Since September 2026 we have run over 11,000 strategy tests (to 10 November 2026) of 464 trading ideas on up to 16 years of prices (the main eight symbols have all of it, about 4 years for the rest), with a broker's real costs. We count every variant, because each extra try raises the bar a real edge has to clear. A strategy test is one setting on one symbol; a judged test is an idea run through our whole method. These are counts from our own tests, not a forecast for your EA.
1. Costs decide most results
Of the 381 failed judged tests whose reasons our learning record lists (measured on 7 November 2026 over 10,864 tests), 86% had costs among the reasons: the edge did not survive realistic, higher costs (a test can have several reasons, so the shares below add up to more than 100%). Our October 2026 cost study used a rule of thumb from 340 tests: edges before costs are rarely above +0.10R a trade, so a trade whose costs take 0.10R of the stop needs a better-than-usual edge just to break even. Check yours below.
2. In our tests, short scalps were the extreme case
We tested 8,506 short-timeframe scalping settings and 48 published scalping bots on real prices with costs. Before costs the best settings made about 0.3 pip a trade; a round trip on EURUSD cost about 1.1 pip in that search (the calculator below uses our later cost model, which is a little higher). 0 of 3,912 indicator and time-anchored settings made money after costs on the data set we ranked them on. Of 4,594 time-of-day windows, one held up, and it was too weak across the whole data set to use. All 48 bots lost money in our test once costs were included.
3. Beat simply being in the market, not zero
46% of those failed tests did no better than simply being in the market, or entering at random, at the same times. A strategy that only makes money because the market moved while it was in a trade has no timing edge.
4. Could it be luck?
55% of them could not be told apart from luck after this many tries. Our luck bar calculator shows how good a backtest has to be once you count your own tries.
Trading-cost calculator
What share of your stop does one round trip cost? This runs in your browser: nothing you type is sent anywhere.
Limits
- These are counts from our own research, on our own data and cost model; another data set or broker can give different numbers.
- A failed idea in our tests says nothing certain about a different EA. Not financial advice.