How we test
This page describes exactly how TrueTest tests a strategy. Every lab, share page and PDF carries the method version it was made with (this page is methods v2.5, at /methods/v2.5 while it is current; /methods/latest always shows the current version; earlier versions are described in the changelog below; judged with the house gate v2.1 statistics). When the method changes, the version number changes, the change is listed in the changelog below, and older results keep the version they were made with.
A TrueTest result is a backtest of our own rebuild of your rules, or a check of the trade record you sent. It is not a track record, not a promise about the future and not financial advice.
1. What we test
Code, settings or rules in words: we read the files as text (they are never run as given), list faults we notice, and rebuild the strategy as a short list of rules: the rule card. You read the rule card and confirm it, or change it, before anything is tested. A rebuild is our copy of your rules, not your EA; if the rules use something our rule engine cannot express, the lab says not testable by rebuild instead of testing something else. Who writes the rule card is explained on the Security page.
A trade record (an MT5 Strategy Tester report, an MT4/MT5 statement or a CSV of trades): we judge the trades you sent, as they are.
2. The price data
Up to 16 years of 15-minute prices for 8 symbols (7 forex pairs and gold: EURUSD, GBPUSD, USDJPY, GBPJPY, EURJPY, AUDUSD, USDCAD, GOLD), from August 2010 to 18 September 2026, in three separate periods that never overlap:
| Period | From | To | Source |
|---|---|---|---|
| A | August 2010 | December 2017 | Dukascopy 1-minute prices built into 15-minute bars on FxPro server time, with FxPro's recorded spread for each hour of the day |
| B' | January 2018 | 11 September 2022 | Dukascopy, built the same way |
| C | 12 September 2022 | 18 September 2026 | FxPro's own 15-minute bars and spreads |
Also: 21 more FxPro pairs from 12 September 2022, shown for information only (they have no periods A and B', so they never decide a verdict). New prices for the forward watch: new daily Dukascopy prices, built the same way as period A and B'.
3. Costs
Normal costs: the spread from our one cost record (see the v2.5 changelog) + 0.2 pip slippage when the trade opens (gold: 2 US cents an ounce) + our safety margin on costs (0.1 pip on most pairs, 0.3 pip on GBPJPY, 48 US cents an ounce on gold) + GBP 5 a lot commission for the round trip + overnight fees at a dated FxPro table (Wednesday night counts 3 times).
Extra costs (the same trades again, made harder): each trade run again with extra costs: the spread and the commission one and a half times as big, plus 1 pip more when the trade opens (gold: 48 US cents an ounce), and any overnight fee you pay one and a half times as big. A result must stay profitable with extra costs.
Trade records: each trade charged an extra 10 a lot in your account's currency (gold: 20), about 1 pip a lot on a US-dollar account.
Costs are modelled on one broker's recorded spreads; your broker can be cheaper or dearer.
4. The checks
- Code faults: wrong server clock, functions never called, unused settings, missing include files, unchecked orders, martingale sizing and more.
- Costs: profit in each period with normal and with extra costs.
- Luck: the t-value (the t-value of the average trade with trades on the same day grouped together (day-clustered), because trades on one day are not independent) must clear the luck bar: max(1.96, t0(N)), where N is the number of versions you tried (the higher of what you tell us and what we count for your email) and t0(N) is the t-value the best of N no-edge tries is expected to reach. The more versions tried, the higher the bar.
- Years: every calendar year shown, with extra costs.
- Drift control: the same trades with the entry shifted one day later. A real signal should beat it.
- Clock sweep: the rules run with the clock moved -3 to +3 hours, because a wrong broker clock is a common hidden fault.
- Look-ahead probe: the data is cut part-way and the signals re-checked; a signal that changes when later data is removed is using the future.
- Neighbour map: each numeric setting one step either side. A result that only works at one exact value is fragile.
- Hidden risk: grids, martingales, one loss bigger than half the profit, drawdown of half the account or more.
- Parity (when you send your own tester report too): how closely our rebuild's trades match yours.
There is no 0-100 score and no badge: each check is shown separately, failed checks first.
5. The verdict words
| Verdict | What it means |
|---|---|
| REAL | Forward or live trades clear the luck bar after costs. Very likely not luck on this record; never a guarantee. |
| PROMISING | A backtest that clears every check. Run it unchanged on new data before trusting it. |
| LIKELY LUCK | Good, but not good enough for the number of versions tried. |
| NO EVIDENCE | The average profit cannot be told apart from zero. |
| FAIL | Loses money on average, or only wins before realistic costs. |
| HIDDEN RISK | The profit comes with a grid, martingale or outsized-loss shape that can wipe an account. |
| TOO FEW | Not enough trades or days to judge (fewer than 30 trades or 20 trading days). |
A backtest can never be REAL: the best a backtest gets is PROMISING. REAL needs new trades made after the strategy was fixed.
6. The forward watch
The newest version you confirmed is frozen and run every day on new prices from the next utc day; it is never changed, and it is kept with your lab; real needs at least 60 forward trades, a t-value above the luck bar, and beating the same signal shifted one day later by a day-clustered z of at least 1.
7. Receipts
Every tested version gets a receipt: a SHA-256 fingerprint of the rules' fingerprint, the verdict, the time, the method version, the judge version, the price-data version, the cost settings and who wrote the rule card. Anyone shown a receipt can check it here; the check shows what it binds, never the rules. Charts and PDF pages carry the lab id, the receipt and the method version.
8. Known limits
- A rebuild is only as good as the rule card: read it carefully before you start the test.
- Swaps use today's dated broker table for all past years; real past swaps differed.
- Bars are 15 minutes: inside one bar we assume the stop was hit before the target.
- News spikes, gaps and slippage beyond the model can make real results worse.
- The luck bar depends on you telling us honestly how many versions you tried.
- Only the 8 symbols above decide a verdict.
8b. Candlestick patterns
Patterns are read on the closed bar only and never look ahead. Forex trades around the clock, so no gap between bars is required. A bar whose open equals its close has no colour: it can be a doji, never part of a coloured pattern. A bar smaller than a tenth of the previous bar's ATR(14) counts as no pattern (a setting; it can be switched off). The rule card prints the same definition next to every pattern it uses. These are TrueTest's own exact definitions: MT5, TradingView and other tools define some patterns differently, so when your code has its own pattern test we rebuild that test as written and list any difference on the rule card. A pattern is only ever compared with zero (bullish above, bearish below). A rule may multiply a value by a fixed number, for example "the lower wick at least 3 times the body"; TrueTest has no addition, subtraction or division yet.
| Pattern | Exact definition |
|---|---|
| a bullish engulfing / a bearish engulfing | its body covers the previous opposite-coloured body, one side larger; no gap needed |
| a bullish harami / a bearish harami | its body sits inside the previous opposite-coloured body, one side smaller |
| a hammer / bullish pin bar / a shooting star / bearish pin bar | one wick at least 2 x the body and at least 60% of the bar, the other wick at most 20% |
| a doji | body at most 10% of the bar |
| an inside bar | high below and low above the previous bar |
| an outside bar | high above and low below the previous bar |
| a morning star / an evening star | a long bar, a small bar, then an opposite-coloured close strictly beyond the long bar's body midpoint |
| three white soldiers / three black crows | three same-coloured long bars, each closing beyond the last and opening inside its body |
| Body and wick sizes | body, upper wick and lower wick as a % of the bar's range, for your own definitions (for example a marubozu) |
9. Changelog
| Version | Date | What changed | Did verdicts change? |
|---|---|---|---|
| methods v2.5 | 10 November 2026 | One cost record. (1) Our lab had priced the same broker history two ways. From 12 September 2022 to 5 October 2023 the broker's record holds its wider no-commission spread, and our lab charged that AND GBP 5 a lot commission: about 1.3 times (gold) to 9 times (AUDUSD) the spread it charges now, plus the commission. Now every lab uses the same cost record as our own research. From 5 October 2023 it is the broker's recorded raw spread for each 15-minute bar; where the record shows no spread (most GBPUSD and EURUSD bars in 2024) we fill in a typical value for that hour, so those years are partly estimated. Before 5 October 2023, and for all prices from August 2010 to September 2022, we use the broker's typical raw spread for that hour of the day as recorded since October 2023 (a wider one before 2012). On top we add our safety margin on costs (0.1 pip on most pairs, 0.3 pip on GBPJPY, 48 US cents an ounce on gold) and GBP 5 a lot commission. (2) What we checked. For USDJPY, GBPJPY and EURJPY our charge was never below 28 live snapshots of that broker's spread, September to November 2026. As a rough check only, our total gold cost was above another broker's typical gold spread in every year from 2022 to 2026; this does not prove our gold spread is right. For EURUSD, GBPUSD, AUDUSD and USDCAD we have not checked the costs against live prices yet: treat them as an estimate; every result that uses them says so. Of 40 symbol-years from 2022 to 2026, 3 were checked against live prices and 37 use the recorded cost without that check. (3) Gold and silver costs were about a quarter too low before 8 November 2026 (their dollar values were read as pounds); fixed. (4) The check that asks 'did the market just drift up?' and the clock check now use only the symbols that decide the verdict; information-only pairs had been mixed in (8 November 2026). (5) On 1-hour and 4-hour labs we charge the widest 15-minute spread inside the entry hour (on 4-hour gold and GBPJPY labs about a fifth more than the opening spread). (6) Gold overnight fees are now turned into pounds at the same rate as every other cost (1.3395 dollars a pound instead of 1.30). | No stored verdict changed (3 lab versions re-run: FAIL before and after). The demo lab's 2022-2026 line with normal costs moved from -0.004 to +0.007 of the risk per trade; with extra costs it still loses. |
| methods v2.4 (6 November 2026) | 6 November 2026, about 13:30 UTC | Code audit, second pass. The drift control (the same trades one day later, without the signal) now uses the bar at the same time of day on the next day that has it; before, it moved a fixed number of bars, which on gold (a daily trading break) landed at another hour most of the time. A record with a profit that is not a finite number (nan, infinity, or a value too long to be real) is refused with the row named, instead of being judged; a damaged file always gets a plain refusal. A lab whose verdict is turned into FAIL by a period check, or WITHHELD, now shows that verdict's own next step and evidence text in every report. When our code scan finds martingale-style lot sizing, a person checks the rebuild before any test. A time exit written as 24.0 or "24" is kept (it was read as no time limit). A forward-watch trade is counted only once it has really closed. | yes, for gold strategies (the drift control check), and for uploaded records with a non-finite profit (now refused) |
| methods v2.3 (6 November 2026) | 6 November 2026, about 11:30 UTC | Code audit. A target in pips, or in ATR with a different period from the stop, is now sized on each bar from that bar's own stop distance; before, it became one ratio taken from the whole history (later prices moved earlier targets, and the tested target was not exactly the one on the rule card). Rules with a fractional indicator period, an empty session (start = end) or a candlestick pattern on a higher timeframe compared with anything but 0 are refused with a clear reason instead of failing. A trade quantity of 1,000 or more is read as units (100,000 a lot). Broker symbol endings such as XAUUSD.m are ignored when your report is matched with our rebuild. | yes, for rules with a pips target and an ATR stop (or ATR stop and target of different periods), and for gold reports with a suffixed symbol (no longer WITHHELD by mistake) |
| methods v2.2 (6 November 2026) | 6 November 2026, about 09:30 UTC | Version label corrected: the two changes of 6 November about 06:40 and 08:30 UTC were first published as v2.1 although the first one changed verdicts; they are v2.2. The "what to do next" line of every verdict now states the evidence first (for example "on this record it loses money after costs; we would not trade it"). /methods/latest always shows the current version; an old version number now says it was replaced (there is no archive of older texts). | no |
| methods v2.2 (6 November 2026) | 6 November 2026, about 08:30 UTC | A rule may multiply a value by a fixed number (for example "the lower wick at least 3 times the body"). TrueTest has no addition, subtraction or division yet. Existing rules give the same trades. | no |
| methods v2.2 (6 November 2026) | 6 November 2026, about 06:40 UTC | A trade record that real trading cannot produce (every trade the same result, no losing trade in 50 or more, results within 1% of each other, or a profit factor above 5) is judged NO EVIDENCE with a "check the test" reason, never PROMISING, REAL or HIDDEN RISK. Results also show how the verdict would read at 1, 10, 100 and 1,000 versions tried. | yes, for such records only: they now read NO EVIDENCE; before, identical results could read NO EVIDENCE or, through a rounding effect, PROMISING or HIDDEN RISK |
| methods v2.1 | 6 November 2026 (rule effective 5 November 2026, 20:44 UTC) | A rebuild that differs from your code (an approximate rebuild) is labelled as such everywhere and can never become REAL. The rule took effect on 5 November 2026 at 20:44 UTC (when the lab service restarted with it); receipts made from then until 6 November 2026 about 01:10 UTC say methods v2 although the rule already applied to them; receipts made after that say methods v2.1. An audit on 6 November 2026 found no lab affected: every lab version judged so far is FAIL, none is REAL and none is an approximate rebuild. | yes, for approximate rebuilds only: they can no longer reach REAL |
| methods v2.1 (6 November 2026) | 6 November 2026 | Candlestick patterns added to the rule grammar (engulfing, harami, pin bar, doji, inside and outside bar, morning and evening star, three soldiers and crows) and body and wick sizes; exact definitions under Candlestick patterns. Existing rules give the same trades (checked on the demo: all 15,526 trades identical). | no |
| methods v2 (6 November 2026) | 6 November 2026 | Report colours follow the site's colour-blind palette (blue for gains, orange for losses). New labs store the exact wording of the AI consent box (consent wording v2). | no |
| methods v2 (5 November 2026, evening) | 5 November 2026 | The forward band is shown as a review-only historical envelope. | no |
| methods v2 | 5 November 2026 | Normal costs gained the house uplift (forex +1 point, GBPJPY +3 points, gold +48 points on top of the recorded spread) and overnight swaps at a dated FxPro table, with the triple night on Wednesday. Verdicts are shown with and without swaps. | yes: costs went up, so some backtests that passed under v1 would not pass under v2 |
| methods v1 | 5 November 2026 (first public labs) | Rebuild engine, three periods A/B'/C, stressed costs, luck bar, drift control, clock sweep, look-ahead probe, neighbour map, forward watch. | first version |
10. Known incidents
Problems that affected results or the service, listed when we find them.
| When | What it affected | Effect on verdicts | Cause | Fix | Status | Listed |
|---|---|---|---|---|---|---|
| 18 September - 6 November 2026 | the forward watch: no new prices were added; 2 test labs were being watched, no customer labs | none: no forward verdict was due (both watched labs are test labs), and an audit on 6 November 2026 found no REAL verdict ever issued | a software library update changed how times are stored, so every new price bar looked older than the last one and was dropped without an error; the price source also limits fast requests | the time conversion was corrected and tested, and the download now paces itself and retries; the missing weeks are being added | fixed on 6 November 2026; prices catching up | 6 November 2026 |
See a full example: the demo lab and the sample PDF report.