Now try to break the system
A useful robustness review asks what would make the system fail, then tests those conditions deliberately. Passing a comfortable backtest is not the same as surviving deployment.
Keep an immutable baseline: source version, 20/50 settings, broker data, dates and account assumptions. Each experiment should change a named factor and preserve the other settings where practical. Decide what would count as unacceptable behaviour before seeing the result.
Build a test matrix
| Dimension | Experiment | Inspect |
|---|---|---|
| Costs | Higher spreads, fees and adverse fills | Profit erosion, turnover and margin effects |
| Parameters | Fast 15/20/25; slow 40/50/60 | Neighbourhood stability, not only the best cell |
| Time | Separate trending, choppy and volatile periods | Concentration and prolonged drawdowns |
| Markets | Preselected additional instruments | Transferability under comparable exposure |
| Operations | Restart, rejected order and interrupted connection | State reconciliation and duplicate prevention |
Increase costs without changing the story
Try the documented baseline and, for example, 1.5× and 2× cost scenarios. These are sensitivity assumptions, not calibrated forecasts. Where the tester or custom-symbol setup supports the required change, rerun the simulation and record the mechanism used. If you instead deduct an extra charge from exported deals, label it an approximation.
Apply adverse slippage to both entry and exit. A reversal closes one position and opens another, so count both legs. Convert pips to account currency using the relevant contract specification and volume; do not reuse a EURUSD pip value across unrelated instruments. Avoid counting the same spread or slippage charge twice. For tester execution and data limitations, consult Testing Trading Strategies.
Change markets carefully
The same 0.1 lots can represent very different exposure on another symbol. First document contract size, tick value, currency conversion, margin and trading hours. Then state whether you are comparing the fixed-lot implementation or a separately specified normalised-exposure variant. A change to sizing creates a new experiment, not a silent improvement to the baseline.
Failure on another instrument does not automatically disprove a deliberately market-specific hypothesis. It does challenge any claim of broad applicability. Select the comparison markets before seeing which ones flatter the system.
Test the failures a profit chart cannot show
- Restart with a position open: reconstruct ownership and exposure from account state before acting.
- Close rejected or partly filled: verify that the reverse entry is blocked until the position is reconciled.
- Entry rejected: confirm the failure is logged and no repeated-tick order loop occurs.
- Missing indicator data: wait rather than trading from incomplete buffers.
- Disconnect and reconnect: inspect stale-signal handling and ensure a missed hour does not trigger an accidental replay.
Use controlled tests and demo observations for these cases; a routine historical simulation does not prove how a deployed terminal behaves during an outage. Keep expected and observed behaviour beside each test. A profitable system that cannot identify its current position is not operationally ready.
Treat risk changes as new versions
Our initial reverse-only exit has no predetermined maximum loss. Before any live-use decision, specify sizing limits, margin checks, loss controls, monitoring and a stop procedure. Adding a protective stop changes the strategy and requires a fresh evaluation; it cannot simply inherit the old backtest’s conclusions.
Produce a decision, not a sales chart
The review should end with a specification, implementation record, reproducible reports, experiment ledger and known limitations. Choose among reject, revise and retest, or proceed to controlled demo observation. Record why. No tests or positive performance outcomes are claimed in this article.
That is the skill this series is intended to demonstrate: turning a trading idea into explicit behaviour, evidence and an accountable engineering decision. The same process applies when the strategy is considerably more interesting than two moving averages.
Building or reviewing your own trading system?
I can help turn the specification, implementation and testing into a coherent engineering process. Discuss your project.
