Backtest accuracy: fees, slippage and honest limits

Backtest accuracy has several layers

Asking whether a backtest is “accurate” hides multiple questions. Separate them so a strong result in one layer does not excuse a weakness in another.

1. Rule accuracy: did the engine implement the strategy graph exactly as specified?
2. Data accuracy: were timestamps, prices, volume, and instrument history suitable for the test?
3. Execution accuracy: could simulated orders plausibly have filled at the assumed prices and sizes?
4. Research validity: were rules frozen before validation, or tuned repeatedly to the same history?
5. Inference quality: does the conclusion stay within what hypothetical history can support?

Fees are necessary, but not sufficient

Trading fees reduce results on every simulated transaction. They matter most when turnover is high or gross edge per trade is small. Use the fee schedule relevant to the venue, product, order type, and assumed account tier, then test a less favorable rate as a sensitivity case.

Fees are only one part of execution cost. A backtest that deducts fees can still be optimistic if it assumes every order fills at a convenient chart price.

Spread: crossing the market has a cost

The spread is the difference between available bid and ask prices. OHLCV candles usually do not preserve that full state. A market buy commonly interacts with the ask, while a market sell interacts with the bid; modelling both at a candle close can overstate performance.

Spread varies by instrument, venue, time, volatility, and liquidity. One constant assumption is better than zero only if its limitation is visible. Sensitivity tests should include wider conditions than the average.

Slippage: intended price is not necessarily fill price

Slippage is the difference between the price a strategy expects and the price at which simulated or actual execution occurs. It can result from price movement during delay, insufficient quantity at the best price, queue position, volatility, or an order large relative to available liquidity.

A single slippage percentage is an approximation, not a universal truth. Test several adverse assumptions and pay special attention to strategies that trade frequently, react to fast moves, or depend on small per-trade differences.

Partial fills and order-state ambiguity

A model may assume an order is either completely filled or not filled. Realistic order lifecycles can be messier: part of the quantity fills, the rest remains open, an order expires, or cancellation races with another fill. That changes exposure, fees, exit size, and subsequent decisions.

Bar data also hides the sequence of prices inside a candle. If a bar touches both a stop and a target, the bar alone may not prove which came first. The simulator needs a documented, conservative rule for ambiguous cases.

Timing errors and look-ahead bias

Look-ahead bias occurs when a decision uses information that was not available at the simulated decision time. Common forms include:

• reading a completed candle and filling earlier within that same candle;
• using an indicator value calculated with future observations;
• selecting assets based on information learned after the test date;
• using revised data that differs from what was originally observable;
• applying an end-of-period ranking before every member’s data exists.

Define signal time and earliest eligible fill time explicitly. Inspect edge trades around transitions, not just ordinary cases.

Data quality and selection bias

Missing bars, duplicated timestamps, wrong time zones, symbol changes, delisted assets, and inconsistent venue data can alter results. A dataset containing only assets that survived to the present can introduce survivorship bias. Choosing the best market after testing many markets is another form of selection.

Keep a record of datasets examined, quality checks performed, and discarded experiments. Without that ledger, the final chart hides how many opportunities the research process had to find a lucky outcome.

Overfitting and parameter cliffs

Every rule, threshold, indicator period, market, and date window adds a choice. Repeatedly selecting the best historical combination can fit noise rather than a repeatable behavior.

Warning signs include:

• a narrow parameter value is profitable while nearby values collapse;
• one small date change reverses the conclusion;
• many exceptions were added after inspecting losing trades;
• performance depends on one asset, one regime, or a few outlier trades;
• the final strategy cannot be explained without referring to the backtest chart.

Freeze the rule graph, reserve out-of-sample data, and prefer stable regions over the single best parameter. No statistical technique can rescue a test whose design kept peeking at its validation set.

How to interpret Koval results today

Koval replays historical OHLCV bars under one written execution contract, with the spread, slippage and commission you configured applied to every fill. It does not model funding, venue fee tiers, partial fills, liquidation or latency. Therefore:

• read net results against the cost assumptions the run states, not against zero;
• inspect every simulated fill, its reference price and its bar context;
• discount high-turnover or small-edge strategies more aggressively;
• do not describe the report as executable, expected, or forecast performance;
• use live paper testing to observe behavior on new market data; sandbox access is currently blocked.

This transparent limitation is more useful than false precision. It tells you which question the benchmark can answer and what evidence is still missing.

How Koval simulates execution

The current fixed-cost backtest uses next-open market entries, protection from the following bar, chosen spread, slippage and commission, and margin equal to notional divided by leverage. Paper sessions using the fixed profile share these assumptions, but matching results also require matching candles, warmup, starting state and ending policy. Legacy profiles can differ.

New archived backtests retain normalized inputs and recorded results. Verify replay creates a separate comparison using matching software and complete evidence. Older or incomplete runs can remain unverifiable. Replay agreement is a consistency check, not a claim about exchange fills.

The app currently blocks exchange sandbox promotion. It does not establish execution parity with a real account. Read what realistic testing means in Koval for the current model, replay requirements and capability matrix.

A conservative review protocol

1. Reproduce the run from the same frozen graph, data scope, and configuration.
2. Verify several entries and exits manually, including ambiguous bars.
3. Separate in-sample design data from untouched out-of-sample data.
4. Review results by regime, asset, and time period rather than only in aggregate.
5. Stress the configured spread, slippage and commission, and estimate the effects the model leaves out.
6. Check neighboring parameters and simpler versions of the same hypothesis.
7. Inspect concentration in a few trades and long underwater periods.
8. Write the conclusion with every material omission beside it.
9. Move to paper trading without changing the rules during the observation window.
10. Where a platform supports it, use a verified exchange sandbox to check adapter behavior. Koval app access remains blocked.

Regulatory warnings are relevant even without real-money execution

Koval itself does not place real-money orders, but clear research language still matters. The U.S. Securities and Exchange Commission’s Investor.gov explains that backtested performance is hypothetical and not actual performance . The U.S. Commodity Futures Trading Commission also warns readers to scrutinise hypothetical trading-system claims and their assumptions .

Those resources do not certify Koval or any strategy. They reinforce the right standard: simulations should be transparent, reproducible, and described without implying future results.

What a credible conclusion sounds like

“Under the frozen rules, selected historical OHLCV data, and the stated spread, slippage, commission and leverage assumptions, this strategy produced the recorded hypothetical behavior. Funding, venue fee tiers, partial fills, liquidation and latency are not modelled. The result requires out-of-sample, paper, and sandbox validation and does not predict future performance.”

That statement is less exciting than a promise. It is also much more useful for deciding what to test next.

For the reviewed execution model, archived replay and live-paper boundaries, read what realistic testing means in Koval.