Backtest vs Live Trading: Reconciling Every Trade
Why live trading drifts from the backtest: fills, latency, data gaps, rolls and code divergence. How to replay your live engine and reconcile trade by trade.
Part 19 of 20 · How to Algo · Previous: Combining strategies · Next: Running algos in production
A strategy is not validated when its backtest passes. It is validated when its live record matches what the backtest says it should have done on the same days. Until then you do not know whether you are trading the strategy you tested.
Most of the bugs in this series were found in that gap: between a research backtest and a live agent, between an archived tape and a live feed, between the price a record booked and the price that traded. This part is about building the machinery to see the gap and reading what it shows.

Four sources of drift
When live results diverge from a backtest, the cause is one of four things:
- Implementation. The live code is not the strategy the backtest tested.
- Fills. Live fills differ from the backtest’s fill assumptions.
- Data. The live agent saw different data than the archive contains: gaps, delays (what latency is realistic), a different contract, a different clock.
- Market. The strategy works as designed and the market was unfavourable.
The simulated example at the top shows why the matching has to be trade by trade: over 320 trades the backtest made +9.7k and the live record +0.8k, and 9 trades explained 67% of the gap. The totals alone would have said “the edge decayed”.
Only the fourth is information about the edge. The first three are engineering problems. Reconciliation exists to separate them, and the order matters: you cannot judge the fourth until you have ruled out the first three.
Replay the production code, not the research script
The most useful tool we built is a replay that streams archived ticks continuously through the actual production agent classes, the same code that runs live, and writes the trades they would have made. Not a research backtest that is supposed to be equivalent: the live code itself.
Three details made it work:
- Stream continuously, not day by day. Agents that keep state across the overnight session lose it if the replay restarts at each day boundary.
- Gate on coverage. A replayed session counts only if the regular session is complete (at most one missing minute, no gap over 60 seconds) and the tape reaches the agent’s own flat time. One of our agents flattens at 17:00 New York; if the tape ends at 16:59, the replay silently loses the open position.
- Prove the tool before using it. On clean sessions after deployment, the replay reproduced the live trades to the cent: 11 of 16 sessions for one agent, 10 of 12 for another. The differences were inspectable.
What it found
We rebuilt the live paper record of all our sleeves for June and July 2026 from archived ticks. The book went from +$5,145 to −$65,725. Two independent errors made up most of the difference.
Fills at the level. One agent booked breakout entries at the breakout level, including on mornings when the first tick of the session was already far beyond it. The fill correction alone took its record from +$61,266 to +$19,873 (the full replay in the chart below ends at +$19.6k); single entries had been booked up to 470.75 points better than anything that traded. Other agents had the same pattern only on stop exits, where it was worth a few dollars a trade. Part 7 gives the fill rule we now enforce everywhere.
Two different kinds of hole. On some days the live feed connection had died silently: the record showed zero trades while the tape showed a full session. On other days the recorder had died while the agents kept trading: the record had trades the archive could not confirm. Neither source was complete on its own. Part 20 covers the watchdogs we added.
The replay also explained small mysteries. On one day, a live agent’s opening-range high was one tick below the archived one. It bought a breakout that, according to the archive, never happened. Live data and archived data are not guaranteed to be identical, and a strategy that triggers on exact levels will notice.

When the live agent is a different strategy
The most consequential reconciliation result was not a fill. One of our paper sleeves had a backtest win rate of 59.7%. Its live record over its first months won 51.3%. Too big a gap to be noise, too small to be an obvious failure.
The cause was in the code: the backtest committed to the oldest waiting price level and waited for price to come back to it; the live agent overwrote its single level slot on every qualifying bar close, so it effectively entered at once instead of waiting. Swapping only that piece of logic in the backtest reproduced the live behaviour: 50.6% wins and a loss over seven years instead of a large gain. The live record matched the live semantics, not the backtest. Part 6 has the table.
The rule we drew from it: a backtest number that no single-pass agent can reproduce is a bug. Write the agent, replay it, and compare before you trust either.
When the market did it
Reconciliation also clears strategies. One intraday agent lost $14.1k in a week of real live trades. The replay of the same week through the same code on recorded ticks lost on the same days in similar amounts. No bug. The week was a sharp V-shaped reversal on NQ, the worst possible tape for that kind of strategy, and it was worse than the worst week in its seven-year backtest. That is a tail event to note against the strategy’s risk budget (Part 20), not a reason to change the code.
Rules for touching a track record
Rebuilding a record is powerful and dangerous. Two lessons came from our own mistakes.
Prove you can rebuild a correct record before correcting a wrong one. Our first repair tool estimated corrections of −$139k and −$189k for two agents. The tool was broken: its stop search had no time reference and scanned the whole session from the open, including the minutes before the entry. A mandatory check, rebuilding records we believed were right to the cent first, caught it twice before anything was written. The rule: if you cannot reproduce a record you think is correct, you are not allowed to correct one you think is wrong.
Read the intent before deleting anything. During the rebuild we removed rows from before each agent’s deployment date, assuming they were accidentally published backtest seeds. They were intentional, documented backfills starting from a fixed date, stated in the launch commits. We restored them and now mark every pre-deployment row explicitly as backfill, produced by the replay over real prices rather than copied from a research file.
Paper trading has a job, and it is not proving the edge
A few weeks of paper trading cannot confirm a strategy’s edge: Part 10 shows how many trades that takes. Paper trading proves something else: that the live system, with its data feed, clock, order logic and fill handling, does what the backtest did. Run it until a meaningful number of trades reconcile, and treat every unreconciled trade as an open bug.
Our execution engine was tested the same way on a paper account before any real account was armed, and it found two real bugs: a missing order-routing subscription that made every order fail, and a reconciliation step that assumed a flatten had succeeded when it had not, leaving the account long two contracts while the engine believed it held one.
Checklist
- Run the production agent code over archived ticks; do not rely on an equivalent research script.
- Replay continuously across session boundaries.
- Gate replayed sessions on data coverage up to the agent’s own flat time.
- Prove the replay reproduces clean live sessions to the cent before using it to correct anything.
- Match every live trade to its replayed twin; classify differences by cause.
- Compare backtest and live win rates and average trades early; a persistent gap is a bug until shown otherwise.
- Label backfilled rows; produce them with the same code over real prices.
- Treat paper trading as a systems test, and run it until trades reconcile.
Once live and backtest agree, the remaining risk is operational: feeds that die quietly, rolls, restarts, and knowing when to stop. Part 20: Running trading algorithms in production, and the kill switch.
Frequently asked questions
Why are my live trading results worse than my backtest?
For four kinds of reasons: the live code implements a different strategy than the backtest, fills differ from the assumed ones, the live data differs from the archived data, or the market is in a regime the backtest saw rarely. Only the last one is about the market. The first three are engineering, and reconciliation is how you find them.
How do you reconcile a backtest with live trades?
Replay the exact market data the live agent saw through the production code and match every live trade to a simulated one: same entry time, same price, same exit. Prove the replay first on clean sessions; ours reproduced live trades to the cent on 11 of 16 sessions for one agent and 10 of 12 for another. Every remaining difference is a bug in one of the two until shown otherwise.
Is paper trading a reliable test of an algorithm?
It is a systems test, not an edge test. A few weeks cannot confirm an edge, but they can prove that the live feed, clock, order logic and fill handling do what the backtest did. Run it until a meaningful number of trades reconcile: weeks for an intraday strategy, months for a daily one.
How much worse than the backtest should live results be?
There is no safe haircut to assume; a persistent gap is a bug until shown otherwise. One of our sleeves won 59.7% of trades in its backtest and 51.3% live. The cause was code: the live agent overwrote its waiting level every bar, and the backtest with that logic won 50.6%.
Can a paper record be backfilled from a backtest?
Only if it is clearly labelled, and only if the backfill is produced by the same code over real recorded prices rather than copied from a research file. Rows from before a strategy went live should be marked as such. A replay through production code over archived ticks is far more credible than a backtest CSV.