Backfill

A backfill loads historical data into a dataset, whether because a source is newly added, a schema changed, or a gap needs repairing. It is what gives a real-time stream a usable past, so a consumer can backtest a signal on years of history rather than waiting for the live stream to accumulate it.

The trap is honesty about time. It is easy to backfill with today's revised, cleaned view of the past and hard to backfill the way the data actually looked at each historical moment - but only the latter is safe for a backtest, because the former smuggles in look-ahead. A rigorous backfill reconstructs point-in-time history, recording what was known when, rather than painting the past with present knowledge.