pub(crate) async fn replay(
dir: &Path,
node: u32,
logs: Ingest<ExportLogsServiceRequest>,
traces: Ingest<ExportTraceServiceRequest>,
metrics: Ingest<ExportMetricsServiceRequest>,
) -> Result<(), Box<dyn Error>>Expand description
Push everything the log holds that no block claims back through the flushers, before the listeners open.
Ordering is the reason this is not a background task. A replayed frame keeps
the sequence it already has, and Ingest::replay puts that sequence back
among the log’s unpublished before it queues the frame, which is what stops
a shard sealing beside it from computing a watermark over it — but that only
covers frames already read off disk. A frame still sitting in a segment is
in nobody’s pending set, so a live export sealing first would claim a
watermark past it and the next boot would not replay it. Draining the whole
log before the first new export is framed is the blunt guarantee that this
never happens. It is also the reason nobody is waiting: the client that sent
this either got its answer before the crash or gave up long before this
process started.
One spawn_blocking for the whole log, not one per frame. The channel is
bounded at ingest.queue and the sends are blocking, so the flushers set the pace and
a multi-gigabyte log is decoded at the rate it can be sealed rather than
into memory all at once.