Struct Wal 

pub struct Wal {
    inner: Mutex<Inner>,
    dir: PathBuf,
    node: u32,
}

An append-only log of OTLP export bodies.

Every method is synchronous and at least one of them issues a syscall that can block under writeback pressure, so callers must not invoke these from a tokio runtime worker — section 5’s rule that blocking work goes through spawn_blocking applies here for the same reason it applies to publish.

Fields

inner: Mutex<Inner>

dir: PathBuf

node: u32

Implementations

impl Wal

pub fn open(root: &Path, node: u32) -> Result<Self>

Open (or create) the log under <root>/.wal/, resuming the sequence counter past anything already on disk.

Resuming from the files rather than from the block watermarks is deliberate: a sequence that went backwards would let a replay confuse a new frame for one a block already covers, and the watermark comparison is >, so the failure would be silent data loss rather than an error.

pub fn append(&self, signal: Signal, body: &[u8]) -> Result<u64>

Append one OTLP export body and return the sequence it was given.

Returns once the bytes are in the page cache. This is the call the acknowledgement waits on, and it does not fsync — see the module docs for exactly what that does and does not survive.

pub fn append_then( &self, signal: Signal, body: &[u8], then: impl FnOnce(u64), ) -> Result<u64>

append, running then on the new sequence before the log’s lock is released.

This exists to make one specific race impossible, and it is not a general-purpose hook.

A block’s watermark is watermark_for’s answer over the sequences that block holds, and that call binary-searches them, so a shard’s list of them has to be ascending. Two exporters calling append concurrently get their sequences in lock order but can be preempted between the return and the enqueue, which would land 6 in a shard’s list ahead of 5. A binary search over an unsorted list is wrong in both directions and one of them is a false hit: the watermark steps over a frame no block holds, and the next replay skips it. That is silent loss, the one failure this log exists to prevent.

So the enqueue happens under the same lock. It costs nothing: the queue hand-off is a pointer move into a reserved slot, next to a write(2) that has already been paid for.

then therefore must not block, must not .await, and must not touch this log — re-entering append from inside it deadlocks.

pub fn reframed(&self, signal: Signal, seq: u64)

Put a sequence this log already handed out back among the unpublished — the replay path, where the frame is read off disk rather than appended.

Without it a recovered frame is invisible to watermark_for, so a block sealed beside it could claim a watermark that steps over it and the next boot would not replay it a second time. That is the one failure this log exists to prevent, and replay is exactly when it would bite: the frames in flight are the ones a crash already nearly lost.

fn lock(&self) -> MutexGuard<'_, Inner>

A panic cannot leave the log’s state inconsistent — every critical section is a write and a counter — so poisoning carries no information.

pub fn watermark_for(&self, signal: Signal, seqs: &[u64]) -> u64

The watermark a block holding seqs may publish: the oldest sequence of this signal that nobody has published, or, if this block is the last of them, one past everything the log has handed out.

seqs must be sorted. Called before the publish, because the answer goes in the directory name; the sequences stay unpublished until published says otherwise, so a sibling shard sealing at the same moment still counts this block’s frames against its own watermark and neither can claim the other’s.

It can be too low — a shard holding an old frame pins every sibling’s watermark behind it — and that is the direction it is allowed to be wrong in: too low re-ingests a frame a block already has, too high loses it. Nothing is pinned for long, because the shard holding the old frame is at most max_block_age from sealing it.

pub fn published(&self, signal: Signal, seqs: &[u64])

Retire the sequences a block has durably published.

Only on success. A block that failed to land leaves its frames here, and that is what keeps the next block’s watermark from claiming them.

pub fn sync(&self) -> Result<()>

Force everything appended so far onto the device.

Called on a timer by a background task to bound power-loss exposure, never from the acknowledgement path. On Apple targets this is F_FULLFSYNC and costs about 4 ms, which is the whole reason it is not on the ack path.

pub fn next_seq(&self) -> u64

The sequence that will be handed to the next append.

pub fn truncate(&self, covered: u64) -> Result<usize>

Delete whole segments whose every frame is below covered, the smallest of the per-signal Watermarks — same exclusive convention.

Returns how many segments were removed. Deletion is per segment rather than per frame because a log is only append-only if nothing ever rewrites its middle; reclaiming a prefix by truncation would mean rewriting offsets that a concurrent reader is part-way through.

The current segment is never removed, whatever its watermark, because append holds it open and unlinking it would leave writes going to a file with no name.

pub fn replay( root: &Path, node: u32, watermarks: Watermarks, f: impl FnMut(Signal, u64, &[u8]) -> Result<()>, ) -> Result<Replayed>

Replay every frame not yet covered by a published block, oldest first.

watermarks is the exclusive watermark per signal, taken from the block directory listing: every sequence below it is inside a published block, which is not the same as one past the last sequence published, because shards publish out of order. A frame is handed to f only if its sequence is at or above its own signal’s watermark, so a block that sealed while another signal’s was still open does not cause a re-ingest.

A torn or corrupt frame ends the replay of that segment rather than failing it: the tail of the last segment is exactly where a crash leaves a half-written frame, and refusing to start because the last write was interrupted would turn a normal crash into an outage. Frames before the tear are complete and are replayed.

f is handed the frame’s own sequence, not a fresh one. Re-appending a replayed frame would give it a number above every watermark, so the block that stored it would claim the new sequence and leave the old one uncovered — and the next boot would replay it again, forever. Carrying the original through to the block is what makes replay converge.

fn segments(dir: &Path, node: u32) -> Result<Vec<(PathBuf, u64)>>

Segments for this node, oldest first.

Ordered by the first sequence in the name rather than by mtime, because mtime has a one-second resolution on some filesystems and two segments can share it. Other nodes’ segments are skipped: a shared volume (section 12) has one log per replica and replaying another’s would double-write its data.

fn roll(&self, inner: &mut Inner) -> Result<()>

Close the current segment and start the next one.

Nothing is forced here — the outgoing segment goes on retired for the background Wal::sync to deal with. See that field for the measured reason. Opening the new file is two syscalls and does not touch the device, so the appender that happens to trigger a roll pays microseconds rather than milliseconds.