Module pipeline 

The ingest path.

One bounded tokio::sync::mpsc channel per shard feeding one flusher task.

The brief called for a lock-free ring buffer. That is the right structure when items are ~150ns order structs arriving millions per second; here an item is a whole export request costing 10^5–10^6 ns to decode and encode, and the realistic arrival rate is 10^2–10^4 per second. At that ratio the queue is never the bottleneck, and a bounded async channel buys the thing a lock-free queue cannot: send().await applies real backpressure that propagates out as HTTP/2 flow control to the exporter, instead of either spinning or dropping. If a queue ever shows up in a profile, this is one type to change.

Flush is spawn_blocking: it fsyncs.

Where the acknowledgement happens is Config::wal’s decision, and it is the only one in this file. Without a log the export is acknowledged after the block directory rename is durable, because OTLP’s retryable status set covers exports in flight at a crash — acking earlier is the one window where data is lost with the client believing it was stored. That costs a whole max_block_age at the tail, which is section 11’s 2.6 s p99. With a log the frame is the durable record, the publish is a background reorganisation of data that is already safe, and the ack costs a write(2). Everything else here — the queue, the carry, the failure contract — is identical either way.

Structs

Config

Engine configuration.

Flushers

The handle over one signal’s flusher shards: what a single JoinHandle meant when there was one of them, kept true now that there are several.

Ingest

The write handle for one signal. R is that signal’s OTLP export request.

Job 🔒

OpenSlot

Every shard of one signal’s open blocks, asked together.

Rejects

What each signal has refused, published and been stuck on, since start.

Shard 🔒

Where the read path asks one flusher shard for a readable copy of its open block (section 4), and where the last copy it produced is cached.

Enums

Rejected

Why an export could not be admitted. None of these is a partial success: OTLP forbids the client from retrying a partial success, so reporting overload that way permanently destroys the data and blames the sender.

Constants

ADMIT_WAIT 🔒

How long an export waits for room in the queue before it is shed.

MAX_SHARDS

The most flusher shards one signal will run.

MIN_FREE 🔒

Free space below which retention stops waiting for the TTL.

SIGNALS

Every signal that has an on-disk directory. Retention sweeps all of them; block::scan treats a missing one as empty, so listing a signal before its encoder exists is harmless.

UNREADY_AFTER

How long a signal has to be unable to store anything before this node calls itself unready.

WAL_SYNC_PERIOD

How long an acknowledged export can sit in the page cache before it is on the device.

Statics

REJECTS

Parallel to SIGNALS.

Functions

answer 🔒

Answer every reader waiting on the open block, if there is nothing left queued ahead of them (section 4).

dir_bytes 🔒

The size of one block, as the filesystem sees it. Best-effort: a block being unlinked by another replica mid-walk is worth a slightly low counter, not an error path on the flush.

flusher 🔒

now_secs 🔒

Wall-clock seconds. Only ever used for rate limits and for ages an operator reads; block timestamps come from the data, never from this clock.

once_a_second 🔒

True at most once per wall-clock second per gate, for whichever caller gets there first.

reclaim 🔒

Drop the oldest blocks, across every signal, until the volume is back above min_free. Returns what was unlinked, oldest first.

rejects_for 🔒

retention 🔒

shard_count

How many flusher shards to run, given the configured value and what the machine reports.

spawn

Start one signal’s ingest pipeline. Returns the handle its receivers push into. Each signal gets its own channels, flusher tasks and block sequences, so a slow flush on one cannot stall another.

spawn_retention

One sweep for all signals, not one per signal: retention is IO against the directory tree, and three tasks waking on the same minute boundary to unlink from the same volume is contention for nothing.

stall_of 🔒

Seconds r has been unable to store an export, if that is long enough to be worth acting on. Split out from stalled so the threshold is testable without writing to a process-wide static that three live flushers also own.

stalled

The first signal this node has been unable to store for longer than UNREADY_AFTER, and for how many seconds. None means every signal is either healthy or has only just started failing.

wal_maintenance 🔒

Sync the log to the device, and drop the segments every signal has published past.

wal_sweep 🔒

One pass of the above. Separate from the loop so a test can drive both kinds of tick without waiting out the sixty seconds of real time between them.

Type Aliases

OpenSlots

The three signals’ open blocks, in SIGNALS order.