Skip to main content

Module telemetry

Module telemetry 

Source
Expand description

Mira storing Mira’s telemetry, in Mira.

Every counter here already existed and is already served, in one shape, by /api/v1/stats. What that endpoint cannot do is answer “when did the shed rate start climbing”, because it is an instant and an instant has no yesterday. The gap between a node that knows everything about itself right now and a node that remembers is a time series database, and this process is one — so the whole feature is a timer, a translation into OTLP, and a call to the same pipeline::Ingest::submit a collector would have used.

No exporter, no scrape endpoint, no second port. That is the point rather than a shortcut: the reason self-monitoring is normally somebody else’s Prometheus is that the thing being monitored cannot be trusted to store its own data, and the reason it can be trusted here is that when Mira is too broken to store this, the operator’s evidence is /health and the process exit code, which do not depend on it. A node that cannot write its own metrics is a node whose missing metrics are the signal.

It is off by default. See crate::config::Config::self_telemetry for why that is the honest default rather than a timid one.

§The reflexive bit

Storing these rows increments mira.ingest.rows, which is reported in the next sample. That is not a bug to be corrected out: the sampler’s own cost is part of the node’s cost, and subtracting it would make the number disagree with the block directory. At the default interval it is three-figures of rows an hour against a floor of millions, so it is visible in arithmetic and invisible on a chart.

Constants§

SCOPE 🔒
The instrumentation scope every series below is published under, so a reader can separate what Mira says about itself from what was sent to it by a service that happens to also be called mira.

Functions§

gauge 🔒
One instantaneous reading. Not a Sum with is_monotonic: false, which is the other legal spelling: a gauge is what every consumer’s default renderer expects for “the value right now”, and being unusual here buys nothing.
kv 🔒
run
Sample forever, until the metrics pipeline is gone.
sample
Everything this node currently knows about itself, as OTLP.
sum 🔒
One cumulative counter: a value that only goes up for the life of the process, which is what is_monotonic promises a reader.
unix_nanos 🔒