Skip to main content

Module query

Module query 

Source
Expand description

The read path: prune blocks, scan columns, materialize JSON.

There is no query planner and no expression tree here, and that is a decision rather than an omission. Mira answers a small, known set of questions — “the last N records matching these filters”, “every span of this trace”, “this metric bucketed over time” — and each is a hand-written scan over a layout designed for it. A general planner spends its budget rediscovering at runtime what this module knows at compile time: which column holds the timestamp, that parent_id is a row index, that block directory names already carry the pruning key. That is most of what the 151 transitive crates of a general engine buy.

Three properties the layout hands the scan, in the order they matter:

  • Blocks prune by name. <min_ts>-<max_ts>-<node>-<seq>-<wal_hi> is the whole index. A time-bounded query opens no file it will not read, and with blocks visited newest-first a limit stops the scan early.
  • parent_id is a row index. Ids are rebased dense per block at ingest, so attaching an attribute to its record is an array store, not a hash join.
  • Dictionary columns compare as u16. severity_text = "ERROR" resolves the string once against the dictionary, then scans a buffer of 16-bit codes and never touches string data again.

Everything runs against mmap’d buffers, so a cold block costs demand paging and a warm one costs memory bandwidth. Nothing allocates per row until materialization, which happens after limit has cut the result to size.

With no write-ahead log, nothing here needs to read the open, unsealed block: ingest acknowledges an export only after the block containing it has been fsynced and renamed into place, so read-your-writes falls out of the durability rule. Turn the log on and the acknowledgement moves ahead of the seal, which breaks that — so search_open takes the open block’s snapshot alongside the directory scan and merges the two into one ordered page. See crate::signal::Open; the argument is ARCHITECTURE section 4.

Structs§

AttrPred 🔒
One attribute term with everything that does not depend on the row resolved once: the four value columns downcast, the query scalar canonicalized, and — for the string column — the predicate already evaluated against the dictionary.
AttrProbe 🔒
The attribute equalities in a search, in the form a block filter answers.
Attrs 🔒
An attribute table, plus the one fact about it that makes rendering a row cheap: whether parent_id ascends.
Block 🔒
The tables of one block, opened and ready to scan.
Child 🔒
A child table and the attribute table keyed by its id.
Cursor
Where the previous page stopped.
Helpers 🔒
Threads borrowed from SPARE, returned on drop — including the drop that unwinds a panicking scan. Leaking one would degrade the process to serial scans permanently, with nothing to point at.
Hit 🔒
One matching record, kept only long enough to sort and materialize.
Results
Scan 🔒
Everything the per-block half of a search reads, and nothing it writes.
Search
A record search: filter, order by time descending, take limit.
Stats
What a search cost, reported alongside the rows.
Term
One conjunct. Terms are AND-ed. There is no OR in V0: a disjunction over attributes is rare enough that supporting it means building a planner for a query nobody has typed yet.

Enums§

Op
Comparison operator. Contains is substring matching on strings and matches nothing on any other type: a filter that cannot apply returns no rows rather than an error, because a query spanning signals with slightly different columns is a normal thing for an agent to try.
Signal
Which signal, and therefore which tables and which time column.
Target
What a term filters on.
Value
A scalar from a query document.

Constants§

MAX_FANOUT 🔒
The widest a single search will fan out across blocks.

Statics§

SPARE 🔒
Threads a search may borrow beyond its own, for the whole process.

Functions§

attr_key 🔒
attr_parents 🔒
parent_ids of the attribute rows whose key matches and whose value satisfies op value.
attr_probes 🔒
canon 🔒
The text an attribute of this value would have been indexed under.
child_tables 🔒
The child tables of each signal: emitted name, row table, attribute table.
cursor 🔒
The sort key of one hit, which is also the cursor a caller pages on.
dict_index 🔒
Position of needle in a dictionary’s value array.
emit_any 🔒
Render a protobuf-encoded AnyValue — the attribute ser column, and a log body that was not a string — as the JSON it describes.
emit_any_value 🔒
Recursion is bounded by prost’s own decode recursion limit (100), which AnyValue::decode above has already enforced on these bytes — a hostile client cannot nest deeply enough here to reach the stack.
emit_attr 🔒
Emit one attribute’s value from whichever column its type names.
emit_attrs 🔒
One {...} merging the attributes of several levels, most specific last.
emit_fields 🔒
Every non-null column of one row, by name.
emit_value 🔒
Emit one cell of a root table.
field_filter 🔒
Narrow sel to the rows of col satisfying op value.
index_parent_of_id 🔒
The parent_id of each child row, indexed by that row’s own id.
keep 🔒
Narrow sel to the rows where p(vals[row]) holds.
keep_by 🔒
keep for a column with no values slice to walk: a bitmap, a variable offset array or a fixed stride, none of which a vector register helps with. The win here is only the closure being monomorphic rather than boxed.
range_probes 🔒
The comparisons in a search that a zone map can answer.
search
Run a search against the blocks under root.
search_open
As search, but also reads open — snapshots of blocks that have been acknowledged and not yet published.
trace_needle 🔒
The trace id this search pins down exactly, if it pins one down.
unhex
Decode hex, either case. None on an odd length or any non-hex byte, so a malformed trace id matches nothing rather than matching a truncated prefix.

Type Aliases§

Scanned 🔒
What one block contributed: the mapping, if it was opened, and its matches.