Expand description
The read path: prune blocks, scan columns, materialize JSON.
There is no query planner and no expression tree here, and that is a
decision rather than an omission. Mira answers a small, known set of
questions — “the last N records matching these filters”, “every span of this
trace”, “this metric bucketed over time” — and each is a hand-written scan
over a layout designed for it. A general planner spends its budget
rediscovering at runtime what this module knows at compile time: which
column holds the timestamp, that parent_id is a row index, that block
directory names already carry the pruning key. That is most of what the 151
transitive crates of a general engine buy.
Three properties the layout hands the scan, in the order they matter:
- Blocks prune by name.
<min_ts>-<max_ts>-<node>-<seq>-<wal_hi>is the whole index. A time-bounded query opens no file it will not read, and with blocks visited newest-first alimitstops the scan early. parent_idis a row index. Ids are rebased dense per block at ingest, so attaching an attribute to its record is an array store, not a hash join.- Dictionary columns compare as
u16.severity_text = "ERROR"resolves the string once against the dictionary, then scans a buffer of 16-bit codes and never touches string data again.
Everything runs against mmap’d buffers, so a cold block costs demand paging
and a warm one costs memory bandwidth. Nothing allocates per row until
materialization, which happens after limit has cut the result to size.
With no write-ahead log, nothing here needs to read the open, unsealed
block: ingest acknowledges an export only after the block containing it has
been fsynced and renamed into place, so read-your-writes falls out of the
durability rule. Turn the log on and the acknowledgement moves ahead of the
seal, which breaks that — so search_open takes the open block’s snapshot
alongside the directory scan and merges the two into one ordered page. See
crate::signal::Open; the argument is ARCHITECTURE section 4.
Structs§
- Attr
Pred 🔒 - One attribute term with everything that does not depend on the row resolved once: the four value columns downcast, the query scalar canonicalized, and — for the string column — the predicate already evaluated against the dictionary.
- Attr
Probe 🔒 - The attribute equalities in a search, in the form a block filter answers.
- Attrs 🔒
- An attribute table, plus the one fact about it that makes rendering a row
cheap: whether
parent_idascends. - Block 🔒
- The tables of one block, opened and ready to scan.
- Child 🔒
- A child table and the attribute table keyed by its
id. - Cursor
- Where the previous page stopped.
- Helpers 🔒
- Threads borrowed from
SPARE, returned on drop — including the drop that unwinds a panicking scan. Leaking one would degrade the process to serial scans permanently, with nothing to point at. - Hit 🔒
- One matching record, kept only long enough to sort and materialize.
- Results
- Scan 🔒
- Everything the per-block half of a search reads, and nothing it writes.
- Search
- A record search: filter, order by time descending, take
limit. - Stats
- What a search cost, reported alongside the rows.
- Term
- One conjunct. Terms are AND-ed. There is no OR in V0: a disjunction over attributes is rare enough that supporting it means building a planner for a query nobody has typed yet.
Enums§
- Op
- Comparison operator.
Containsis substring matching on strings and matches nothing on any other type: a filter that cannot apply returns no rows rather than an error, because a query spanning signals with slightly different columns is a normal thing for an agent to try. - Signal
- Which signal, and therefore which tables and which time column.
- Target
- What a term filters on.
- Value
- A scalar from a query document.
Constants§
- MAX_
FANOUT 🔒 - The widest a single search will fan out across blocks.
Statics§
- SPARE 🔒
- Threads a search may borrow beyond its own, for the whole process.
Functions§
- attr_
key 🔒 - attr_
parents 🔒 parent_ids of the attribute rows whose key matches and whose value satisfiesop value.- attr_
probes 🔒 - canon 🔒
- The text an attribute of this value would have been indexed under.
- child_
tables 🔒 - The child tables of each signal: emitted name, row table, attribute table.
- cursor 🔒
- The sort key of one hit, which is also the cursor a caller pages on.
- dict_
index 🔒 - Position of
needlein a dictionary’s value array. - emit_
any 🔒 - Render a protobuf-encoded
AnyValue— the attributesercolumn, and a log body that was not a string — as the JSON it describes. - emit_
any_ 🔒value - Recursion is bounded by prost’s own decode recursion limit (100), which
AnyValue::decodeabove has already enforced on these bytes — a hostile client cannot nest deeply enough here to reach the stack. - emit_
attr 🔒 - Emit one attribute’s value from whichever column its
typenames. - emit_
attrs 🔒 - One
{...}merging the attributes of several levels, most specific last. - emit_
fields 🔒 - Every non-null column of one row, by name.
- emit_
value 🔒 - Emit one cell of a root table.
- field_
filter 🔒 - Narrow
selto the rows ofcolsatisfyingop value. - index_
parent_ 🔒of_ id - The
parent_idof each child row, indexed by that row’s ownid. - keep 🔒
- Narrow
selto the rows wherep(vals[row])holds. - keep_by 🔒
keepfor a column with no values slice to walk: a bitmap, a variable offset array or a fixed stride, none of which a vector register helps with. The win here is only the closure being monomorphic rather than boxed.- range_
probes 🔒 - The comparisons in a search that a zone map can answer.
- search
- Run a search against the blocks under
root. - search_
open - As
search, but also readsopen— snapshots of blocks that have been acknowledged and not yet published. - trace_
needle 🔒 - The trace id this search pins down exactly, if it pins one down.
- unhex
- Decode hex, either case.
Noneon an odd length or any non-hex byte, so a malformed trace id matches nothing rather than matching a truncated prefix.
Type Aliases§
- Scanned 🔒
- What one block contributed: the mapping, if it was opened, and its matches.