static SPARE: LazyLock<AtomicUsize>Expand description
Threads a search may borrow beyond its own, for the whole process.
Fan-out is only worth anything when there is a core going spare, and this is the measurement that says so. One search over 37 blocks and 19M rows: 1.81 s serial, 0.20 s fanned out. Eight concurrent searches on the same twelve cores: throughput unchanged to within noise, and the short classes’ p99 an order of magnitude worse — the cores were already busy, so every extra thread was scheduler work and nothing else.
So the budget is shared rather than per-search. A search takes what is idle and runs serially when nothing is, which makes the two cases above the same code with no mode to pick and nothing to configure. Bounded by the core count less the caller’s own thread, because that is what “idle” means here.
Deliberately not a fair queue: a search never waits for the budget, it only asks. Waiting would trade the thing being optimised — latency — for a share of a resource the caller is about to finish with anyway.