Benchmarks
The subject is jdx/hk — a mid-size Rust CLI with C dependencies, pinned to a fixed commit — rather than mbx's own workspace, which is too small to measure anything useful against. Every scenario is one CI actually hits.
The build scenarios run the same cargo build --locked through plain Cargo, mbx, and kache, where each tool can make a meaningful comparison. Timings are wall clock around one build. When Cargo appears, it is an uncached baseline measured in that same scenario; cache rows compare only with that result. The warm and worktree scenarios omit Cargo because a fresh target/ or a different checkout gives it nothing to reuse. The contention scenario is the exception, comparing sequential and parallel lint strategies and measuring the machine rather than a single build.
cold
empty store, fresh target -- a first build on a new machine
cargofastestmbxkachewarm
warm store, fresh target -- CI restoring the same commit; plain Cargo has no cache to restore
mbxfastestkachecommit
cache tools warmed at the parent commit, build the child -- Cargo is the uncached push-to-push baseline
cargombxfastestkacheworktree
warm store, second checkout at a different path; plain Cargo cannot reuse the first checkout's target
mbxfastestkachetoolchain
store warmed on the pinned toolchain, rebuild on another
Guard held: of 779 predicted compilations, a different compiler let mbx look up 2 — the ones that do not depend on rustc at all.
contention
six overlapping check, Clippy, and test jobs -- sequential for context, then parallel with and without mbx's machine-wide compiler limit
sequentialcontextparallel · scheduler offparallel controlparallel · mbx scheduledfastestMeasured separately on Namespace Linux xlarge (32 vCPU). View run.
Measured on Linux-6.17.0-1022-azure-x86_64-with-glibc2.39 (GitHub Actions 1000682470) with Rust 1.97.1, building hk at fc29ead under mbx 1.0.0, cargo 1.97.1 (c980f4866 2026-06-30), rustc 1.97.1 (8bab26f4f 2026-07-14), kache 0.16.0. The run that produced them has the per-build logs.
What each scenario reproduces
cold
An empty store and a fresh target/. What a new machine, or a CI job with no cache to restore, actually does. There is nothing to hit, so a cache can only cost time here, and the gap to cargo is the overhead every cold build pays.
warm
The store from the cold build, a fresh target/. This is the common CI shape: a runner restores a cache, then builds a commit it has already seen. cargo is not run, because with a wiped target/ it would just repeat its cold number.
commit
The store is warmed at one commit and the build runs at the next one. Push-to-push CI: most of the dependency graph is unchanged, a few crates are not. cargo's baseline here is a cold build, because that is what cargo does with an empty target/.
worktree
The store is warmed in one checkout, and the timed build runs in a second checkout at a different path. This is the claim that absolute paths did not enter the keys. A cache that keys on paths reports a cold build here.
toolchain
Not a timing. The store is warmed on the pinned Rust and the build reruns on a different one, and the run fails unless essentially none of the predicted compilations were even looked up. A compiler change invalidates every invocation digest at once; the failure mode worth guarding against is a cache that claims a hit anyway. Not zero hits, though: a handful of actions do not depend on rustc — a build script's C object is compiled by the C compiler, which did not change — and those legitimately survive.
It also explains the opposite surprise: a warm build that reports no hits after a runner image picked up a new Rust is not a broken store, it is this.
contention
Six overlapping Rust CI jobs from a cold store: default and all-targets/all- features variants of cargo check, Clippy, and test compilation. The sequential row is context, with the commands sharing one target directory. The parallel rows use separate targets so Cargo's target lock does not serialize them, matching separate CI steps. The apples-to-apples scheduler comparison is between those two parallel rows.
All three rows use the same mbx binary. The mbx and mbx-unscheduled rows run the parallel shape with the machine-wide scheduler on and off, which isolates what mbx contributes from parallelism itself. Cargo bounds only the compilers it starts and knows nothing about the Cargo process beside it; the scheduler gives both processes one machine-wide permit pool and deduplicates identical work in flight. The wall clock shows whether the switch beats the sequential baseline. Peak compilers and lowest free memory show whether it got there by safely sharing the machine or merely oversubscribing it.
How the comparison is kept fair
- The registry is fetched once, before any timed build, into a shared
CARGO_HOME. No cell is timed while it downloads crates. - The toolchain is pinned per subject. hk does not pin one itself, and a runner-image Rust bump changes every invocation digest simultaneously — that reads as a cache that stopped working, and it would land in the series as a step change.
- Each cell gets its own store and
target/, created fresh. No tool ever benefits from another's leftovers. CARGO_INCREMENTAL=0, matching what CI does, and anyRUSTC_WRAPPERinherited from the caller is cleared so the uncached baseline is really uncached.- Both caches run local-only. A remote would measure a network.
- Every scenario's rows come from one CI run, so the tools within it are comparable. A scenario refreshed separately carries its own run link; otherwise the provenance line below the cards applies.
Validity gates
A benchmark that measured nothing still renders numbers, so the run fails and nothing publishes unless:
- the warm and cross-worktree builds report cache hits and restored output files — a fast build that restored nothing was fast for some other reason;
- each warm build beat its own cold build;
- the toolchain-change build loaded predictions and then looked up almost none of them — and that it ran at all, because a guard that was skipped is not a guard that passed;
- the contention run sampled the machine, saw compilers running, and kept the scheduled batch inside its permits — and that the unscheduled batch went past them, because a bound nothing pushed against proves nothing.
A run that fails these is not rendered here at all — the page shows its empty state rather than numbers it cannot stand behind.
Running it yourself
mise run benchThat builds mbx, clones the pinned subject, and runs the cold, warm, and next-commit scenarios. kache is included automatically when it is on PATH and skipped with a note when it is not. mise run bench:refresh is what CI runs: every scenario, writing benchmarks/results.json. It also needs MBX_BENCH_ALTERNATE_TOOLCHAIN set to a second installed Rust, since that is what the compiler-change guard switches to.
The numbers on this page are refreshed by the bench-refresh workflow, which runs weekly, only when the published numbers were measured with an older mbx than the one on main, and opens a pull request rather than publishing directly. Nobody's laptop numbers reach this page.
What this does not measure
Everything hk does not do. hk is a mid-size CLI; it is not Firefox, and a project with a very different dependency shape — heavy proc macros, a large C component, many small leaf crates — will see different ratios. It is also Linux-only: the limits page covers what changes on macOS and Windows.
For instruction-counted measurements of mbx's own startup path, and cold/warm correctness runs against this workspace, see benchmarks/.