memtrace warmup

Pre-pay the embedding model’s cold-start cost — download, ONNX session init, and optional CoreML compile — before it happens on the clock during start or CI.

Usage#

memtrace warmup [--model <name>]

memtrace warmup runs a one-shot embedding-model warmup so the first real indexing run — or memtrace start — doesn't pay the cold-start cost. It loads the active embedding model, downloading it via fastembed if not already cached, runs a single dummy embedding to force ONNX Runtime session initialization (and the CoreML graph compile, if enabled, on Apple Silicon), and prints wall-clock timing plus the on-disk cache locations.

It is a plain CLI command: it does not start the daemon, open any network port, or touch the code/vector graph or MemDB. No repository indexing happens.

What it pre-pays#

The warmup work is a single call that embeds one deliberately fresh string ("warmup") — chosen so it can't hit the embed cache — forcing the full cold-start chain to run exactly once, on your schedule instead of the user's:

Model download. If the active model isn't already cached, fastembed downloads it to ~/.memtrace/fastembed_cache (or FASTEMBED_CACHE_DIR if set).
ONNX Runtime session init. A pre-flight dlopen probe runs first (probe_ort_runtime(), the same probe memtrace start uses); on success, the fastembed session is created and the first real inference runs through it.
CoreML graph compile — only when MEMTRACE_ENABLE_COREML=1 is set on macOS/Apple Silicon. The compile is documented as taking 60–300 seconds cold.

On success, warmup marks the process warm (an in-process flag, mostly informational for a one-shot CLI) and prints elapsed seconds plus the embedding dimension returned. The durable benefit is the on-disk model/session cache — the next memtrace start hits a warm cache instead of a cold one. It also reports two cache paths: the fastembed model cache with the on-disk size of the largest .onnx file found (best-effort — "size unknown" if unreadable), and the per-repo embed cache directory (~/.memtrace/embed-cache/).

COREML COMPILE IS NOT PART OF THE DEFAULT COST

CoreML/ANE compile on Apple Silicon happens only when MEMTRACE_ENABLE_COREML=1 is explicitly set — it is not part of the default first-run cost, on warmup or on start. Without it, warmup loads the model and runs the first ONNX inference on CPU.

When to use it#

Run memtrace warmup ahead of time whenever the cold-start cost would otherwise land on someone waiting:

Before memtrace start on a cold machine. A fresh install with no cached model pays the download + session-init cost on the first real indexing batch; warming up first moves that cost off the critical path.
In CI or image-build steps. Baking a warm fastembed cache into a container image or CI cache means every job after the first doesn't re-download the model or re-initialize the ONNX session.
Before switching embedding models. Run memtrace warmup --model <name> to pre-pay a candidate model's download and init cost before committing to it with memtrace embed set.
Before enabling CoreML. On Apple Silicon, MEMTRACE_ENABLE_COREML=1 memtrace warmup forces the 60–300s graph compile up front, so the first memtrace start afterward doesn't trip the embed-batch circuit breaker on that spike.

Flags#

Flag	Description
`--model <name>`	Overrides the active embedding model for this run only, by setting `MEMTRACE_EMBED_MODEL` in the current process before resolving/loading the model — the same env-driven resolution path `memtrace start` uses. Not persisted; it only affects this one invocation.

Environment variables#

Variable	Default	Description
`MEMTRACE_EMBED_MODEL`	`(unset — built-in default)`	Read to pick the active embedding model. `--model <name>` sets this var for the current process before warmup runs.
`MEMTRACE_ENABLE_COREML`	`unset (disabled)`	On macOS/Apple Silicon, enables the native CoreML execution provider. When enabled, warmup pays the CoreML graph compile (60–300s cold) so `memtrace start` doesn't trip the embed-batch breaker on that spike later.
`MEMTRACE_DISABLE_COREML`	`unset`	Hard override that forces CoreML off even if `MEMTRACE_ENABLE_COREML` is set — the documented workaround for CoreML compile crashes under memory pressure.
`MEMTRACE_ORT_DYLIB_PATH`	`unset`	Override for the ONNX Runtime dylib path, consulted before warmup's ORT probe runs — mirrors the same resolution `memtrace start` uses.
`ORT_DYLIB_PATH`	`unset`	Standard `ort` crate env var for the ONNX Runtime dylib location; bridged from `MEMTRACE_ORT_DYLIB_PATH` or the bundled dylib next to the binary unless already set (skipped on macOS/aarch64 static-link builds).
`FASTEMBED_CACHE_DIR`	`~/.memtrace/fastembed_cache`	Overrides where downloaded ONNX model files are cached. Warmup only reads this to report the cache path and on-disk model size — it does not set it.
`MEMTRACE_SKIP_EMBED / MEMTRACE_NO_EMBED`	`unset`	If set, embedding generation short-circuits with an error, which warmup reports as a failure and exits 1 — there's nothing to warm. The failure-path hints also suggest `MEMTRACE_SKIP_EMBED=1 memtrace start` as an alternative to running warmup at all.

Gotchas#

AVX2 IS A HARD PRECONDITION ON X86_64

On x86_64 hosts without AVX2 support, warmup refuses to run at all unless you've supplied your own ORT dylib override — the bundled ONNX Runtime build requires AVX2 and would otherwise crash with an illegal-instruction trap. This check happens before any embedding logic runs.

--MODEL IS PROCESS-LOCAL ONLY

--model <name> sets an env var for this process only — it does not persist to any config file or to memtrace embed set's configuration. Running memtrace start afterward without repeating the same env var uses the normal default resolution again.

A TRIPPED CIRCUIT BREAKER BLOCKS WARMUP TOO

The embed circuit breaker is consulted inside the same embedding call warmup uses. If it was previously tripped — e.g. from a prior failed run in the same daemon/process context — warmup can fail immediately without attempting to reload the model. This is a shared safety mechanism, not something specific to warmup.

On failure, warmup prints elapsed time, the error, and a fixed list of common causes: missing libonnxruntime (set MEMTRACE_ORT_DYLIB_PATH), a CoreML compile killed by memory pressure (unset MEMTRACE_ENABLE_COREML or set MEMTRACE_DISABLE_COREML=1), or a model download blocked by network access to huggingface.co. It then exits 1.

Reported fastembed cache sizes are best-effort: if the cache directory tree can't be fully read, the report just shows "size unknown" rather than erroring the whole command. On macOS/aarch64 with CoreML enabled, warmup also prints an informational pointer to ~/Library/Caches/com.apple.coremlcompiler/ — that cache is OS-managed, not Memtrace's own, so clearing Memtrace's caches won't clear it.

Examples#

terminal

$ memtrace warmup
# Warm the currently-configured default embedding model;
# prints timing and cache paths, exits 0 on success.

$ memtrace warmup --model bge-small
# Warm a specific model for this run only (sets MEMTRACE_EMBED_MODEL
# in-process, not persisted) — useful before switching models to
# pre-pay the download/compile cost.

$ MEMTRACE_ENABLE_COREML=1 memtrace warmup
# On Apple Silicon, also forces the CoreML graph compile up front
# (can take 60-300s cold) so the first `memtrace start` doesn't trip
# the embed-batch circuit breaker on that spike.

$ MEMTRACE_ORT_DYLIB_PATH=/usr/local/lib/libonnxruntime.dylib memtrace warmup
# Point warmup at a manually-installed ONNX Runtime dylib
# if the bundled one fails the pre-flight probe.

See also memtrace embed to switch models or providers, memtrace start for the boot sequence warmup pre-pays, and Embedders configuration.