memtrace gpu

Optional Windows NVIDIA CUDA acceleration. Memtrace runs without this command; install it only when you want the CUDA provider instead of DirectML or CPU fallback.

Usage#

memtrace gpu manages the optional CUDA execution provider for Windows NVIDIA hosts. Since v0.8.0, the ~400MB onnxruntime_providers_cuda.dll is no longer bundled in the @memtrace/win32-x64 npm package — it exceeded npm's payload limit. Instead it ships as a GitHub release asset that the npm postinstall fetches automatically on NVIDIA machines.

This command is the manual escape hatch: offline installs, GPUs added after install, detection misses, or forced re-downloads. The engine always probes CUDA → DirectML → CPU at startup, so the CUDA provider is an accelerator, never a requirement.

A bare memtrace gpu (or --help/-h) prints the help text and exits 0. An unknown argument prints an error plus the help text and exits 2.

install-cuda#

Downloads memtrace-windows-x86_64-cuda-ep.zip from the matching GitHub release on syncable-dev/memtrace-public, verifies its sha256 against the release's checksums.txt, extracts onnxruntime_providers_cuda.dll, and places it next to the running memtrace binary.

Three guard rails run before any network I/O, all overridable with --force:

• The DLL is already present → "nothing to do", exit 0.
• The binary is running from a *-noavx2 npm platform package → hard error, since Microsoft's prebuilt CUDA provider only pairs with the default AVX2 ONNX Runtime.
• No NVIDIA GPU is detected → skip the ~400MB download, exit 0.

checksums.txt (~1KB) is fetched first, deliberately doubling as an offline / release-exists probe so the 400MB transfer never starts if it can't be verified. The zip is streamed to a dot-prefixed staging file in the binary's own directory, sha256-hashed while writing, with progress printed roughly every 50MB. A sha256 mismatch aborts with an error and deletes the staged zip; on match, the DLL is extracted from the zip into a second staging file and the zip is deleted.

If an old DLL exists — the --force re-download path — it's removed before the rename. On success, the command prints the installed path and notes the engine picks up the provider on the next start; no restart is performed automatically. Without the DLL, NVIDIA hosts silently run embeddings on DirectML.

status#

Shows whether the CUDA provider DLL is installed next to the binary (with path and size), whether the NVIDIA GPU probes detect a GPU, and the engine's startup provider probe order (CUDA → DirectML → CPU). Suggests memtrace gpu install-cuda (~400MB) when the DLL is missing. On non-Windows platforms it prints that there's nothing to manage.

NVIDIA detection is best-effort and non-fatal: first nvidia-smi -L (looking for "GPU " in stdout), then a PowerShell (Get-CimInstance Win32_VideoController).Name fallback for PATH-less driver installs (case-insensitive match for "nvidia"). Any detection error counts as "not detected" — --force is the documented override.

Provider fallback order#

No prompts, no ports, no license/tier gating anywhere in this command — the only network access is the two HTTPS GETs to the release asset URLs, and the only files touched are the two staging files and the final DLL, all inside the binary's own directory.

The engine always probes execution providers in the same fixed order at startup:

• CUDA — Windows + NVIDIA GPU + DLL installed. Fastest, opt-in via this command (or the npm postinstall auto-fetch).
• DirectML — Windows fallback when CUDA isn't available. Works on any DirectX 12 GPU, not just NVIDIA.
• CPU — universal fallback everywhere, including when neither of the above applies.

Flags#

Environment variables#

Gotchas#

The DLL is installed next to whichever binary executed the command, which is where the engine's provider probe looks. Skip paths ("already installed", "no GPU detected", non-Windows) exit 0; a parse error (unknown argument, dangling --version) exits 2.

Examples#

$ memtrace gpu status
# Check whether the CUDA provider DLL is installed next to the binary
# and whether an NVIDIA GPU is detected.

$ memtrace gpu install-cuda
# Fetch and install the ~400MB CUDA execution provider for this
# binary's release version; skips if already installed or no NVIDIA GPU detected.

$ memtrace gpu install-cuda --force
# Re-download and replace the DLL even if it is already present, no GPU
# is detected, or the install uses the no-AVX2 runtime.
# Run `memtrace stop` first if a running memtrace has the DLL loaded.

$ memtrace gpu install-cuda --version 0.8.3
# Fetch the asset from the v0.8.3 release tag instead of this binary's
# own version — useful when running a local build with no published release.

$ MEMTRACE_DISABLE_CUDA=1 memtrace start
# Force the DirectML/CPU fallback path even on an NVIDIA host
# that already has the CUDA provider DLL installed.

See also memtrace install, memtrace start, memtrace stop, and memtrace doctor.