Where your Spark's 128 GB actually goes — and why the memory numbers lie
Field note · measured on GB10 unified memory
You have 128 GB. Your model is 70 GB. It won't load, or it loads and something wedges, or the machine OOM-reboots when you were sure there was headroom. On GB10 the memory accounting genuinely misleads you, because CPU and GPU share one pool and every tool you'd normally trust was written for machines where they don't.
MemAvailable lies on unified memory
The number you'd normally check to answer "will this fit" is computed on assumptions that don't hold when the GPU draws from the same pool as the CPU. We've watched a second model get loaded because the available-memory math said it fit — and the node OOM-rebooted, taking the running model with it. On unified memory, free-memory arithmetic is not a safe gate. Gate on what's actually resident and serving, not on a subtraction.
"Used" RAM after big reads is mostly reclaimable
Load a big model from disk and watch "used" memory spike and stay high — and panic, because now there's "no room." Much of that is page cache the kernel will hand back the instant something needs it. It looks like a leak; it's a cache. Reclaiming it is a specific operation, and knowing the difference between reclaimable cache and a genuine allocation is the difference between "I'm out of memory" and "I have 40 GB I didn't know about."
The mental model: on GB10, treat memory as one shared budget with three claimants — your model weights, your KV cache, and everything the OS and other processes are holding (including reclaimable cache). Tools that assume a separate GPU pool will mis-report all three.
The knobs that wedge you
- Setting the GPU memory fraction too high starves the CPU-side of the shared pool and silently wedges a downstream component — the fraction is taken from everyone, not from a dedicated GPU reserve.
- Hard-killing a wedged server can leak tens of GB per node that no process shows as owning — reclaimable, but only if you know the sequence.
- KV cache is the silent third claimant — it grows with context and precision, and at long context it competes directly with weights for the same bytes.
Accounting for 128 GB of unified memory correctly is the difference between confidently loading a model that fits and repeatedly OOM-rebooting a machine that had the room all along. The tools mislead; the budget is knowable.
OOM errors on a machine that should have room?
We map exactly where your 128 GB is going — weights, KV, reclaimable cache, and the leaks — on real GB10 hardware, and hand you a memory budget you can actually trust. Or it's free.
Book a 1-day diagnostic →Free to read and share. Numbers are from hardware we've measured directly; your versions and topology will differ — measure your own.