← Brown Cyber Services
Theme Text Layout

Memory Ladder

This chart places 23 open-weight models by the real size of their downloadable files, so you can see which ones fit in your hardware's memory, with capacity lines you can set yourself. It's for anyone choosing a model to run on their own hardware, and you'll know what fits before you download anything.

Which open-weight models fit in how much memory. Footprints are the actual Q4-class GGUF file sizes on Hugging Face as of October 2026, not parameter arithmetic. Memory decides what fits; bandwidth decides how fast it feels. For the second half, see Geryon.

Rule of thumb: weights at Q4 ≈ 0.55–0.65 GB per billion parameters, then add 10–20% for KV cache and runtime. A model fits when its bar ends left of your line with that margin to spare.

The ladder

DenseMoE (total weights; only the active slice is read per token)Estimated, no public Q4 filePreset hardware (toggle above)Your line (from the box)

Tiers

Models

Sizes are the main GGUF shard set at the quant listed, read from the repo file listing. Vision projectors and MTP draft heads are excluded. Qwen3.8-Flash-Next's figure includes its 51B n-gram embedding table, which Qwen designed to offload to system RAM; without it the GPU-resident part is roughly 80 GB.

Why file size and not parameter count

Quantization isn't uniform. Unsloth's dynamic quants keep attention and embedding tensors at higher precision, Mamba state projections stay in bf16, MoE routers stay in fp32, and ternary models pack below 2 bits. The published file is what you will actually load, so that's what the ladder plots. Where no public Q4 file exists the bar is grey and the number is ~0.6 GB/B.

What the hardware lines mean

Each line is usable memory, not the sticker. A discrete GPU loses ~0.5–1 GB to the display and runtime. Apple unified memory defaults to ~75% for the GPU (the 128 GB line sits at 96 GB; it can be raised to ~120 GB with a wired-limit sysctl). An 8-GPU node is the sum of its cards and needs tensor or pipeline parallelism to use it as one pool.

© 2026 Brown Cyber Services Inc · All rights reserved · Colorado · Access logs retained 90 days