Updated October 2026
What can I run with X GB? Memory, model size and speed in one table
The question behind every hardware question. How much memory each model size needs, which models fit at 8, 12, 16, 24, 32, 64 and 128GB, and how to estimate tokens per second from bandwidth before you buy anything.
As an Amazon Associate I earn from qualifying purchases. Product links on this page go to Amazon; they are affiliate links, and the price you pay is the same.
The picks
NVIDIA GeForce RTX 5060 Ti (16GB)
The cheapest way to the 16GB line: 8B to 14B models at full quality, 30B-class at a squeeze.
For local LLMs, holding the model matters more than speed. This is the lowest price of entry to 16GB with CUDA. Get the 16GB version; the 8GB one is useless for this.
NVIDIA GeForce RTX 3090 (24GB, renewed)
The cheapest way to the 24GB line: 30B-class at full quality, which is where most people should aim.
For LLM inference, VRAM capacity matters more than compute. Two of these give you 48GB for less than one 5090.
Mac Studio (M5 Max, 128GB unified memory)
The cheapest quiet way to the 128GB line: 70B at full quality and gpt-oss-120B, at a speed you'll use daily.
The smallest quiet box that runs 70B models at a speed you'd use every day. The 128GB Strix Halo boxes fit the same models at less than half the bandwidth.
The one rule, again
A model has to fit in fast memory. On a PC that’s the graphics card’s VRAM; on a Mac or a Strix Halo box it’s unified memory. If it fits, it runs at a speed set by memory bandwidth. If it doesn’t, it spills to system RAM and runs about ten times slower, or doesn’t load at all.
So two numbers decide everything: how much memory (what fits) and how fast it is (tokens per second). This page gives you both.
How much memory a model needs
Rule of thumb at 4-bit quantization (Q4_K_M in Ollama and llama.cpp, which is the default most people run): model size in GB ≈ parameters in billions × 0.6, plus 1 to 3GB for context. At 8-bit, double it. At full 16-bit, quadruple it.
| Model class | Examples | Size at Q4 | Fits in |
|---|---|---|---|
| 3B to 4B | Llama 3.2 3B, Gemma 3 4B, Qwen3 4B | 2 to 3GB | anything, including 8GB cards and phones |
| 7B to 9B | Llama 3.1 8B, Qwen3 8B, Gemma 3 12B at a squeeze | 5 to 6GB | 8GB card comfortably, 12GB with long context |
| 12B to 14B | Gemma 3 12B, Qwen3 14B, Phi-4 14B | 8 to 9GB | 12GB card minimum, 16GB comfortable |
| 20B to 24B | gpt-oss-20B (MXFP4, 13GB), Mistral Small 24B | 13 to 15GB | 16GB card, tight; 24GB comfortable |
| 27B to 32B | Gemma 3 27B, Qwen3 32B, QwQ 32B | 17 to 20GB | 24GB card. On 16GB only at Q3 with visible quality loss |
| 70B | Llama 3.3 70B, Qwen2.5 72B | 40 to 43GB | two 24GB cards, a 64GB Mac (barely), 96GB+ anything |
| 120B MoE | gpt-oss-120B (MXFP4) | 61 to 65GB | 96GB or 128GB unified memory, or 3 x 24GB cards |
| 235B MoE | Qwen3 235B-A22B | 130 to 140GB | Mac Studio 256GB or 512GB |
| 400B+ | Llama 3.1 405B, DeepSeek V3 / R1 (671B) | 230 to 400GB | Mac Studio M5 Ultra 512GB; nothing else on a desk |
Two things the table hides. Mixture-of-experts models (gpt-oss, Qwen3 235B, DeepSeek) are big on disk but only run a fraction of their parameters per token, so they generate faster than a dense model of the same size; gpt-oss-120B on a 128GB machine is quicker than Llama 70B on the same machine. And context eats memory: a 32K-token conversation on a 30B model can add 4 to 6GB. If you plan long contexts, buy the next size up.
What each amount of memory buys you
| You have | Run at full quality | Run with compromises | Don’t bother |
|---|---|---|---|
| 8GB | 3B to 8B | 12B at Q3 | 14B and up |
| 12GB | up to 14B | 20B class | 27B and up |
| 16GB | up to 14B, gpt-oss-20B | 27B to 32B at Q3 or short context | 70B |
| 24GB | up to 32B | 70B only with heavy CPU offload | 120B |
| 32GB (5090) | up to 32B with long context | 70B at Q3, awkward | 120B |
| 48GB (two 24GB cards) | 70B | 120B at Q3 | 235B |
| 64GB (Mac mini, Strix Halo 64) | 70B, barely, short context | 120B no | 235B |
| 96GB to 128GB | 70B comfortably, gpt-oss-120B | 235B at Q3, slow | 400B |
| 256GB to 512GB (M5 Ultra) | everything that exists, including 671B at Q4 |
How fast it will run
Generation speed is memory-bound. A usable estimate:
tokens per second ≈ 0.7 × memory bandwidth (GB/s) ÷ model size (GB)
The 0.7 covers real-world inefficiency; well-tuned setups do better, Windows laptops do worse. Bandwidth figures for current hardware:
| Hardware | Bandwidth | 8B model (5GB) | 32B model (20GB) | 70B model (42GB) |
|---|---|---|---|---|
| RTX 5060 Ti 16GB | 448GB/s | ~60 t/s | doesn’t fit | no |
| RTX 5070 Ti | 896GB/s | ~125 t/s | doesn’t fit at Q4 | no |
| RTX 3090 (24GB) | 936GB/s | ~130 t/s | ~33 t/s | no |
| RTX 4090 (24GB) | 1,008GB/s | ~140 t/s | ~35 t/s | no |
| RTX 5090 (32GB) | 1,792GB/s | ~250 t/s | ~63 t/s | no |
| Strix Halo 128GB | 256GB/s | ~36 t/s | ~9 t/s | ~4 t/s |
| DGX Spark 128GB | 273GB/s | ~38 t/s | ~10 t/s | ~4.5 t/s |
| Mac mini M5 Pro 64GB | 307GB/s | ~43 t/s | ~11 t/s | ~5 t/s |
| Mac Studio M5 Max 128GB | 614GB/s | ~86 t/s | ~21 t/s | ~10 t/s |
| Mac Studio M5 Ultra 512GB | 1,229GB/s | ~170 t/s | ~43 t/s | ~20 t/s |
What those numbers mean in practice: 10 tokens a second is reading speed, fine for chat. 30 and up feels instant. Under 5 is a coding agent you’ll stop using. The 70B column is why the Mac Studio M5 Max is the pick for 70B models and why the 128GB PC boxes are better at holding them than running them.
Prompt processing (reading what you paste in) is compute-bound, not bandwidth-bound, and NVIDIA cards are far ahead there. If your workload is long documents in and short answers out, weight the GPU numbers up.
Which page next
- Picking a graphics card: Best GPU for local AI
- Picking a Mac: Best Mac for local LLMs
- Everything else in the box: Build a local AI box
- The three 128GB desktop boxes side by side: DGX Spark vs Mac Studio vs Strix Halo