Run your first local model in 10 minutes

Before you buy anything

Run a model on whatever you own first. A 2020 laptop runs a 3B model. A gaming PC with any 8GB card runs an 8B model well. Ten minutes on your current machine tells you more about what you want than any guide, including this one.

Step 1: pick a tool

Two choices, both free, both fine.

Ollama (ollama.com): a command-line tool and a background service. Fastest to install, scriptable, and what most other local-AI software talks to. Pick this if you’re comfortable in a terminal or you want to connect other apps to it.

LM Studio (lmstudio.ai): a desktop app with a model browser, a chat window and a settings panel that shows you what’s happening. Pick this if you want to see memory use and GPU offload with your own eyes, or you’d rather not type commands.

Both use your GPU automatically: CUDA on NVIDIA, Metal on a Mac, Vulkan or ROCm on AMD. Nothing to configure for a first run.

Step 2: find out how much fast memory you have

• PC with a graphics card: Task Manager, Performance tab, GPU, “Dedicated GPU memory.” That number is your budget.
• Mac: Apple menu, About This Mac, Memory. Subtract about 8GB for macOS and your apps; the rest is your budget.
• Laptop with no discrete GPU: your budget is system RAM minus 8GB, and everything will run on the CPU, slowly. Start with a 3B model.

Step 3: pick a first model

Your budget	Start with	Why
under 6GB	gemma3:4b or llama3.2:3b	small, fast, surprisingly capable
6 to 10GB	llama3.1:8b or qwen3:8b	the standard first model; good at everything, great at nothing
10 to 15GB	gemma3:12b or qwen3:14b	the first size that feels like a real assistant
16 to 23GB	gpt-oss:20b	OpenAI’s open model, 13GB, strong at reasoning and code
24GB or more	qwen3:32b or gemma3:27b	30B class, the sweet spot of quality per GB
64GB or more (Mac)	llama3.3:70b	runs; see the speed note below

Those are Ollama names. LM Studio’s search finds the same models; pick the Q4_K_M file when it asks.

Step 4: run it

Ollama, in a terminal:

ollama run qwen3:8b

The first run downloads the model (5GB for an 8B model; give it a few minutes). Then you’re in a chat. Type, read, repeat. /bye exits.

LM Studio: open the app, search for the model, click download, then open the chat tab, pick it from the dropdown at the top, and type.

Step 5: check it’s actually using the GPU

This is the step people skip and then conclude local AI is slow.

• Ollama: in a second terminal, ollama ps. The PROCESSOR column should say 100% GPU. If it says CPU or a split like 60%/40%, the model didn’t fit in VRAM and spilled to system RAM.
• LM Studio: the model settings show “GPU offload” as a number of layers. It should be at the maximum. If the app lowered it, the model didn’t fit.

If it spilled, pick a smaller model or a tighter quantization (Q3 instead of Q4) and try again. Speed will jump five to ten times.

What to expect

On a card that fits the model, an 8B model generates faster than you can read, 60 to 140 tokens a second depending on the card. A 32B model on a 24GB card does about 30. On a Mac, divide those by two to three; on a laptop CPU, by ten. The memory table has the full numbers.

The first thing most people notice: an 8B model is good at drafting, summarising and explaining, and bad at facts it has to recall. That’s not your hardware; it’s the model size. The jump to 30B is where recall and reasoning get noticeably better, and 24GB of VRAM is what that jump costs.

The three things that go wrong

1. It’s slow. It spilled to RAM. Step 5.
2. It won’t load at all. The model is bigger than your memory even at Q4. Go down a row in the table.
3. It’s using the GPU and still slow on a laptop. The laptop is throttling. Plug it in, set the power mode to performance, and accept that a laptop runs at half the speed of the same chip in a desktop.

Where to go from here

Once something works, the question becomes what you want to run and how fast. What can I run with X GB answers the first half; the GPU guide and Mac guide answer the second.