The short answer
For most people, 16 GB of graphics memory is where local AI starts to feel useful, and 24 GB is where it gets good. On a Mac, count about two-thirds of your memory up to 32 GB and a bit more on bigger Macs (about 56 GB of 64 GB), because macOS keeps the rest.
| Memory | Graphics card | Mac (shared memory) |
|---|---|---|
| 8 GB | Ornith 1.5 9B Q4_K_M | — |
| 12 GB | Gemma 4 12B Q4_K_M | — |
| 16 GB | Qwen3.8 27B UD-IQ3_S | Gemma 4 12B Q4_K_M |
| 24 GB | Qwen3.6 35B-A3B UD-IQ4_XS | Qwen3.8 27B UD-IQ3_S |
| 32 GB | Qwen3.6 35B-A3B Q4_K_M | Qwen3.6 35B-A3B UD-IQ4_XS |
| 48 GB | Kimi Linear 48B-A3B Q4_K_M | Kimi Linear 48B-A3B Q4_K_M |
| 64 GB | Qwen3-Coder Next 80B-A3B Q4_K_M | Qwen3-Coder Next 80B-A3B Q4_K_M |
| 96 GB | Qwen3.8 Flash Next 125B-A6B (+51B n-gram emb.) UD-IQ3_XXS | Qwen3.5 122B-A10B Q4_K_M |
| 128 GB | — | DeepSeek V4 Flash 0731 UD-IQ3_XXS |
What uses the memory
- The model file. The biggest part. A 14-billion-parameter model in the usual Q4 version is about 9 GB.
- The conversation. Everything you and the model have said so far is kept in memory (the “KV cache”). Short chats need little. A 128K-token conversation can need as much as the model itself.
- The app. Ollama or LM Studio needs about 1 GB on top.
If the total is bigger than your graphics memory, the model either will not load, or it spills into normal RAM and becomes 5 to 10 times slower.
How big is a model?
Each parameter is a number. The original models use 16 bits per number. Smaller versions (“quantised”) squeeze each number into fewer bits:
- Q8 — about 1.06 GB per billion parameters. Almost no quality loss.
- Q4 — about 0.6 GB per billion parameters. Small quality loss. This is what most people use.
- Q3 and below — fits bigger models into less memory, but the model gets clearly worse.
Some models are “mixture of experts” (MoE): only a small part works on each word. They still need memory for all their parameters, but they answer much faster.
Try it
- Model 8.4 GB
- Conversation memory 0.6 GB
- App overhead 1.0 GB
Runs great Needs about 10.0 GB. Rough guide — real models differ; check the checker for exact sizes.
Mac, graphics card or plain RAM?
- Graphics card: fastest per GB, but the memory is small (8–32 GB on normal cards).
- Mac: one pool of memory shared by everything. A 64 GB or 128 GB Mac can hold much bigger models than any normal graphics card, but answers come slower than on a fast card. By default macOS lets the graphics side use part of the memory — about two-thirds on small Macs, more on big ones; a command can raise this.
- No graphics card: it works, but slowly. Fine for small models; painful for big ones.
More on this choice in Mac or PC for local AI.