NVIDIA · NVIDIA Nemotron Open
NVIDIA Nemotron 3
NVIDIA reasoning models with a hybrid design that keeps conversation memory tiny, even at long context.
How much memory it needs
The model file, plus memory for the conversation, plus about 1 GB for the app. Longer conversations need more.
| Size | Version | File | At 8K | At 32K | At 128K |
|---|---|---|---|---|---|
| 4B | Q4_K_M | 2.8 GB | 4.0 GB | 4.4 GB | 6.0 GB |
| Q8_0 | 4.2 GB | 5.4 GB | 5.8 GB | 7.4 GB | |
| 32B (3B active) | Q4_K_M | 24.6 GB | 25.6 GB | 25.8 GB | 26.4 GB |
| Q8_0 | 33.6 GB | 34.6 GB | 34.8 GB | 35.4 GB | |
| 124B (12B active) | Q4_K_M | 86.0 GB | 87.1 GB | 87.3 GB | 88.1 GB |
| Q8_0 | 128 GB | 130 GB | 130 GB | 131 GB |
Which computers run it
The best size for each machine at an 8K-token conversation. Speeds are estimates from memory bandwidth.
| Computer | Best version | Fit | Speed |
|---|---|---|---|
| Laptop or PC, no graphics card, 32 GB | 4B Q8_0 | Runs great | 12–19tok/sSmooth |
| RTX 4060 · 8 GB | 4B Q8_0 | Runs great | 35–60tok/sFast |
| RTX 3060 · 12 GB | 4B Q8_0 | Runs great | 50–75tok/sFast |
| RTX 5060 Ti · 16 GB | 4B Q8_0 | Runs great | 60–90tok/sFast |
| RTX 4090 · 24 GB | 4B Q8_0 | Runs great | 120–190tok/sFast |
| RTX 5090 · 32 GB | 32B Q4_K_M | Runs well | 140–270tok/sFast |
| Mac with M4 · 16 GB | 4B Q8_0 | Runs great | 16–25tok/sSmooth |
| Mac with M4 Pro · 48 GB | 32B Q8_0 | Runs well | 30–70tok/sFast |
| Mac with M5 Max · 128 GB | 124B Q4_K_M | Runs great | 30–60tok/sFast |
| Ryzen AI Max+ 395 · 128 GB | 124B Q4_K_M | Runs well | 11–20tok/sSmooth |
| DGX Spark · 128 GB | 124B Q4_K_M | Runs well | 12–25tok/sSmooth |
| RTX PRO 6000 · 96 GB | 124B Q4_K_M | Runs well | 60–120tok/sFast |
| Mac Studio with M5 Ultra · 256 GB | 124B Q8_0 | Runs great | 20–45tok/sSmooth |
| Mac Studio with M3 Ultra · 512 GB | 124B Q8_0 | Runs great | 15–30tok/sSmooth |
What it is good and bad at
Good at
- Very long conversations without much extra memory
- Fast mixture-of-experts sizes
Not good at
- Nano 30B-A3B file is larger than similar 30B models (about 25 GB at Q4)
Start it
Install Ollama, then run one of these in a terminal. LM Studio users can search the same name in the app.
ollama run nemotron-3-nano:4bollama run nemotron-3-nano:30bollama run nemotron-3-super:120b