Mistral AI · Apache 2.0
Devstral Small 2
Mistral’s 24B coding-agent model. Mistral says it runs on one RTX 4090 or a 32 GB Mac.
How much memory it needs
The model file, plus memory for the conversation, plus about 1 GB for the app. Longer conversations need more.
| Size | Version | File | At 8K | At 32K | At 128K |
|---|---|---|---|---|---|
| 24B | Q4_K_M | 14.3 GB | 17.6 GB | 21.6 GB | 37.7 GB |
| Q8_0 | 25.1 GB | 28.3 GB | 32.3 GB | 48.4 GB |
Which computers run it
The best size for each machine at an 8K-token conversation. Speeds are estimates from memory bandwidth.
| Computer | Best version | Fit | Speed |
|---|---|---|---|
| Laptop or PC, no graphics card, 32 GB | 24B Q4_K_M | Runs well | 3.5–5.5tok/sSlow |
| RTX 4060 · 8 GB | 24B Q4_K_M | Spills into system RAM | 4.5–7tok/sSlow |
| RTX 3060 · 12 GB | 24B Q4_K_M | Spills into system RAM | 6.5–10tok/sReadable |
| RTX 5060 Ti · 16 GB | 24B Q4_K_M | Spills into system RAM | 12–19tok/sSmooth |
| RTX 4090 · 24 GB | 24B Q4_K_M | Runs great | 40–60tok/sFast |
| RTX 5090 · 32 GB | 24B Q8_0 | Runs well | 40–60tok/sFast |
| Mac with M4 · 16 GB | 24B Q4_K_M | Too big | — |
| Mac with M4 Pro · 48 GB | 24B Q8_0 | Runs great | 6–10tok/sReadable |
| Mac with M5 Max · 128 GB | 24B Q8_0 | Runs great | 13–25tok/sSmooth |
| Ryzen AI Max+ 395 · 128 GB | 24B Q8_0 | Runs great | 6–9.5tok/sReadable |
| DGX Spark · 128 GB | 24B Q8_0 | Runs great | 6–10tok/sReadable |
| RTX PRO 6000 · 96 GB | 24B Q8_0 | Runs great | 40–60tok/sFast |
| Mac Studio with M5 Ultra · 256 GB | 24B Q8_0 | Runs great | 20–35tok/sSmooth |
| Mac Studio with M3 Ultra · 512 GB | 24B Q8_0 | Runs great | 14–25tok/sSmooth |
What it is good and bad at
Good at
- Working through a codebase and editing many files
Not good at
- Coding only
- Conversation memory grows fast at long context
Start it
Install Ollama, then run one of these in a terminal. LM Studio users can search the same name in the app.
ollama run devstral-small-2:24b