Mistral AI · Mistral Modified MIT
Mistral Medium 3.5
Mistral’s 128B dense flagship. Fits 96–128 GB machines, but every word uses all 128B parameters, so it is slow.
How much memory it needs
The model file, plus memory for the conversation, plus about 1 GB for the app. Longer conversations need more.
| Size | Version | File | At 8K | At 32K | At 128K |
|---|---|---|---|---|---|
| 128B | Q4_K_M | 74.9 GB | 84.2 GB | 93.1 GB | 129 GB |
| Q8_0 | 133 GB | 142 GB | 151 GB | 186 GB |
Which computers run it
The best size for each machine at an 8K-token conversation. Speeds are estimates from memory bandwidth.
| Computer | Best version | Fit | Speed |
|---|---|---|---|
| Laptop or PC, no graphics card, 32 GB | 128B Q4_K_M | Too big | — |
| RTX 4060 · 8 GB | 128B Q4_K_M | Too big | — |
| RTX 3060 · 12 GB | 128B Q4_K_M | Too big | — |
| RTX 5060 Ti · 16 GB | 128B Q4_K_M | Too big | — |
| RTX 4090 · 24 GB | 128B Q4_K_M | Too big | — |
| RTX 5090 · 32 GB | 128B Q4_K_M | Too big | — |
| Mac with M4 · 16 GB | 128B Q4_K_M | Too big | — |
| Mac with M4 Pro · 48 GB | 128B Q4_K_M | Too big | — |
| Mac with M5 Max · 128 GB | 128B Q4_K_M | Runs great | 4.5–8tok/sSlow |
| Ryzen AI Max+ 395 · 128 GB | 128B Q4_K_M | Runs well | 2–3tok/sSlow |
| DGX Spark · 128 GB | 128B Q4_K_M | Runs well | 2–3.5tok/sSlow |
| RTX PRO 6000 · 96 GB | 128B Q4_K_M | Runs well | 14–20tok/sSmooth |
| Mac Studio with M5 Ultra · 256 GB | 128B Q4_K_M | Runs great | 7–12tok/sReadable |
| Mac Studio with M3 Ultra · 512 GB | 128B Q4_K_M | Runs great | 5–8tok/sReadable |
What it is good and bad at
Good at
- Strong general model you can keep on your own machine
Not good at
- Slow: a dense 128B model
- Licence excludes companies with over $20M monthly revenue
Start it
Install Ollama, then run one of these in a terminal. LM Studio users can search the same name in the app.
ollama run mistral-medium-3.5:128b