DeepSeek · MIT
DeepSeek V4 Flash
DeepSeek’s 284B model with a 1M context and very small conversation memory. Fits 192 GB at 4-bit and 128 GB at 3-bit.
How much memory it needs
The model file, plus memory for the conversation, plus about 1 GB for the app. Longer conversations need more.
| Size | Version | File | At 8K | At 32K | At 128K |
|---|---|---|---|---|---|
| 304B (13B active) | UD-IQ3_XXS | 104 GB | 105 GB | 105 GB | 106 GB |
| UD-Q3_K_XL | 128 GB | 129 GB | 129 GB | 130 GB | |
| UD-IQ4_XS | 137 GB | 138 GB | 138 GB | 139 GB | |
| UD-Q4_K_XL | 155 GB | 156 GB | 156 GB | 157 GB |
Which computers run it
The best size for each machine at an 8K-token conversation. Speeds are estimates from memory bandwidth.
| Computer | Best version | Fit | Speed |
|---|---|---|---|
| Laptop or PC, no graphics card, 32 GB | 304B UD-IQ3_XXS | Too big | — |
| RTX 4060 · 8 GB | 304B UD-IQ3_XXS | Too big | — |
| RTX 3060 · 12 GB | 304B UD-IQ3_XXS | Too big | — |
| RTX 5060 Ti · 16 GB | 304B UD-IQ3_XXS | Too big | — |
| RTX 4090 · 24 GB | 304B UD-IQ3_XXS | Too big | — |
| RTX 5090 · 32 GB | 304B UD-IQ3_XXS | Too big | — |
| Mac with M4 · 16 GB | 304B UD-IQ3_XXS | Too big | — |
| Mac with M4 Pro · 48 GB | 304B UD-IQ3_XXS | Too big | — |
| Mac with M5 Max · 128 GB | 304B UD-IQ3_XXS | Runs well | 40–90tok/sFast |
| Ryzen AI Max+ 395 · 128 GB | 304B UD-IQ3_XXS | Fits with a tweak | 20–40tok/sSmooth |
| DGX Spark · 128 GB | 304B UD-IQ3_XXS | Just fits | 20–40tok/sSmooth |
| RTX PRO 6000 · 96 GB | 304B UD-IQ3_XXS | Spills into system RAM | 45–85tok/sFast |
| Mac Studio with M5 Ultra · 256 GB | 304B UD-Q4_K_XL | Runs great | 35–70tok/sFast |
| Mac Studio with M3 Ultra · 512 GB | 304B UD-Q4_K_XL | Runs great | 25–55tok/sFast |
What it is good and bad at
Good at
- Long documents: tiny memory cost per word of context
- Only 13B parameters work per word
- MIT licence
Not good at
- Text only
- The newer V4.1 Flash does not run in llama.cpp yet