OpenBMB · Apache 2.0
MiniCPM5
Tiny 1B and 2B models for phones and old laptops, with open training data.
How much memory it needs
The model file, plus memory for the conversation, plus about 1 GB for the app. Longer conversations need more.
| Size | Version | File | At 8K | At 32K | At 128K |
|---|---|---|---|---|---|
| 1.1B | MXFP4 | 0.7 GB | 1.9 GB | 2.5 GB | 4.9 GB |
| 2.5B | Q4_K_M | 1.6 GB | 2.9 GB | 4.0 GB | 8.2 GB |
| Q8_0 | 2.7 GB | 4.0 GB | 5.1 GB | 9.3 GB |
Which computers run it
The best size for each machine at an 8K-token conversation. Speeds are estimates from memory bandwidth.
| Computer | Best version | Fit | Speed |
|---|---|---|---|
| Laptop or PC, no graphics card, 32 GB | 2.5B Q8_0 | Runs great | 18–30tok/sSmooth |
| RTX 4060 · 8 GB | 2.5B Q8_0 | Runs great | 50–80tok/sFast |
| RTX 3060 · 12 GB | 2.5B Q8_0 | Runs great | 65–110tok/sFast |
| RTX 5060 Ti · 16 GB | 2.5B Q8_0 | Runs great | 80–130tok/sFast |
| RTX 4090 · 24 GB | 2.5B Q8_0 | Runs great | 170–260tok/sFast |
| RTX 5090 · 32 GB | 2.5B Q8_0 | Runs great | 260–410tok/sFast |
| Mac with M4 · 16 GB | 2.5B Q8_0 | Runs great | 20–40tok/sSmooth |
| Mac with M4 Pro · 48 GB | 2.5B Q8_0 | Runs great | 50–85tok/sFast |
| Mac with M5 Max · 128 GB | 2.5B Q8_0 | Runs great | 100–170tok/sFast |
| Ryzen AI Max+ 395 · 128 GB | 2.5B Q8_0 | Runs great | 50–75tok/sFast |
| DGX Spark · 128 GB | 2.5B Q8_0 | Runs great | 50–80tok/sFast |
| RTX PRO 6000 · 96 GB | 2.5B Q8_0 | Runs great | 260–410tok/sFast |
| Mac Studio with M5 Ultra · 256 GB | 2.5B Q8_0 | Runs great | 140–240tok/sFast |
| Mac Studio with M3 Ultra · 512 GB | 2.5B Q8_0 | Runs great | 100–180tok/sFast |
What it is good and bad at
Good at
- Runs on almost anything
- 128K context
Not good at
- Small models make many more mistakes
- No Ollama library entry