The AI Bench
IBM · Apache 2.0

IBM Granite 4.2

IBM’s business-focused models (3B, 8B, 30B) with a thinking mode and tool calls.

How much memory it needs

The model file, plus memory for the conversation, plus about 1 GB for the app. Longer conversations need more.

SizeVersionFileAt 8KAt 32KAt 128K
3.7BQ4_K_M2.2 GB3.9 GB5.9 GB14.0 GB
Q8_03.9 GB5.6 GB7.6 GB15.6 GB
8.8BQ4_K_M5.3 GB7.7 GB11.7 GB27.8 GB
Q8_09.3 GB11.7 GB15.7 GB31.8 GB
29BQ4_K_M17.7 GB20.9 GB27.3 GB53.1 GB
Q8_031.1 GB34.3 GB40.7 GB66.5 GB

Which computers run it

The best size for each machine at an 8K-token conversation. Speeds are estimates from memory bandwidth.

ComputerBest versionFitSpeed
Laptop or PC, no graphics card, 32 GB8.8B Q4_K_MRuns great8–13tok/sReadable
RTX 4060 · 8 GB3.7B Q8_0Runs great35–55tok/sFast
RTX 3060 · 12 GB8.8B Q4_K_MRuns great30–50tok/sFast
RTX 5060 Ti · 16 GB8.8B Q8_0Runs great25–40tok/sSmooth
RTX 4090 · 24 GB29B Q4_K_MRuns well30–45tok/sSmooth
RTX 5090 · 32 GB29B Q4_K_MRuns great50–80tok/sFast
Mac with M4 · 16 GB8.8B Q4_K_MRuns great10–18tok/sSmooth
Mac with M4 Pro · 48 GB29B Q8_0Runs well5–8tok/sReadable
Mac with M5 Max · 128 GB29B Q8_0Runs great11–18tok/sSmooth
Ryzen AI Max+ 395 · 128 GB29B Q4_K_MRuns great8–12tok/sReadable
DGX Spark · 128 GB29B Q4_K_MRuns great8.5–13tok/sReadable
RTX PRO 6000 · 96 GB29B Q8_0Runs great30–50tok/sFast
Mac Studio with M5 Ultra · 256 GB29B Q8_0Runs great16–25tok/sSmooth
Mac Studio with M3 Ultra · 512 GB29B Q8_0Runs great11–19tok/sSmooth

What it is good and bad at

Good at

  • Tool calling and business tasks
  • Apache 2.0 with no extra terms

Not good at

  • The 30B is dense and needs about 18 GB at Q4

Start it

Install Ollama, then run one of these in a terminal. LM Studio users can search the same name in the app.

ollama run granite4.2:3b
ollama run granite4.2:8b
ollama run granite4.2:30b