The AI Bench
Alibaba (Qwen) · Apache 2.0

Qwen3

The 2025 Qwen text models. Older now, but supported everywhere and well tested.

How much memory it needs

The model file, plus memory for the conversation, plus about 1 GB for the app. Longer conversations need more.

SizeVersionFileAt 8KAt 32KAt 128K
0.8BMXFP40.4 GB2.3 GB5.2 GB—
2BMXFP41.1 GB3.0 GB5.9 GB—
4BQ4_K_M2.5 GB4.7 GB8.3 GB22.8 GB
Q8_04.3 GB6.5 GB10.1 GB24.6 GB
8.2BQ4_K_M5.0 GB7.2 GB10.9 GB—
Q8_08.7 GB10.9 GB14.5 GB—
15BQ4_K_M9.0 GB11.3 GB15.4 GB—
Q8_015.7 GB18.0 GB22.1 GB—
31B (3.3B active)Q4_K_M18.6 GB20.4 GB22.8 GB32.4 GB
Q8_032.5 GB34.3 GB36.7 GB46.4 GB
33BQ4_K_M19.8 GB22.9 GB29.4 GB—
Q8_034.8 GB38.0 GB44.4 GB—

Which computers run it

The best size for each machine at an 8K-token conversation. Speeds are estimates from memory bandwidth.

ComputerBest versionFitSpeed
Laptop or PC, no graphics card, 32 GB8.2B Q8_0Runs great5.5–8.5tok/sReadable
RTX 4060 · 8 GB8.2B Q4_K_MRuns well25–40tok/sSmooth
RTX 3060 · 12 GB8.2B Q4_K_MRuns great35–55tok/sFast
RTX 5060 Ti · 16 GB15B Q4_K_MRuns great25–40tok/sSmooth
RTX 4090 · 24 GB31B Q4_K_MRuns well90–180tok/sFast
RTX 5090 · 32 GB33B Q4_K_MRuns great45–75tok/sFast
Mac with M4 · 16 GB8.2B Q4_K_MRuns great11–19tok/sSmooth
Mac with M4 Pro · 48 GB33B Q4_K_MRuns great7–12tok/sReadable
Mac with M5 Max · 128 GB33B Q8_0Runs great9.5–16tok/sReadable
Ryzen AI Max+ 395 · 128 GB33B Q4_K_MRuns great7–11tok/sReadable
DGX Spark · 128 GB33B Q4_K_MRuns great7.5–12tok/sReadable
RTX PRO 6000 · 96 GB33B Q8_0Runs great30–45tok/sSmooth
Mac Studio with M5 Ultra · 256 GB33B Q8_0Runs great15–25tok/sSmooth
Mac Studio with M3 Ultra · 512 GB33B Q8_0Runs great10–17tok/sSmooth

Measured speeds

Real runs, not estimates. All measurements.

ComputerVersionPromptSpeedSource
NVIDIA GeForce RTX 4090 24 GBQ4_K (assumed Q4_K_M)4K207.3 tok/s2026-03
NVIDIA GeForce RTX 4090 24 GBQ4_K (assumed Q4_K_M)4K39.6 tok/s2026-03
NVIDIA GeForce RTX 4070 12 GBQ4_K (assumed Q4_K_M)4K42.5 tok/s2026-03
NVIDIA GeForce RTX 4070 Ti 12 GBQ4_K (assumed Q4_K_M)4K75.8 tok/s2026-03
NVIDIA GeForce RTX 4060 Ti 16 GBQ4_K (assumed Q4_K_M)4K27.4 tok/s2026-03
AMD Ryzen AI Max+ 395 "Strix Halo" / Radeon 8060S (Framework Desktop, 128 GB LPDDR5x-8000)UD-Q4_K_XL51278 tok/s2025-08
AMD Ryzen AI Max+ 395 "Strix Halo" / Radeon 8060S (Framework Desktop, 128 GB LPDDR5x-8000)Q8_05126.4 tok/s2025-08
Apple M5 Max (40-core GPU, 128 GB)MLX 4-bit4K119.1 tok/s2026-06-20
NVIDIA GeForce RTX 5060 Ti 16 GBQ4_K32K25.9 tok/s2026-03

What it is good and bad at

Good at

  • Runs in every app and tool
  • Thinking mode for harder questions

Not good at

  • Text only — no image input
  • Most sizes have a short 40K context
  • Replaced by Qwen3.5 and Qwen3.8

Start it

Install Ollama, then run one of these in a terminal. LM Studio users can search the same name in the app.

ollama run qwen3:4b-instruct-2507-q4_K_M
ollama run qwen3:8b
ollama run qwen3:14b
ollama run qwen3:30b-a3b-instruct-2507-q4_K_M
ollama run qwen3:32b