The AI Bench
Alibaba (Qwen) · Apache 2.0

Qwen3.5

A full family from phone size (0.8B) to 122B, every size reading images. A strong pick for small machines.

How much memory it needs

The model file, plus memory for the conversation, plus about 1 GB for the app. Longer conversations need more.

SizeVersionFileAt 8KAt 32KAt 128K
0.9BMXFP40.5 GB1.8 GB2.1 GB3.3 GB
2.3BMXFP41.3 GB3.1 GB3.4 GB4.5 GB
4.7BQ4_K_M2.7 GB4.7 GB5.5 GB8.7 GB
Q8_04.5 GB6.4 GB7.2 GB10.4 GB
9.7BQ4_K_M5.7 GB7.9 GB8.7 GB11.9 GB
Q8_09.5 GB11.7 GB12.5 GB15.7 GB
28BQ4_K_M18.0 GB20.4 GB22.1 GB28.5 GB
Q8_029.1 GB31.6 GB33.2 GB39.6 GB
36B (3B active)Q4_K_M22.0 GB24.1 GB24.6 GB26.6 GB
Q8_036.9 GB39.0 GB39.5 GB41.5 GB
125B (10B active)Q4_K_M76.5 GB78.7 GB79.3 GB81.7 GB
Q8_0130 GB132 GB133 GB135 GB

Which computers run it

The best size for each machine at an 8K-token conversation. Speeds are estimates from memory bandwidth.

ComputerBest versionFitSpeed
Laptop or PC, no graphics card, 32 GB9.7B Q8_0Runs great5.5–8.5tok/sReadable
RTX 4060 · 8 GB4.7B Q8_0Runs well35–55tok/sFast
RTX 3060 · 12 GB9.7B Q4_K_MRuns great35–55tok/sFast
RTX 5060 Ti · 16 GB9.7B Q8_0Runs great25–45tok/sSmooth
RTX 4090 · 24 GB28B Q4_K_MRuns well30–50tok/sFast
RTX 5090 · 32 GB36B Q4_K_MRuns well150–290tok/sFast
Mac with M4 · 16 GB9.7B Q4_K_MRuns great12–20tok/sSmooth
Mac with M4 Pro · 48 GB36B Q4_K_MRuns great40–90tok/sFast
Mac with M5 Max · 128 GB125B Q4_K_MRuns great35–75tok/sFast
Ryzen AI Max+ 395 · 128 GB125B Q4_K_MRuns well15–30tok/sSmooth
DGX Spark · 128 GB125B Q4_K_MRuns great16–30tok/sSmooth
RTX PRO 6000 · 96 GB125B Q4_K_MRuns well75–150tok/sFast
Mac Studio with M5 Ultra · 256 GB125B Q8_0Runs great25–50tok/sSmooth
Mac Studio with M3 Ultra · 512 GB125B Q8_0Runs great17–35tok/sSmooth

Measured speeds

Real runs, not estimates. All measurements.

ComputerVersionPromptSpeedSource
NVIDIA DGX Spark (GB10, 128 GB LPDDR5x)Q8_051251.33 tok/s2026-03-10
Apple M5 Max (18-core CPU, 40-core GPU, 14-inch MacBook Pro, 128 GB)MLX 4-bit4K65.9 tok/s2026-03-11
NVIDIA GeForce RTX 5090 32 GBMXFP4256K97.3 tok/s2026-03
NVIDIA GeForce RTX 3090 24 GB (used)MXFP4128K79.4 tok/s2026-03
NVIDIA DGX Spark (GB10, 128 GB LPDDR5x)Q4_K_M51259.98 tok/s2026-03-30

What it is good and bad at

Good at

  • A size for every machine, from 8 GB laptops to 128 GB
  • Images and video in every size
  • Small conversation memory thanks to its hybrid design

Not good at

  • The small sizes (0.8B–4B) make more mistakes on hard questions

Start it

Install Ollama, then run one of these in a terminal. LM Studio users can search the same name in the app.

ollama run qwen3.5:4b
ollama run qwen3.5:9b
ollama run qwen3.5:27b
ollama run qwen3.5:35b-a3b
ollama run qwen3.5:122b-a10b