The AI Bench
Allen Institute for AI (Ai2) · Apache 2.0

Olmo 3

Ai2’s fully open models (7B and 32B), each in a chat and a step-by-step thinking version.

How much memory it needs

The model file, plus memory for the conversation, plus about 1 GB for the app. Longer conversations need more.

SizeVersionFileAt 8KAt 32K
7.3BQ4_K_M4.5 GB6.9 GB11.4 GB
Q8_07.8 GB10.2 GB14.7 GB
7.3BQ4_K_M4.5 GB6.9 GB11.4 GB
Q8_07.8 GB10.2 GB14.7 GB
32BQ4_K_M19.5 GB21.2 GB23.4 GB
Q8_034.3 GB36.0 GB38.2 GB
32BQ4_K_M19.5 GB21.2 GB23.4 GB
Q8_034.3 GB36.0 GB38.2 GB

Which computers run it

The best size for each machine at an 8K-token conversation. Speeds are estimates from memory bandwidth.

ComputerBest versionFitSpeed
Laptop or PC, no graphics card, 32 GB7.3B Q8_0Runs great6–9.5tok/sReadable
RTX 4060 · 8 GB7.3B Q4_K_MRuns well25–45tok/sSmooth
RTX 3060 · 12 GB7.3B Q8_0Runs well25–35tok/sSmooth
RTX 5060 Ti · 16 GB7.3B Q8_0Runs great30–45tok/sSmooth
RTX 4090 · 24 GB32B Q4_K_MRuns well30–45tok/sSmooth
RTX 5090 · 32 GB32B Q4_K_MRuns great50–80tok/sFast
Mac with M4 · 16 GB7.3B Q8_0Runs well7.5–13tok/sReadable
Mac with M4 Pro · 48 GB32B Q4_K_MRuns great8–13tok/sReadable
Mac with M5 Max · 128 GB32B Q8_0Runs great10–17tok/sSmooth
Ryzen AI Max+ 395 · 128 GB32B Q4_K_MRuns great7.5–12tok/sReadable
DGX Spark · 128 GB32B Q4_K_MRuns great8–13tok/sReadable
RTX PRO 6000 · 96 GB32B Q8_0Runs great30–50tok/sSmooth
Mac Studio with M5 Ultra · 256 GB32B Q8_0Runs great15–25tok/sSmooth
Mac Studio with M3 Ultra · 512 GB32B Q8_0Runs great11–18tok/sSmooth

What it is good and bad at

Good at

  • Fully open data, code and checkpoints

Not good at

  • English-focused
  • 7B has a large conversation memory for its size

Start it

Install Ollama, then run one of these in a terminal. LM Studio users can search the same name in the app.

ollama run olmo-3:7b-instruct
ollama run olmo-3:7b-think
ollama run olmo-3.1:32b-instruct
ollama run olmo-3.1:32b-think