Ternary-Bonsai-2-27B: a 27B model in 5.95 GB, with a runtime catch

PrismML released Ternary-Bonsai-2-27B on September 16, 2026 under Apache 2.0. It keeps Qwen3.8-27B's architecture and 262K context while ternarizing the language weights to a claimed 1.72 effective bits per weight: 5.95 GB in PTQ1_0 or 7.21 GB in PQ2_0. Hugging Face showed 2.23 million downloads by September 22. That adoption earns attention; the custom runtime requirement keeps it out of the planner for now.

Verdict: a 27B Qwen derivative compressed to 6–7 GB is genuinely interesting, but it needs PrismML's forked runtimes and independent quality checks

The take

The model card says the conversion preserves 98.2% of the FP16 checkpoint's average across 14 thinking-mode benchmarks, with especially small losses in code and maths. Those numbers are PrismML's own evaluation of its own compression. Its speed table is also vendor-measured: 28.1 tok/s on an M5 Pro and 129.9 tok/s on an RTX 5090 in the supported runtime. We have not independently reproduced either result.

The catch is not cosmetic: stock llama.cpp does not understand PrismML's PTQ/PQ tensor types. The card warns that upstream tools may reject the model or load it incorrectly; users need PrismML's pinned llama.cpp fork/binaries or its MLX path. That makes this a new quantization system, not a drop-in smaller GGUF.

Our call: watch, do not promote. A real 27B-class model in 6–7 GB could redraw the low-tier map, but millions of automated downloads and a vendor benchmark are not yet evidence that the quality/runtime combination survives normal workloads. Qwen3.8-27B remains the dependable 24 GB pick.

Where this fits

Models: Qwen3.8-27B · BitCPM4-CANN family (0.5B / 1B / 3B / 8B, native 1.58-bit) · Qwen 3.5 9B

Hardware: RTX 5060 Ti 16 GB · Mac Mini M4 16 GB · NVIDIA RTX 5090

Sources

Next step

Try this in the planner→