Mage-VL — the rare small model with a genuinely new idea

Microsoft released Mage-VL on July 25 under Apache 2.0: a ~4.7B vision-language model for image and video understanding whose visual encoder was trained from scratch. The idea is the interesting part. Instead of sampling video frames at fixed intervals, it reads the video codec — keeping every anchor-frame patch and only the motion-salient patches from predicted frames, using where the codec spent its bits as the signal for where the detail is. That cuts visual tokens by over 75% for up to a 3.5× wall-clock speedup at matched accuracy. At roughly 3–4 GB quantized it runs on hardware you already own, and it has taken 272,000 downloads in about nine days.

Verdict: A 4B Apache-2.0 video model that reads the codec instead of the frames — 3.5× faster, and it actually runs on your laptop

The take

Verified from the model card and the HF API: `microsoft/Mage-VL`, created July 25 2026, `apache-2.0`, ~4.7B parameters in a single safetensors shard, arXiv 2607.24904. The architecture is a from-scratch Codec-ViT ("Mage-ViT") feeding a 4B Qwen3 decoder. It is codec-agnostic by design — traditional H.264/HEVC via motion vectors and residual energy, or a neural codec (DCVC-RT) via its learned rate map, with no architecture change or retraining between them. On top of that sits a dual-process design: a lightweight "cognition gate" watches each rolling codec window and stays silent on routine content, waking the full model only when a response-worthy event completes. One checkpoint covers image understanding, offline video, and that proactive streaming mode — the repo bundles the codec processor, the neural codec package and the gate weights, so there is nothing to assemble.

The benchmark framing is the honest kind. Microsoft holds the 4B Qwen3 backbone fixed and swaps only the visual encoder against Qwen3-VL-4B, which isolates the contribution of the idea being tested rather than confounding it with a bigger decoder. On that comparison it improves on every reported video and temporal-grounding benchmark, with the largest gains on localization-heavy tasks: +22.5 on QVHighlight, +24.5 on VideoEval-Pro, +17.1 on ActivityNet, +11.0 on VSI-Bench. On stills it is a narrower win — DocVQA 95.14 vs 94.69, ChartQA 84.88 vs 83.96, OCRBench 81.80 vs 81.60 — and it actually loses TextVQA (77.28 vs 80.55). So: a clear video-and-temporal win, a marginal image win, and one loss they printed.

Our call: model entry at /models/mage-vl/, no planner pick yet. We have no video-understanding use case in the planner, so there is no slot for it to take — and the numbers are vendor-run, however well-designed the comparison. But this is the drop from this window most likely to be useful to an actual reader, because it is the one you can run: about 10 GB at BF16, 3–4 GB quantized, on a Mac mini or any 8 GB card. It is not in the Ollama library, so reach for transformers with `trust_remote_code=True` or SGLang. If you have ever wanted to ask questions of a long video without paying per frame, this is worth an afternoon.

Where this fits

Models: Mage-VL (4B codec-native video VLM) · MiniCPM-V-4.6 (1B vision-language) · MOSS-VL-0408 (Base + Instruct) · Qwen3-Omni-30B-A3B-Instruct

Hardware: Mac Mini M4 16 GB · NVIDIA RTX 3060 12 GB · RTX 5060 Ti 16 GB

Sources

Next step

Try this in the planner→