Magistral Small 1.2 24B

24B

Mistral Magistral

Mistral's reasoning model on a Mistral Small 3.2 base, with reasoning wrapped in [THINK] tags. Q4 ~14GB puts explicit chain-of-thought on a 16GB card — the level below the 70B-class reasoners most guides assume.

⬇ 24.4K HF downloads♥ 305 likesmistralai/Magistral-Small-2509· stats from 8/14/2026

Consumer GPUMac / Apple Silicon

131K

Max Context

Quant Variants

GGUF Q6_K

Best Quality

99.4%

Accuracy Retained

Calculate VRAM Hugging Face Compare

Quantization Variants

Per-quant VRAM, quality loss, and inference speed on RTX 4090

Measured = site benchmarks · Estimated = formula · Community = public reports

Format	Level	BPW	VRAM	PPL Loss	Speed	Source	Actions
GGUF	Q4_K_M	4.85	14.3 GB	2.9%	60 tok/s	Community	Calc HF
GGUF	Q5_K_M	5.68	16.8 GB	1.4%	54 tok/s	Estimated	Calc HF
GGUF	Q6_K	6.56	19.4 GB	0.6%	47 tok/s	Estimated	Calc HF
AWQ	INT4	4	13.0 GB	4.0%	76 tok/s	Estimated	Calc HF
EXL2	4.65bpw	4.65	13.9 GB	2.5%	86 tok/s	Estimated	Calc HF

Similar models

Compare with Qwen3-VL 30B-A3B

30B-A3B

Qwen3-VL 30B-A3B Instruct

Alibaba Qwen3-VL

Consumer GPUPro GPU

15.2 GBmin VRAM·99.7%accuracy

Multimodal MoE with only ~3B active parameters, so it stays responsive on Apple unified memory and survives CPU offload far better than a dense 30B. Q4 ~19GB — a 24GB card holds it outright.

27B

Gemma 3 27B IT

Google Gemma 3

Consumer GPUPro GPU

13.0 GBmin VRAM·97.2%accuracy

Gemma 3 large instruct with long context and multimodal support. Q4 ~16GB — dual-GPU or 24GB card with short ctx.

30B-A3B

Qwen3 30B-A3B Instruct

Alibaba Qwen3

Consumer GPUMac / Apple Silicon

17.5 GBmin VRAM·98.9%accuracy

Qwen3 MoE with only 3B active params. Q4 ~19GB file; outperforms QwQ-32B on 16GB cards.

21B MoE

GPT-OSS 20B

OpenAI GPT-OSS

Consumer GPUMac / Apple Silicon

11.9 GBmin VRAM·100.0%accuracy

OpenAI open-weight MoE (21B total / 3.6B active), shipped natively in MXFP4 — ~12.8GB runs on a 16GB card with no quality tax. Only 3.6B active params means CPU-offload stays usable.

How to actually run this

Deployment guides for this model and this class of hardware.