Mistral Small 24B Instruct

24B

Mistral AI

Mistral's efficient 24B. Strong multilingual; fits on 24GB with Q4.

⬇ 4.4K HF downloads♥ 53 likesbartowski/Mistral-Small-Instruct-2409-GGUF· stats from 6/24/2026

Consumer GPUPro GPU

33K

Max Context

Quant Variants

EXL2 4.65bpw

Best Quality

97.8%

Accuracy Retained

Quantization Variants

Per-quant VRAM, quality loss, and inference speed on RTX 4090

Format	Level	BPW	VRAM	PPL Loss	Speed	Actions
GGUF	Q4_K_M	4.85	14.2 GB	2.9%	62 tok/s	Calc HF
AWQ	INT4	4	14.2 GB	3.8%	78 tok/s	Calc HF
EXL2	4.65bpw	4.65	13.5 GB	2.2%	88 tok/s	Calc HF