Gemma 3 12B IT

12B

Google Gemma 3

Mid-size Gemma 3 with vision. Fits 16GB at Q4; excellent multilingual chat.

Consumer GPUMac / Apple Silicon

131K

Max Context

3

Quant Variants

GGUF Q5_K_M

Best Quality

98.7%

Accuracy Retained

Calculate VRAM Hugging Face Compare

Quantization Variants

Per-quant VRAM, quality loss, and inference speed on RTX 4090

Measured = site benchmarks · Estimated = formula · Community = public reports

Format	Level	BPW	VRAM	PPL Loss	Speed	Source	Actions
GGUF	Q4_K_M	4.85	8.8 GB	2.9%	105 tok/s	Estimated	Calc HF
GGUF	Q5_K_M	5.68	10.2 GB	1.3%	92 tok/s	Estimated	Calc HF
AWQ	INT4	4	8.0 GB	3.9%	128 tok/s	Estimated	Calc HF

Similar models

Compare with Gemma 3

Gemma 3 4B IT

Google Gemma 3

Consumer GPUMac / Apple Silicon

3.0 GBmin VRAM·99.8%accuracy

Google Gemma 3 multimodal 4B. 128K context; strong vision + text on 8GB cards.

Gemma 3 27B IT

Google Gemma 3

Consumer GPUPro GPU

13.0 GBmin VRAM·97.2%accuracy

Gemma 3 large instruct with long context and multimodal support. Q4 ~16GB — dual-GPU or 24GB card with short ctx.

Qwen3-VL 8B Instruct

Alibaba Qwen3-VL

Consumer GPUMac / Apple Silicon

5.3 GBmin VRAM·99.7%accuracy

Current-generation vision-language model that still fits a single 8–12GB card at Q4 (~5.9GB). The realistic multimodal option for people without a 24GB GPU — note the vision encoder adds VRAM the KV-cache math below does not model.

Qwen2.5 14B Instruct

Alibaba Qwen2.5

Consumer GPUMac / Apple Silicon

9.2 GBmin VRAM·98.6%accuracy

The sweet spot between performance and resource usage. 16GB VRAM with Q4.

How to actually run this

Deployment guides for this model and this class of hardware.