Gemma 3 4B IT

4B

Google Gemma 3

Google Gemma 3 multimodal 4B. 128K context; strong vision + text on 8GB cards.

Consumer GPUMac / Apple SiliconCPU / VPS

131K

Max Context

3

Quant Variants

GGUF Q8_0

Best Quality

99.8%

Accuracy Retained

Calculate VRAMHugging FaceCompare

Quantization Variants

Per-quant VRAM, quality loss, and inference speed on RTX 4090

Measured = site benchmarks · Estimated = formula · Community = public reports

FormatLevelBPWVRAMPPL LossSpeedSourceActions
GGUFQ4_K_M4.853.4 GB3.2%175 tok/sEstimated
CalcHF
GGUFQ8_08.55.2 GB0.2%145 tok/sEstimated
CalcHF
AWQINT443.0 GB4.2%210 tok/sEstimated
CalcHF

Similar models

Compare with Gemma 3
12B

Gemma 3 12B IT

Google Gemma 3

Consumer GPUMac / Apple Silicon
8.0 GBmin VRAM·98.7%accuracy

Mid-size Gemma 3 with vision. Fits 16GB at Q4; excellent multilingual chat.

27B

Gemma 3 27B IT

Google Gemma 3

Consumer GPUPro GPU
13.0 GBmin VRAM·97.2%accuracy

Gemma 3 large instruct with long context and multimodal support. Q4 ~16GB — dual-GPU or 24GB card with short ctx.

8B

Llama 3.1 8B Instruct

Meta Llama 3.1

Consumer GPUMac / Apple Silicon
3.2 GBmin VRAM·99.9%accuracy

Meta's flagship 8B model with 128K context. Best-in-class for local deployment.

7B

Qwen2.5 7B Instruct

Alibaba Qwen2.5

Consumer GPUMac / Apple Silicon
4.8 GBmin VRAM·99.3%accuracy

Alibaba's highly optimized 7B. Punches well above its weight, especially in coding.

How to actually run this

Deployment guides for this model and this class of hardware.