Back to Quant Hub

Granite 3.1 8B Instruct

8B

IBM Granite

IBM's enterprise-grade 8B. Strong RAG and tool-use; permissive Apache 2.0 license.

⬇ 2.0K HF downloads♥ 12 likesbartowski/granite-3.1-8b-instruct-GGUF· stats from 6/24/2026
Consumer GPUMac / Apple SiliconCPU / VPS

131K

Max Context

2

Quant Variants

GGUF Q4_K_M

Best Quality

97.0%

Accuracy Retained

Calculate VRAMHugging FaceCompare

Quantization Variants

Per-quant VRAM, quality loss, and inference speed on RTX 4090

FormatLevelBPWVRAMPPL LossSpeedActions
GGUFQ4_K_M4.855.8 GB3.0%142 tok/s
CalcHF
GPTQINT445.0 GB4.5%195 tok/s
CalcHF