Back to Quant Hub

Zephyr 7B Beta

7B

HuggingFaceH4

DPO-aligned Mistral 7B. Classic choice for helpful, harmless chat baselines.

Consumer GPUMac / Apple SiliconCPU / VPS

33K

Max Context

2

Quant Variants

GGUF Q6_K

Best Quality

99.0%

Accuracy Retained

Calculate VRAM Hugging Face Compare

Quantization Variants

Per-quant VRAM, quality loss, and inference speed on RTX 4090

Format	Level	BPW	VRAM	PPL Loss	Speed	Actions
GGUF	Q4_K_M	4.85	5.2 GB	3.3%	155 tok/s	Calc HF
GGUF	Q6_K	6.56	6.8 GB	1.0%	132 tok/s	Calc HF