Quant Hub

Meta Llama 3.1

Meta's flagship 8B model with 128K context. Best-in-class for local deployment.

Formats

Meta Llama 3.1

Meta's frontier 70B model. Requires 40GB+ VRAM; dual 3090 or M2 Ultra.

Formats

Meta Llama 3.2

Tiny but capable. Runs on 4GB VRAM or 8GB RAM, even on phones via llama.cpp.

Formats

Alibaba Qwen2.5

Alibaba's highly optimized 7B. Punches well above its weight, especially in coding.

Formats

Alibaba Qwen2.5

The sweet spot between performance and resource usage. 16GB VRAM with Q4.

Formats

Alibaba Qwen2.5

Near-GPT-4 reasoning on a 24GB VRAM card (Q4_K_S). Groundbreaking value.

Formats

DeepSeek

MoE architecture coding model. Active params ~2.4B, total ~16B. Exceptional code quality.

Formats

Microsoft Phi

Microsoft's tiny powerhouse. Best 4B model for on-device deployment.

Formats

Mistral AI

Mistral + NVIDIA collaboration. 128K context, excellent multilingual support.

Formats

Google Gemma 2

Google's compact Gemma 2 with sliding window attention. Punches above 9B.

Formats

Alibaba Qwen2.5

Flagship Qwen2.5. Requires dual 4090 or A100 80G. Exceptional reasoning at scale.

Formats

DeepSeek

R1 reasoning distilled into 14B. Huge community interest; excellent chain-of-thought.

Formats

Meta Llama 3.3

Latest Meta 70B with improved multilingual. Drop-in upgrade from Llama 3.1 70B.

Formats

Mistral AI

Mistral's efficient 24B. Strong multilingual; fits on 24GB with Q4.

Formats

Alibaba Qwen2.5

Top-tier open coding model. HumanEval competitive with GPT-4o on 32B scale.

Formats

Alibaba Qwen2.5

Best 7B coding model. Ideal for local dev assistants on 8–16GB VRAM.

Formats

Alibaba Qwen2.5

Tiny Qwen2.5 for edge devices. Runs on 4GB VRAM or Raspberry Pi class hardware.

Formats

Meta Llama 3.2

Ultra-light Llama for mobile and embedded. Sub-2GB VRAM with Q4.

Formats

DeepSeek

R1 reasoning in Llama 70B architecture. Top open reasoning model for dual-GPU setups.

Formats

Mistral AI

Mistral's dedicated code model. 80+ language support, Fill-in-the-Middle capable.

Formats

Mistral AI

Classic MoE model. ~13B active params per token; needs 32GB+ VRAM for Q4.

Formats

Cohere

Cohere's RAG-optimised model. Excellent retrieval-augmented generation.

Formats

01.AI Yi

01.AI's strong bilingual (EN/ZH) model. Competitive with Qwen 32B.

Formats

Upstage

Depth-upscaled 10.7B punching above weight. Strong on reasoning benchmarks.

Formats

BigCode

BigCode's open code model trained on 600+ languages. Great for polyglot dev.

Formats

Meta Llama 3.2

Multimodal Llama with image understanding. Vision encoder adds ~2GB VRAM overhead.

Formats

Alibaba Qwen2

Vision-language model with video understanding. Strong OCR and chart reading.

Formats

NousResearch

Fine-tuned Llama 3.1 8B with improved roleplay and instruction following.

Formats

Microsoft / WizardLM

Evol-Instruct fine-tuned Mistral-based 7B. Strong complex instruction handling.

Formats

IBM Granite

IBM's enterprise-grade 8B. Strong RAG and tool-use; permissive Apache 2.0 license.

Formats

Google Gemma 2

Ultra-compact Gemma 2. Runs on 4GB VRAM; great for edge prototyping.

Formats

Google Gemma 2

Largest open Gemma 2. Strong reasoning; needs 24GB+ VRAM at Q4.

Formats

Alibaba Qwen2.5

Smallest Qwen2.5. Ideal for Raspberry Pi, phones, and ultra-low-latency demos.

Formats

Alibaba Qwen2.5

Tiny Qwen with 128K context. Surprisingly capable for summarisation and chat.

Formats

Microsoft Phi

Microsoft's mid-size Phi-3. Excellent quality-per-GB on 16GB cards.

Formats

Microsoft Phi

Latest Phi mini with improved math and code. Strong 4B-class performer.

Formats

Mistral AI

Classic Mistral 7B v0.3. Still a reliable baseline for local chat APIs.

Formats

DeepSeek

MoE general model (~2.4B active). Long context and strong multilingual chat.

Formats

DeepSeek

R1 reasoning in a 7B footprint. Best value for 8–12GB VRAM CoT experiments.

Formats

DeepSeek

R1 distilled to 32B. Near-frontier reasoning on a single 24GB card (Q3/Q4).

Formats

Meta Llama 3.2

Flagship multimodal Llama. Requires dual 4090 or A100; vision adds ~3GB overhead.

Formats

Allen AI OLMo

Fully open training pipeline from Allen AI. Great for reproducibility research.

Formats

Shanghai AI Lab

Strong bilingual (EN/ZH) 7B from Shanghai AI Lab. Competitive with Qwen 7B.

Formats

Shanghai AI Lab

Mid-size InternLM2 with excellent Chinese comprehension. Fits 24GB at Q4.

Formats

Cohere For AI

Multilingual specialist covering 23 languages. Strong for non-English local apps.

Formats

OpenChat

C-RLFT fine-tuned Llama 3.1 8B. Known for natural conversational tone.

Formats

HuggingFaceH4

DPO-aligned Mistral 7B. Classic choice for helpful, harmless chat baselines.

Formats

Stability AI

Stability AI's 12B chat model. Solid general-purpose option for 16GB GPUs.

Formats

TII UAE

Technology Innovation Institute's latest Falcon. Good multilingual and code mix.

Formats

AI21 Labs

Hybrid SSM-Transformer with 256K context. Efficient long-document QA on 16GB.

Formats

Databricks

MoE flagship (~36B active). Needs multi-GPU; strong code and reasoning at scale.

Formats