Open-Weight LLM VRAM & Hardware Sizing Directory (28 Models)

Load in VRAM Calculator → GGUF Inspector → Generate vLLM / Ollama Config → Compare Live API Token Rates on DevToken →

Open-Weight LLM VRAM Quick Reference (28 Models @ Q4_K_M & FP16)

Showing 28 of 28 models

Verified open-weight LLM architectures with resident parameters, attention mechanism, and baseline VRAM requirements
Model & Family	Topology & Attention	Resident / Active	FP16 (8K ctx)	Q4_K_M (32K ctx)	Minimum Practical Tier	Deep Dive
DeepSeek-R1 (671B MoE) DeepSeek · MIT	MOE MLA	671B / 37B	1289.2 GB	393.7 GB	8× NVIDIA H100 80GB	VRAM Guide →
DeepSeek-V3-0324 (671B MoE) DeepSeek · MIT	MOE MLA	671B / 37B	1289.2 GB	393.7 GB	8× NVIDIA H100 80GB	VRAM Guide →
DeepSeek-R1-Distill-Llama-70B DeepSeek · MIT	DENSE GQA	70.6B	139.3 GB	52.5 GB	NVIDIA A100 80GB	VRAM Guide →
DeepSeek-R1-Distill-Qwen-32B DeepSeek · MIT	DENSE GQA	32.8B	66.1 GB	28.3 GB	RTX 5090 32GB	VRAM Guide →
Qwen3-235B-A22B (MoE) Qwen · Apache-2.0	MOE GQA	235B / 22B	453.5 GB	143.7 GB	Mac Studio 192GB	VRAM Guide →
Qwen3-30B-A3B (MoE) Qwen · Apache-2.0	MOE GQA	30.5B / 3.3B	60.3 GB	21.8 GB	RTX 3090 24GB	VRAM Guide →
Qwen3-32B Qwen · Apache-2.0	DENSE GQA	32.8B	66.1 GB	28.3 GB	RTX 5090 32GB	VRAM Guide →
Qwen3-14B Qwen · Apache-2.0	DENSE GQA	14.8B	30.9 GB	14.8 GB	RTX 4060 Ti 16GB	VRAM Guide →
Qwen3-8B Qwen · Apache-2.0	DENSE GQA	8.2B	18.0 GB	10.4 GB	RTX 3060 12GB	VRAM Guide →
QwQ-32B (Reasoning) Qwen · Apache-2.0	DENSE GQA	32.8B	66.1 GB	28.3 GB	RTX 5090 32GB	VRAM Guide →
Qwen2.5-Coder-32B-Instruct Qwen · Apache-2.0	DENSE GQA	32.8B	66.1 GB	28.3 GB	RTX 5090 32GB	VRAM Guide →
Qwen2.5-72B-Instruct Qwen · Qwen License	DENSE GQA	72.7B	143.4 GB	53.7 GB	NVIDIA A100 80GB	VRAM Guide →
Llama 4 Maverick (400B / 17B-128E MoE) Meta Llama · Llama 4 Community	MOE GQA	402B / 17.2B	774.0 GB	241.0 GB	Mac Studio 512GB	VRAM Guide →
Llama 4 Scout (109B / 17B-16E MoE) Meta Llama · Llama 4 Community	MOE HYBRID_SWA	109B / 17.2B	211.8 GB	67.2 GB	NVIDIA A100 80GB	VRAM Guide →
Llama 3.3 70B Instruct Meta Llama · Llama 3.3 Community	DENSE GQA	70.6B	139.3 GB	52.5 GB	NVIDIA A100 80GB	VRAM Guide →
Llama 3.1 405B Instruct Meta Llama · Llama 3.1 Community	DENSE GQA	405B	782.8 GB	253.2 GB	Mac Studio 512GB	VRAM Guide →
Llama 3.1 8B Instruct Meta Llama · Llama 3.1 Community	DENSE GQA	8.03B	17.6 GB	9.8 GB	RTX 3060 12GB	VRAM Guide →
Gemma 3 27B IT (Multimodal) Google Gemma · Gemma Terms	DENSE HYBRID_SWA	27.4B	54.8 GB	20.1 GB	RTX 3090 24GB	VRAM Guide →
Gemma 3 12B IT (Multimodal) Google Gemma · Gemma Terms	DENSE HYBRID_SWA	12.2B	25.3 GB	10.5 GB	RTX 3060 12GB	VRAM Guide →
Mistral Small 3.1 (24B) Mistral · Apache-2.0	DENSE GQA	24B	48.5 GB	20.2 GB	RTX 3090 24GB	VRAM Guide →
Mistral Large 2411 (123B) Mistral · Mistral Research	DENSE GQA	123B	240.4 GB	84.2 GB	RTX PRO 6000 96GB	VRAM Guide →
Codestral 22B Mistral · Mistral Non-Production	DENSE GQA	22.2B	45.6 GB	21.2 GB	RTX 3090 24GB	VRAM Guide →
Devstral Small (24B SWE Agent) Mistral · Apache-2.0	DENSE GQA	23.6B	47.7 GB	19.9 GB	RTX 3090 24GB	VRAM Guide →
GLM-4-32B-0414 Zhipu GLM · MIT	DENSE GQA	32B	63.1 GB	21.8 GB	RTX 3090 24GB	VRAM Guide →
GLM-4-9B-0414 Zhipu GLM · MIT	DENSE GQA	9.4B	19.5 GB	7.9 GB	RTX 3060 12GB	VRAM Guide →
Microsoft Phi-4 (14B) Microsoft Phi · MIT	DENSE GQA	14.7B	31.0 GB	16.0 GB	RTX 3090 24GB	VRAM Guide →
Microsoft Phi-4-mini-instruct (3.8B) Microsoft Phi · MIT	DENSE GQA	3.84B	9.4 GB	7.3 GB	RTX 3060 12GB	VRAM Guide →
Cohere Command A (111B) Cohere · CC-BY-NC-4.0	DENSE HYBRID_SWA	111B	215.8 GB	68.9 GB	NVIDIA A100 80GB	VRAM Guide →

Browse by Model Family & Architecture

DeepSeek (4)

32.8B – 671B · MLA, GQA

DeepSeek-R1 (671B MoE) · DeepSeek-V3-0324 (671B MoE) · DeepSeek-R1-Distill-Llama-70B · DeepSeek-R1-Distill-Qwen-32B

Qwen (8)

8.2B – 235B · GQA

Qwen3-235B-A22B (MoE) · Qwen3-30B-A3B (MoE) · Qwen3-32B · Qwen3-14B · Qwen3-8B

Meta Llama (5)

8.03B – 405B · GQA, HYBRID_SWA

Llama 4 Maverick (400B / 17B-128E MoE) · Llama 4 Scout (109B / 17B-16E MoE) · Llama 3.3 70B Instruct · Llama 3.1 405B Instruct · Llama 3.1 8B Instruct

Google Gemma (2)

12.2B – 27.4B · HYBRID_SWA

Gemma 3 27B IT (Multimodal) · Gemma 3 12B IT (Multimodal)

Mistral (4)

22.2B – 123B · GQA

Mistral Small 3.1 (24B) · Mistral Large 2411 (123B) · Codestral 22B · Devstral Small (24B SWE Agent)

Zhipu GLM (2)

9.4B – 32B · GQA

GLM-4-32B-0414 · GLM-4-9B-0414

Microsoft Phi (2)

3.84B – 14.7B · GQA

Microsoft Phi-4 (14B) · Microsoft Phi-4-mini-instruct (3.8B)

Cohere (1)

111B – 111B · HYBRID_SWA

Cohere Command A (111B)

How Each Model Page Computes Precision & Context Ceilings

Every model page pre-computes exact weight + KV cache + runtime workspace allocations across FP16, FP8, Q8_0, Q6_K, Q5_K_M, Q4_K_M, AWQ INT4, and IQ4_XS against consumer GPUs, Apple Silicon unified memory, and datacenter HBM accelerators, paired with an interactive quantization and context simulator.

Back-of-the-envelope rules ignore three dominant variables in modern open-weight inference: GGUF super-block quantization scales (where Q4_K_M averages 4.85 bits/weight rather than 4.00), KV cache growth across long context windows and concurrent continuous-batching slots, and Mixture-of-Experts routing where all 671B or 235B parameters sit in VRAM while only 37B or 22B stream per generated token.

VRAMLab evaluates each model from its verified transformer geometry—branching explicitly across Multi-Head Attention (MHA), Grouped-Query Attention (GQA), DeepSeek Multi-head Latent Attention (MLA), and hybrid local/global sliding-window stacks—so capacity plans match real vLLM, SGLang, Ollama, and llama.cpp allocations.