Microsoft Phi-4 (14B) VRAM Requirements, GGUF Sizes & GPU Gui…

Interactive Microsoft Phi-4 (14B) Memory & Context Simulator

Weight Quantization (bpw)

KV Cache Precision

Target GPU / Unified Memory Tier

Devices (TP)

Context Length (tokens) 32,768 tokens

Memory Breakdown & Bandwidth Ceiling

Fits Comfortably

Model Weights: 51.9% KV: 39.1% Overhead: 9%

Total Required Memory

16.00 GB

Per-Device Allocation: 16.00 GB

Model Weights

8.30 GB

KV Cache (FP16): 6.25 GB

Est. Single-Stream Decode

37.5 tok/s

Runtime & TP Overhead: 1.45 GB

Max Context Before OOM

71,519

tokens

Architecture KV compression saves 75.0% vs naive MHA FP16 (25.00 GB).

Precision & Quantization VRAM Matrix (8K / 32K / 128K Context)

Total GPU memory required across weight precisions and context lengths (batch = 1, FP16 KV cache)
Precision / Format	Effective bpw	Weights Only	Total @ 8K ctx	Total @ 32K ctx	Total @ 128K ctx	Recommended Tier (@ 32K)
FP16 / BF16	16.00 bpw	27.38 GB	30.97 GB	35.65 GB	54.40 GB	Apple M4 Pro 48GB
FP8	8.50 bpw	14.55 GB	17.75 GB	22.43 GB	41.18 GB	RTX 3090 24GB
Q8_0	8.50 bpw	14.55 GB	17.75 GB	22.43 GB	41.18 GB	RTX 3090 24GB
Q6_K	6.56 bpw	11.23 GB	14.33 GB	19.02 GB	37.77 GB	RTX 3090 24GB
Q5_K_M	5.69 bpw	9.74 GB	12.79 GB	17.48 GB	36.23 GB	RTX 3090 24GB
Q4_K_M	4.85 bpw	8.30 GB	11.31 GB	16.00 GB	34.75 GB	RTX 3090 24GB
AWQ INT4	4.50 bpw	7.70 GB	10.70 GB	15.38 GB	34.13 GB	RTX 4060 Ti 16GB
IQ4_XS	4.25 bpw	7.27 GB	10.26 GB	14.94 GB	33.69 GB	RTX 4060 Ti 16GB

Hardware Compatibility & Decode Speed (Microsoft Phi-4 (14B) @ Q4_K_M, 32K Context)

Fit status, memory utilization, and estimated decode throughput across flagship GPUs and Apple Silicon
Hardware Tier	Memory & Bandwidth	Required (32K)	Fit Verdict	Est. Decode Speed
RTX 3090 24GB	24 GB · 936 GB/s	16.00 GB (67%)	Fits Comfortably	37.5 tok/s
RTX 4090 24GB	24 GB · 1008 GB/s	16.00 GB (67%)	Fits Comfortably	40.4 tok/s
RTX 5090 32GB	32 GB · 1792 GB/s	16.00 GB (50%)	Fits Comfortably	71.8 tok/s
Apple M-Max 64GB	64 GB · 546 GB/s	15.54 GB (24%)	Fits Comfortably	20.6 tok/s
Mac Studio 128GB	128 GB · 546 GB/s	15.54 GB (12%)	Fits Comfortably	20.6 tok/s
Mac Studio 192GB	192 GB · 800 GB/s	15.54 GB (8%)	Fits Comfortably	30.2 tok/s
NVIDIA A100 80GB	80 GB · 2039 GB/s	16.00 GB (20%)	Fits Comfortably	86.5 tok/s
NVIDIA H100 80GB	80 GB · 3350 GB/s	16.00 GB (20%)	Fits Comfortably	142.0 tok/s

Verified Transformer Architecture Parameters

Checkpoint ID: microsoft/phi-4
Ollama Tag: phi4:14b
License / Release: MIT · 2025-Q1
Topology & Attention: Dense Transformer
Attention Mechanism: Grouped-Query Attention (GQA)
Total Params (B): 14.7B
Active Params / Token (B): 14.7B
Hidden Layers / Hidden Size: 40 / 5120
Query Heads / KV Heads: 40Q / 10KV (d=128)
Vocab / Max Position: 100,352 / 16,384

Quantized formats store per-block scale factors and zero-points alongside packed integer weights. A 4-bit GGUF Q4_K_M checkpoint averages 4.85 effective bits per parameter (not 4.00), while Q8_0 averages 8.50 bpw. For Mixture-of-Experts models, all routed and shared experts remain resident in memory to prevent token-by-token host paging.

Standard Grouped-Query Attention allocates 2 × L × H_kv × d_head × S × B × bytes_kv across layers. DeepSeek Multi-head Latent Attention (MLA) compresses keys and values into L × (d_kv_lora + d_qk_rope) × S × B × bytes_kv, slashing long-context memory by over 90%. Hybrid sliding-window models cap local-attention layers at min(S, window_size) while scaling global layers linearly with S.

Quick-Start vLLM & Ollama Commands for Microsoft Phi-4 (14B)

vllm serve microsoft/phi-4 \
  --tensor-parallel-size 1 \
  --max-model-len 32768 \
  --max-num-seqs 8 \
  --gpu-memory-utilization 0.95 \
  --kv-cache-dtype fp8 \
  --enable-prefix-caching \
  --port 8000

# Save as Modelfile, then run: ollama create phi-4-14b-custom -f Modelfile
FROM phi4:14b
PARAMETER num_ctx 32768
PARAMETER num_batch 512
PARAMETER flash_attn true

# Environment variables for ollama serve:
# OLLAMA_NUM_PARALLEL=8 OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 ollama serve