NVIDIA RTX 50

Can RTX 5090 run Gemma 4?

32 GB VRAM profile for desktop local Gemma 4 runs. The checker below ranks Gemma 4 E2B, E4B, 12B, 26B A4B, and 31B specifically for RTX 5090.

GPU
VRAM32 GB
BW~1792 GB/s
RAM16 GB
CORES21760
WebGPU
6RUNS GREAT
1RUNS WELL
0DECENT
0TIGHT FIT
0BARELY RUNS
0TOO HEAVY
Gemma 4 only
Gemma 4 12BbestQ4_K_M

16GB+ unified-memory Macs and desktops with room for context.

Memory: 7.1 GBUse: 22%Speed: ~177 tok/sScore: 97/100
RUNS GREAT
97/100

The instruct 12B when you want chat, vision, and a 256K window on 16GB+ machines.

Memory: 7.1 GBUse: 22%Speed: ~177 tok/sScore: 97/100
RUNS GREAT
97/100

Better phones, Apple silicon laptops, and mainstream desktops.

Memory: 4.6 GBUse: 14%Speed: ~273 tok/sScore: 96/100
RUNS GREAT
96/100

Phones, tablets, old laptops, and the fastest local proof.

Memory: 3.1 GBUse: 10%Speed: ~405 tok/sScore: 95/100
RUNS GREAT
95/100

Gemma-family MoE when you want the newer diffusion decode path and multimodal prompts.

Memory: 13.8 GBUse: 43%Speed: ~91 tok/sScore: 89/100
RUNS GREAT
89/100
Gemma 4 26B-A4B IT4B activeQ4_K_M

Strong desktop GPUs and disciplined 24GB+ Apple silicon setups.

Memory: 15.8 GBUse: 49%Speed: ~79 tok/sScore: 86/100
RUNS GREAT
86/100

High-end workstations where local quality matters more than convenience.

Memory: 19 GBUse: 59%Speed: ~66 tok/sScore: 78/100
RUNS WELL
78/100

Full Model Directory

Explore 130+ Other Open-Weight Models on This Hardware

Compare Llama 3.3, Qwen 3, DeepSeek-R1, Mistral, and Phi on RTX 5090, or run a side-by-side GPU and Apple Silicon benchmark comparison.

Hardware Memory Analysis

Usable Memory Pool & Bandwidth Profile for RTX 5090

RTX 5090 provides 32 GB of dedicated graphics memory at ~1792 GB/s memory bandwidth. Keeping quantized model weights plus KV cache within ~85% of dedicated VRAM avoids silent Windows WDDM or Linux PCIe spillover into ~50 GB/s system DDR5 RAM, where throughput drops sharply.

Practical Model Budget

32 GB

Memory Bandwidth

~1792 GB/s

Recommended Checkpoint

Gemma 4 31B (Q4_K_M)

Gemma 4 Checkpoint Compatibility Matrix on RTX 5090

Evaluated at Q4_K_M baseline weights plus 0.5 GB baseline runtime overhead against RTX 5090's 32 GB usable budget. Leave at least 1.5–4 GB spare headroom for multi-turn KV cache expansion.

ModelQ4_K_M MemoryEst. SpeedGradePractical Headroom Verdict
Gemma 4 E2B3.1 GB~405 tok/sRuns greatComfortable fit (+28.9 GB spare for long context)
Gemma 4 E4B4.6 GB~273 tok/sRuns greatComfortable fit (+27.4 GB spare for long context)
Gemma 4 12B7.1 GB~177 tok/sRuns greatComfortable fit (+24.9 GB spare for long context)
Gemma 4 12B IT7.1 GB~177 tok/sRuns greatComfortable fit (+24.9 GB spare for long context)
Gemma 4 26B-A4B IT15.8 GB~79 tok/sRuns greatComfortable fit (+16.2 GB spare for long context)
DiffusionGemma 26B-A4B IT13.8 GB~91 tok/sRuns greatComfortable fit (+18.2 GB spare for long context)
Gemma 4 31B19 GB~66 tok/sRuns wellComfortable fit (+13 GB spare for long context)

NVIDIA CUDA, Flash Attention & PCIe Limits for RTX 5090

  • Run via LM Studio CUDA runtime, Ollama (`OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0`), or `llama.cpp` with `-ngl 99`.
  • Verify 100% GPU residency with `ollama ps` or Windows Task Manager Dedicated GPU Memory; if Shared GPU Memory spikes, step down one quant or cap context at 4K–8K.
  • If experimenting with Gemma 4 26B-A4B MoE on 12–16 GB RTX cards, test `llama-server -ot ".ffn_.*_exps.=CPU"` paired with 32 GB+ system RAM.

Frequently Asked Questions for RTX 5090

What is the best Gemma 4 model to run on RTX 5090?
Based on RTX 5090's 32 GB practical memory budget and ~1792 GB/s bandwidth, Gemma 4 31B (Q4_K_M, 19 GB) is the strongest comfortable fit (~66 tok/s).
Can RTX 5090 run Gemma 4 26B-A4B MoE locally?
Yes. At Q4_K_M (15.8 GB), Gemma 4 26B-A4B runs cleanly on RTX 5090 with 16.2 GB remaining for KV cache and runtime buffers.
How fast does RTX 5090 generate tokens with Gemma 4?
With ~1792 GB/s memory bandwidth, Gemma 4 31B at Q4_K_M averages roughly 66 tokens per second when fully resident without swap or PCIe spillover.
Which quantization format works best on RTX 5090?
Start with Q4_K_M (~4.5 bits/weight) to keep memory headroom free for 8K+ context windows. Step up to Q6_K or Q8_0 only if Q4_K_M consumes well under 60% of available VRAM or unified memory.