Can I Run Gemma 4 locally?

Find out which Gemma 4 models your machine can actually run — or jump to Download Gemma 4 · Android APK Guide · Compare Models & Hardware Fit.

GPU
VRAM16 GB
BW~288 GB/s
RAM32 GB
CORES--
WebGPU
1RUNS GREAT
1RUNS WELL
2DECENT
0TIGHT FIT
3BARELY RUNS
0TOO HEAVY
Gemma 4 only
Gemma 4 E2BbestQ4_K_M

Phones, tablets, old laptops, and the fastest local proof.

Memory: 3.1 GBUse: 19%Speed: ~65 tok/sScore: 91/100
RUNS GREAT
91/100

Better phones, Apple silicon laptops, and mainstream desktops.

Memory: 4.6 GBUse: 29%Speed: ~44 tok/sScore: 83/100
RUNS WELL
83/100

16GB+ unified-memory Macs and desktops with room for context.

Memory: 7.1 GBUse: 44%Speed: ~28 tok/sScore: 67/100
DECENT
67/100

The instruct 12B when you want chat, vision, and a 256K window on 16GB+ machines.

Memory: 7.1 GBUse: 44%Speed: ~28 tok/sScore: 67/100
DECENT
67/100

Gemma-family MoE when you want the newer diffusion decode path and multimodal prompts.

Memory: 13.8 GBUse: 86%Speed: ~15 tok/sScore: 29/100
BARELY RUNS
29/100
Gemma 4 26B-A4B IT4B activeQ4_K_M

Strong desktop GPUs and disciplined 24GB+ Apple silicon setups.

Memory: 15.8 GBUse: 99%Speed: ~13 tok/sScore: 24/100
BARELY RUNS
24/100

High-end workstations where local quality matters more than convenience.

Memory: 19 GBUse: 119%Speed: ~5 tok/sScore: 11/100
BARELY RUNS
11/100
Tool 2

Check other local AI models

Evaluate the same hardware profile across open-weight LLM, reasoning, coding, and multimodal models beyond Gemma 4.

Other models
Phi-3.5 MiniMicrosoftMITQ2_K

Microsoft's efficient small model with long context

Memory: 1.7 GBUse: 11%Speed: ~119 tok/sScore: 95/100
RUNS GREAT
95/100
Phi-4 Mini ReasoningMicrosoftMITQ2_K

Lightweight reasoning model

Memory: 1.7 GBUse: 11%Speed: ~119 tok/sScore: 95/100
RUNS GREAT
95/100
Qwen 3 4BAlibabaApache 2.0Q2_K

Compact Qwen 3 for general tasks

Memory: 1.8 GBUse: 11%Speed: ~112 tok/sScore: 95/100
RUNS GREAT
95/100
Gemma 3 4BGoogleGemmaQ2_K

Multimodal Gemma with 128K context

Memory: 1.8 GBUse: 11%Speed: ~112 tok/sScore: 95/100
RUNS GREAT
95/100
Qwen 3.5 4BAlibabaApache 2.0Q4_K_M

Small multimodal Qwen 3.5

Memory: 2.5 GBUse: 16%Speed: ~81 tok/sScore: 95/100
RUNS GREAT
95/100
Gemma 4 E2BGoogleGemmaQ2_K

Gemma 4 efficient base model (official)

Memory: 2.1 GBUse: 13%Speed: ~96 tok/sScore: 95/100
RUNS GREAT
95/100
Mistral 7B v0.3Mistral AIApache 2.0Q2_K

High-quality 7B with sliding window attention

Memory: 2.7 GBUse: 17%Speed: ~75 tok/sScore: 95/100
RUNS GREAT
95/100
Qwen 2.5 7BAlibabaApache 2.0Q2_K

Strong multilingual and coding capabilities

Memory: 2.7 GBUse: 17%Speed: ~75 tok/sScore: 95/100
RUNS GREAT
95/100
Qwen 2.5 Coder 7BAlibabaApache 2.0Q2_K

Dedicated coding model

Memory: 2.7 GBUse: 17%Speed: ~75 tok/sScore: 95/100
RUNS GREAT
95/100
DeepSeek R1 Distill 7BDeepSeekMITQ2_K

R1 reasoning distilled into Qwen 7B

Memory: 2.7 GBUse: 17%Speed: ~75 tok/sScore: 95/100
RUNS GREAT
95/100
Qwen3-VL 4BAlibabaApache 2.0Q2_K

Compact dedicated vision-language model — OCR & image chat on edge

Memory: 1.9 GBUse: 12%Speed: ~106 tok/sScore: 95/100
RUNS GREAT
95/100
Llama 3.2 3BMetaLlama 3.2 CommunityQ2_K

Lightweight Llama for mobile and edge

Memory: 1.5 GBUse: 9%Speed: ~134 tok/sScore: 94/100
RUNS GREAT
94/100
SmolLM3 3BHuggingFaceApache 2.0Q2_K

Lightweight multilingual reasoning

Memory: 1.5 GBUse: 9%Speed: ~134 tok/sScore: 94/100
RUNS GREAT
94/100
Granite 4.1 3BIBMApache 2.0Q2_K

Compact enterprise model for edge and constrained environments

Memory: 1.5 GBUse: 9%Speed: ~134 tok/sScore: 94/100
RUNS GREAT
94/100
Ministral 3 3BMistral AIApache 2.0Q2_K

Current-gen tiny Ministral — edge chat with 256K context

Memory: 1.5 GBUse: 9%Speed: ~134 tok/sScore: 94/100
RUNS GREAT
94/100
Qwen 2.5 Coder 1.5BAlibabaApache 2.0Q2_K

Ultra-lightweight coding model

Memory: 1 GBUse: 6%Speed: ~202 tok/sScore: 93/100
RUNS GREAT
93/100
DeepSeek R1 1.5BDeepSeekMITQ2_K

Tiny reasoning model distilled from R1

Memory: 1 GBUse: 6%Speed: ~202 tok/sScore: 93/100
RUNS GREAT
93/100
Qwen 3 1.7BAlibabaApache 2.0Q2_K

Compact multilingual Qwen 3

Memory: 1 GBUse: 6%Speed: ~202 tok/sScore: 93/100
RUNS GREAT
93/100
Qwen 3.5 2BAlibabaApache 2.0Q4_K_M

Small multimodal Qwen 3.5

Memory: 1.5 GBUse: 9%Speed: ~134 tok/sScore: 93/100
RUNS GREAT
93/100
Gemma 2 2BGoogleGemmaQ2_K

Google's compact open model

Memory: 1.1 GBUse: 7%Speed: ~183 tok/sScore: 93/100
RUNS GREAT
93/100
Qwen 3.5 0.8BAlibabaApache 2.0Q4_K_M

Ultra-tiny model for embedded and edge

Memory: 0.9 GBUse: 6%Speed: ~224 tok/sScore: 92/100
RUNS GREAT
92/100
Llama 3.2 1BMetaLlama 3.2 CommunityQ2_K

Meta's smallest Llama for edge devices

Memory: 0.8 GBUse: 5%Speed: ~252 tok/sScore: 92/100
RUNS GREAT
92/100
Gemma 3 1BGoogleGemmaQ2_K

Google's tiny Gemma for on-device

Memory: 0.8 GBUse: 5%Speed: ~252 tok/sScore: 92/100
RUNS GREAT
92/100
TinyLlama 1.1BCommunityApache 2.0Q2_K

Ultralight model for constrained devices

Memory: 0.9 GBUse: 6%Speed: ~224 tok/sScore: 92/100
RUNS GREAT
92/100
Gemma 4 E4BGoogleGemmaQ2_K

Gemma 4 balanced base model (official)

Memory: 3.1 GBUse: 19%Speed: ~65 tok/sScore: 92/100
RUNS GREAT
92/100
Llama 3.1 8BMetaLlama 3.1 CommunityQ2_K

Meta's versatile 8B — great quality/speed ratio

Memory: 3.1 GBUse: 19%Speed: ~65 tok/sScore: 92/100
RUNS GREAT
92/100
Qwen 3 8BAlibabaApache 2.0Q2_K

Qwen 3 with thinking mode support

Memory: 3.1 GBUse: 19%Speed: ~65 tok/sScore: 92/100
RUNS GREAT
92/100
Ministral 8BMistral AIMRLQ2_K

Mistral's efficient 8B model

Memory: 3.1 GBUse: 19%Speed: ~65 tok/sScore: 92/100
RUNS GREAT
92/100
Granite 4.1 8BIBMApache 2.0Q2_K

Balanced general-purpose enterprise model

Memory: 3.1 GBUse: 19%Speed: ~65 tok/sScore: 92/100
RUNS GREAT
92/100
Qwen 3 0.6BAlibabaApache 2.0Q4_K_M

Ultra-light Qwen 3 model for constrained devices

Memory: 0.8 GBUse: 5%Speed: ~252 tok/sScore: 91/100
RUNS GREAT
91/100
Gemma 4 E2B ITGoogleApache 2.0Q4_K_M

Gemma 4 efficient instruct model (official)

Memory: 3.1 GBUse: 19%Speed: ~65 tok/sScore: 91/100
RUNS GREAT
91/100
Qwen3-VL 8BAlibabaApache 2.0Q2_K

The community-favourite local VLM — superb OCR, receipts & captioning

Memory: 3.3 GBUse: 21%Speed: ~61 tok/sScore: 91/100
RUNS GREAT
91/100
Gemma 2 9BGoogleGemmaQ2_K

Google's best mid-size open model

Memory: 3.4 GBUse: 21%Speed: ~59 tok/sScore: 90/100
RUNS GREAT
90/100
GLM-4 9BZhipu AIGLM-4Q2_K

Multilingual model supporting 26 languages with 128K context

Memory: 3.4 GBUse: 21%Speed: ~59 tok/sScore: 90/100
RUNS GREAT
90/100
Nemotron Nano 9B v2NVIDIANVIDIA OpenQ2_K

Hybrid Mamba2 architecture for reasoning

Memory: 3.4 GBUse: 21%Speed: ~59 tok/sScore: 90/100
RUNS GREAT
90/100
Ornith 1.0 9BDeepReinforceMITQ2_K

Self-improving agentic coding model optimized for terminal and software engineering tasks

Memory: 3.4 GBUse: 21%Speed: ~59 tok/sScore: 90/100
RUNS GREAT
90/100
Llama 3.2 11B VisionMetaLlama 3.2 CommunityQ2_K

Multimodal vision and text model

Memory: 4 GBUse: 25%Speed: ~50 tok/sScore: 87/100
RUNS GREAT
87/100
Gemma 4 12BGoogleGemmaQ2_K

Gemma 4 mid-size base model (official)

Memory: 4.3 GBUse: 27%Speed: ~47 tok/sScore: 85/100
RUNS GREAT
85/100
Gemma 3 12BGoogleGemmaQ2_K

Multimodal Gemma with 128K context

Memory: 4.3 GBUse: 27%Speed: ~47 tok/sScore: 85/100
RUNS GREAT
85/100
Mistral Nemo 12BMistral AIApache 2.0Q2_K

Multilingual 12B with 128K context

Memory: 4.3 GBUse: 27%Speed: ~47 tok/sScore: 85/100
RUNS GREAT
85/100
Gemma 4 12B ITGoogleApache 2.0Q2_K

Gemma 4 mid-size instruct — multimodal any-to-any

Memory: 4.3 GBUse: 27%Speed: ~47 tok/sScore: 85/100
RUNS GREAT
85/100
Gemma 4 E4B ITGoogleApache 2.0Q4_K_M

Gemma 4 balanced instruct model (official)

Memory: 4.6 GBUse: 29%Speed: ~44 tok/sScore: 83/100
RUNS WELL
83/100
Qwen 2.5 14BAlibabaApache 2.0Q2_K

Excellent quality for its size class

Memory: 5 GBUse: 31%Speed: ~40 tok/sScore: 82/100
RUNS WELL
82/100
Phi-4 14BMicrosoftMITQ2_K

Microsoft's reasoning-focused model

Memory: 5 GBUse: 31%Speed: ~40 tok/sScore: 82/100
RUNS WELL
82/100
Qwen 3 14BAlibabaApache 2.0Q2_K

Strong all-rounder with thinking mode

Memory: 5 GBUse: 31%Speed: ~40 tok/sScore: 82/100
RUNS WELL
82/100

R1 reasoning distilled into Qwen 14B

Memory: 5 GBUse: 31%Speed: ~40 tok/sScore: 82/100
RUNS WELL
82/100
Ministral 3 14BMistral AIApache 2.0Q2_K

Current-gen Ministral mid-size — local assistant with 256K context

Memory: 5 GBUse: 31%Speed: ~40 tok/sScore: 82/100
RUNS WELL
82/100
Qwen 3.5 9BAlibabaApache 2.0Q4_K_M

Multimodal Qwen 3.5 mid-size

Memory: 5.1 GBUse: 32%Speed: ~40 tok/sScore: 80/100
RUNS WELL
80/100
GPT-OSS 20BOpenAIApache 2.03.6B activeQ2_K

OpenAI's open-weight MoE with configurable reasoning

Memory: 7.2 GBUse: 45%Speed: ~28 tok/sScore: 68/100
DECENT
68/100
LFM2 24BLiquid AILiquid AI2.3B activeQ2_K

Hybrid MoE with convolution+attention layers — 2.3B active

Memory: 8.2 GBUse: 51%Speed: ~25 tok/sScore: 63/100
DECENT
63/100
Devstral Small 2 24BMistral AIApache 2.0Q2_K

Coding-focused model with 256K context — 68% SWE-bench

Memory: 8.2 GBUse: 51%Speed: ~25 tok/sScore: 63/100
DECENT
63/100
Mistral Small 3.1 24BMistral AIApache 2.0Q2_K

Multimodal Mistral with vision support

Memory: 8.2 GBUse: 51%Speed: ~25 tok/sScore: 63/100
DECENT
63/100
DiffusionGemma 26B-A4B ITGoogleApache 2.04B activeQ2_K

Discrete diffusion MoE — 1100+ tok/s on H100, multimodal (text/image/video)

Memory: 8.8 GBUse: 55%Speed: ~23 tok/sScore: 60/100
DECENT
60/100
Gemma 2 27BGoogleGemmaQ2_K

Google's largest Gemma 2 model

Memory: 9.1 GBUse: 57%Speed: ~22 tok/sScore: 59/100
DECENT
59/100
Gemma 3 27BGoogleGemmaQ2_K

Google's flagship Gemma 3 model

Memory: 9.1 GBUse: 57%Speed: ~22 tok/sScore: 59/100
DECENT
59/100
Gemma 4 26B-A4BGoogleGemma4B activeQ2_K

Gemma 4 MoE base model (official)

Memory: 9.1 GBUse: 57%Speed: ~22 tok/sScore: 59/100
DECENT
59/100
Qwen 3.8 27BAlibabaApache 2.0Q2_K

Flagship dense Qwen 3.8 — native multimodal all-rounder with video understanding

Memory: 9.1 GBUse: 57%Speed: ~22 tok/sScore: 59/100
DECENT
59/100
Qwen 3.6 27BAlibabaApache 2.0Q2_K

Flagship dense Qwen 3.6 — native multimodal all-rounder

Memory: 9.4 GBUse: 59%Speed: ~21 tok/sScore: 57/100
DECENT
57/100
Qwen 3 30B-A3BAlibabaApache 2.03.3B activeQ2_K

MoE with only 3.3B active — extremely efficient

Memory: 10.1 GBUse: 63%Speed: ~20 tok/sScore: 55/100
DECENT
55/100
Nemotron 3 Nano 30BNVIDIANVIDIA Open3B activeQ2_K

MoE with 1M context and 3B active

Memory: 10.1 GBUse: 63%Speed: ~20 tok/sScore: 55/100
DECENT
55/100
Granite 4.1 30BIBMApache 2.0Q2_K

High-capacity enterprise model for complex reasoning and tool use

Memory: 10.1 GBUse: 63%Speed: ~20 tok/sScore: 55/100
DECENT
55/100
North Mini CodeCohereApache 2.03B activeQ2_K

Open agentic coding MoE with 3B active — built for software engineering and terminal tasks

Memory: 10.1 GBUse: 63%Speed: ~20 tok/sScore: 55/100
DECENT
55/100
Qwen 3 Coder 30B-A3BAlibabaApache 2.03B activeQ2_K

Efficient agentic coding MoE — 3B active, 256K context

Memory: 10.1 GBUse: 63%Speed: ~20 tok/sScore: 55/100
DECENT
55/100
Muse Glimmer 30BMetaApache 2.0Q2_K

Open agentic 30B distilled from Muse Spark — tool use, vision and local recovery on a single GPU

Memory: 10.1 GBUse: 63%Speed: ~20 tok/sScore: 55/100
DECENT
55/100
Qwen3-VL 30B-A3BAlibabaApache 2.03B activeQ2_K

Efficient vision MoE — 3B active, strong temporal & document understanding

Memory: 10.4 GBUse: 65%Speed: ~19 tok/sScore: 53/100
TIGHT FIT
53/100
Qwen 2.5 32BAlibabaApache 2.0Q2_K

High-quality reasoning and multilingual

Memory: 10.7 GBUse: 67%Speed: ~19 tok/sScore: 52/100
TIGHT FIT
52/100
Qwen 2.5 Coder 32BAlibabaApache 2.0Q2_K

Best open-source coding model at release

Memory: 10.7 GBUse: 67%Speed: ~19 tok/sScore: 52/100
TIGHT FIT
52/100
Qwen 3 32BAlibabaApache 2.0Q2_K

Qwen 3 flagship dense model

Memory: 10.7 GBUse: 67%Speed: ~19 tok/sScore: 52/100
TIGHT FIT
52/100

R1 reasoning distilled into Qwen 32B — sweet spot

Memory: 10.7 GBUse: 67%Speed: ~19 tok/sScore: 52/100
TIGHT FIT
52/100
EXAONE 4.0 32BLG AIEXAONE AIQ2_K

Hybrid reasoning, multilingual

Memory: 10.7 GBUse: 67%Speed: ~19 tok/sScore: 52/100
TIGHT FIT
52/100
OLMo 2 32BAllen AIApache 2.0Q2_K

Fully open research model by Allen AI

Memory: 10.7 GBUse: 67%Speed: ~19 tok/sScore: 52/100
TIGHT FIT
52/100
Gemma 4 31BGoogleGemmaQ2_K

Gemma 4 flagship base model (official)

Memory: 11.1 GBUse: 69%Speed: ~18 tok/sScore: 50/100
TIGHT FIT
50/100
Command R 35BCohereCC BY-NC 4.0Q2_K

Optimized for retrieval-augmented generation

Memory: 11.7 GBUse: 73%Speed: ~17 tok/sScore: 47/100
TIGHT FIT
47/100
Agents-A1 35B-A3BInternScienceApache 2.03B activeQ2_K

Efficient multimodal agentic MoE for long-horizon search, engineering and scientific research

Memory: 11.7 GBUse: 73%Speed: ~17 tok/sScore: 47/100
TIGHT FIT
47/100
Ornith 1.0 35B-A3BDeepReinforceMIT3B activeQ2_K

Agentic coding MoE with a 3B active working set and self-improving training

Memory: 11.7 GBUse: 73%Speed: ~17 tok/sScore: 47/100
TIGHT FIT
47/100
Qwen 3.6 35B-A3BAlibabaApache 2.03B activeQ2_K

Big-model quality at 3B-active speed — the mid-hardware sweet spot

Memory: 12 GBUse: 75%Speed: ~17 tok/sScore: 47/100
TIGHT FIT
47/100
Gemma 4 26B-A4B ITGoogleGemma4B activeQ4_K_M

Gemma 4 MoE instruct model (official)

Memory: 14.3 GBUse: 89%Speed: ~14 tok/sScore: 28/100
BARELY RUNS
28/100
Qwen 3.5 27BAlibabaApache 2.0Q4_K_M

Flagship native multimodal Qwen 3.5

Memory: 14.7 GBUse: 92%Speed: ~14 tok/sScore: 27/100
BARELY RUNS
27/100
Mixtral 8x7BMistral AIApache 2.012.9B activeQ2_K

MoE with 12.9B active params

Memory: 15.5 GBUse: 97%Speed: ~13 tok/sScore: 26/100
BARELY RUNS
26/100
Qwen 3.5 35B-A3BAlibabaApache 2.03B activeQ4_K_M

Efficient multimodal MoE with 3B active

Memory: 18.4 GBUse: 115%Speed: ~6 tok/sScore: 12/100
BARELY RUNS
12/100
Llama 3.3 70BMetaLlama 3.3 CommunityQ2_K

Best open model at 70B class

Memory: 22.9 GBUse: 143%Speed: ~3 tok/sScore: 10/100
BARELY RUNS
10/100
Qwen 2.5 72BAlibabaQwenQ2_K

Alibaba's flagship open model

Memory: 23.6 GBUse: 148%Speed: ~3 tok/sScore: 10/100
BARELY RUNS
10/100
Nemotron 3.5 Bolt 74B-A9BNVIDIAOpenMDW 1.19BQ2_K

Run Nemotron 3.5 Bolt 74B-A9B locally. Check hardware compatibility, VRAM requirements, and performance.

Memory: 24.2 GBUse: 151%Speed: ~3 tok/sScore: 10/100
BARELY RUNS
10/100
Qwen 3 Next 80B-A3BAlibabaApache 2.03B activeQ2_K

High-sparsity MoE — extreme low activation ratio for fast inference at 80B scale

Memory: 26.1 GBUse: 163%Speed: ~2 tok/sScore: 9/100
BARELY RUNS
9/100
Qwen 3 Coder Next 80B-A3BAlibabaApache 2.03B activeQ2_K

Ultra-efficient agentic coding MoE optimized for tool-calling coding agents

Memory: 26.1 GBUse: 163%Speed: ~2 tok/sScore: 9/100
BARELY RUNS
9/100
Llama 4 Scout 17BMetaLlama 4 Community17B activeQ2_K

MoE with 16 experts, 17B active params

Memory: 35.4 GBUse: 221%Speed: ~1 tok/sScore: 8/100
BARELY RUNS
8/100
Gemma 4 31B ITGoogleGemmaQ6_K

Gemma 4 flagship instruct model (official)

Memory: 25.9 GBUse: 162%Speed: ~2 tok/sScore: 8/100
BARELY RUNS
8/100
GPT-OSS 120BOpenAIApache 2.05.1B activeQ2_K

OpenAI's flagship open-weight MoE — 52.6% SWE-bench

Memory: 38 GBUse: 238%Speed: ~1 tok/sScore: 8/100
BARELY RUNS
8/100
GLM-4.5 AirZ.aiMIT12B activeQ2_K

Consumer-friendly GLM MoE — 12B active, strong agentic & tool use

Memory: 34.4 GBUse: 215%Speed: ~1 tok/sScore: 8/100
BARELY RUNS
8/100
Devstral 2 123BMistral AIMRLQ2_K

Dense 123B coding model — 72.2% SWE-bench Verified

Memory: 39.9 GBUse: 249%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
Qwen 3.5 122B-A10BAlibabaApache 2.010B activeQ4_K_M

Large multimodal MoE

Memory: 63 GBUse: 394%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
Mixtral 8x22BMistral AIApache 2.039B activeQ2_K

Large MoE with 39B active params

Memory: 45.6 GBUse: 285%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
Llama 4 Maverick 17B-128EMetaLlama 4 Community17B activeQ2_K

Multimodal MoE with 128 experts — 17B active, 1M context

Memory: 128.6 GBUse: 804%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
Qwen 3 235B-A22BAlibabaApache 2.022B activeQ2_K

Massive MoE with 22B active — frontier quality

Memory: 75.7 GBUse: 473%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
Qwen 3.5 397B-A17BAlibabaApache 2.017B activeQ2_K

Largest multimodal Qwen 3.5 MoE

Memory: 127.6 GBUse: 798%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
Llama 3.1 405BMetaLlama 3.1 CommunityQ2_K

Largest open-weight dense model by Meta

Memory: 130.2 GBUse: 814%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
Qwen 3 Coder 480BAlibabaApache 2.035B activeQ2_K

Largest open coding MoE — 35B active

Memory: 154.2 GBUse: 964%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
DeepSeek R1DeepSeekMIT37B activeQ2_K

Massive MoE reasoning model — 37B active

Memory: 215.3 GBUse: 1346%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
DeepSeek V3.1DeepSeekMIT37B activeQ2_K

Improved V3 with hybrid thinking and tool use

Memory: 215.3 GBUse: 1346%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
DeepSeek V3.2DeepSeekMIT37B activeQ2_K

State-of-the-art MoE — 37B active params

Memory: 219.8 GBUse: 1374%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
Kimi K2Moonshot AIKimi32B activeQ2_K

1T-param MoE with 384 experts — 32B active, strong agentic coding

Memory: 320.6 GBUse: 2004%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
Qwen 3 VL 235B-A22BAlibabaApache 2.022B activeQ2_K

Flagship vision-language MoE — frontier multimodal reasoning and agentic GUI control

Memory: 75.7 GBUse: 473%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
Hy3TencentApache 2.021B activeQ2_K

Production-focused agentic MoE with strong coding, tool use and long-context reasoning

Memory: 94.9 GBUse: 593%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
GLM-4.6Z.aiMIT32B activeQ2_K

Large GLM MoE with strong coding and 200K context

Memory: 114.8 GBUse: 718%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
GLM-5Zhipu AIMIT40B activeQ2_K

MoE with 256 experts, 40B active — frontier-class agentic coding

Memory: 238.7 GBUse: 1492%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
GLM-5.1Zhipu AIMIT40B activeQ2_K

Improved agentic coding — SOTA SWE-bench Pro, long-horizon tasks

Memory: 241.9 GBUse: 1512%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
GLM-5.2Z.aiMIT40B activeQ2_K

Frontier open-weight coder — top SWE-bench, 1M context

Memory: 241.6 GBUse: 1510%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
GLM-5.3Z.aiMIT40B activeQ2_K

Same 753B / 40B-active base as GLM-5.2 — post-training lifts coding and long-horizon agents, 1M context

Memory: 241.6 GBUse: 1510%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
LongCat 2.0MeituanMIT48B activeQ2_K

Frontier-scale agentic and coding MoE with sparse attention and native 1M context

Memory: 512.7 GBUse: 3204%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
DeepSeek V4 FlashDeepSeekMIT13B activeQ2_K

Efficient long-context V4 — 13B active, 1M context

Memory: 51.1 GBUse: 319%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
DeepSeek V4 ProDeepSeekMIT49B activeQ2_K

Flagship V4 MoE — 49B active, 1M context

Memory: 512.7 GBUse: 3204%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
Kimi K2.6Moonshot AIKimi32B activeQ2_K

Natively multimodal 1T MoE — 32B active, frontier agentic

Memory: 339.2 GBUse: 2120%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
Kimi K3Moonshot AIKimi104B activeQ2_K

Frontier 2.8T multimodal MoE — 104B active, native video understanding, 1M context

Memory: 890.5 GBUse: 5566%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
Qwen 3.8 2.4T-A95BAlibabaQwen95B activeQ2_K

Frontier Qwen 3.8 MoE — 95B active, 1M context

Memory: 768.8 GBUse: 4805%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
MiniMax M3MiniMaxMiniMax Community23B activeQ2_K

Native multimodal MoE — understands text, image and long video with 1M context

Memory: 137.5 GBUse: 859%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
Mistral Small 4 119BMistral AIApache 2.06.5B activeQ2_K

Sparse Mistral Small 4 — 6.5B active, strong local all-rounder

Memory: 38.6 GBUse: 241%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
Qwen 3.8 Flash 128BAlibabaApache 2.06B activeQ2_K

125B sparse MoE (6B active) — runs on 12GB VRAM gaming PCs with Strata engine up to 96 tok/s

Memory: 40.5 GBUse: 253%Speed: ~1 tok/sScore: 0/100
TOO HEAVY
0/100
From the community

Gemma 4 in the wild

Real developers sharing what they’re running — on the exact models and hardware this checker helps you evaluate.

Updated October 2026
@alexriva_aiOct 4, 2026
Gemma 4 26B-A4B (MoE)MacBook Pro M4 Max (128GB)
85 tokens/s continuous decode across 128k context window — 0 slowdown

MoE architecture activating only 3.8B active parameters hits its absolute peak efficiency on Apple's high-bandwidth unified memory.

1.4k likes • 84 replies • 912 bookmarks
@local_llm_benchOct 2, 2026
Gemma 4 31B DenseRTX 3090 / 4090 (24GB VRAM)
34 tokens/s throughput with Ollama + iMatrix Q4_K_M entirely in 24GB VRAM

For users prioritizing dense reasoning quality, 31B fits comfortably in consumer 24GB cards without offloading to slower system RAM.

980 likes • 51 replies • 674 bookmarks
@markus_chen_devSep 28, 2026
Gemma 4 E4BMac mini M4 (24GB)
84 tokens/s background daemon consuming just 4.2GB RAM with near-zero idle draw

Proves that budget desktop hardware can host an ultra-fast, 24/7 private coding assistant and semantic daemon.

840 likes • 39 replies • 530 bookmarks
@old_pgmrs_willSep 24, 2026
Gemma 4 26B-A4BAMD Ryzen AI Max+ 395 (128GB APU)
MTP speculative tuning boosts 26B-A4B from 41 t/s to ~60 tokens/s

AMD's large unified memory APUs combined with multi-token prediction offer a genuine high-speed alternative to discrete GPUs.

620 likes • 34 replies • 410 bookmarks
@analogalokSep 19, 2026
Gemma 4 26B-A4B QATRTX 4060 8GB Laptop (Offline)
Zero-WiFi agent built full browser extension: React UI, local llama.cpp backend, Vision on demand

Demonstrates that quantized Gemma 4 MoE allows true autonomous software building completely offline on affordable laptops.

750 likes • 42 replies • 880 bookmarks
@UnslothAIUpdated Sep 2026
Gemma 4 12B & 31BLow VRAM / Consumer GPUs
2× faster with MTP GGUFs — 12B hits 162 tokens/s, 31B hits 101 t/s

MTP speculative decoding makes larger Gemma 4 variants practical on modest hardware with almost no quality loss.

2.8k likes • 94 replies • 2150 bookmarks

GGUF Quantization Cheat Sheet

Gemma 4 GGUF VRAM & Unified Memory Quick Reference (8K Context)

Active memory figures include GGUF weights plus an 8,192-token KV cache and runtime compute buffers. On Windows/Linux GPUs, match against Dedicated VRAM; on Apple Silicon Macs, stay within ~75% of unified RAM.

Model CheckpointArchitectureQ4_K_M DiskSafe Q4_K_M VRAMQ6_K VRAMFP16 / Q8_0Recommended Hardware
Gemma 4 E2B ITDense (~5B)~3.1–3.4 GB4.5–6 GB~5.5 GB~10.8 GB8GB+ Android/iPhone, 8GB Mac, 6GB RTX Laptop
Gemma 4 E4B ITDense (~8B)~4.6–5.3 GB6.5–8 GB~8.2 GB~16.9 GB16GB Apple Silicon Mac, 8GB+ RTX GPU, 12GB+ Phone
Gemma 4 12B ITDense (12B · 256K)~7.1 GB9.5–12 GB~11.8 GB~25.1 GBRTX 3060 12GB / 4070, 16GB–24GB M2/M3/M4 Mac
Gemma 4 26B-A4B ITSparse MoE (4B active)~15.8 GB18.5–22 GB~21.5 GB~28.1 GB (Q8_0)24GB RTX 3090/4090, 32GB–36GB MacBook Pro / Studio
DiffusionGemma 26B-A4BDiffusion MoE (4B active)~13.8 GB16.5–20 GB~22.0 GB~53.8 GB24GB Desktop GPU, 32GB+ Unified Memory Mac
Gemma 4 31BDense Flagship (~31B)~19.0 GB22.5–24 GB~29.5 GB~68.1 GB24GB–32GB RTX 4090/5090, 48GB–64GB Mac Studio

Engineering Methodology

How Local Gemma 4 & LLM Memory Sizing Works

Choosing a local model is rarely about total disk space alone. A quantized GGUF file that looks small in a file manager still needs contiguous high-speed memory for model weights, runtime compute scratchpads, and a dynamically growing Key-Value (KV) cache as your conversation lengthens. Our hardware checker estimates real usability using the total active budget: Total Memory = Quantized Weights + KV Cache + Runtime Buffers.

1. Why Memory Bandwidth Governs Tokens per Second

Autoregressive token generation is memory-bandwidth bound on almost every consumer laptop and desktop GPU. Each generated token requires reading active model parameters from VRAM or unified memory. As a rule of thumb, sustainable speed scales roughly with (Bandwidth GB/s / Active Weights GB) × efficiency. That is why an RTX 4090 (1,008 GB/s GDDR6X) or Apple M4 Max (546 GB/s unified memory) generates replies dramatically faster than a midrange card with the same VRAM capacity but a narrower 128-bit memory bus.

2. Discrete VRAM vs. Apple Unified Memory vs. Mobile LMK

On Windows and Linux desktops with NVIDIA, AMD, or Intel GPUs, exceeding dedicated VRAM by even 1 GB triggers silent layer spilling across the PCIe bus into system DDR5 (~50 GB/s), cutting throughput by 5–10×. Apple Silicon Macs avoid PCIe spillover via shared unified memory, though macOS typically caps safe GPU wired allocation near ~75% of total RAM. On Android and iOS, OS memory watchdogs (LMK and Jetsam) reserve 45–50% of physical RAM, making compact E2B and E4B builds essential on 8 GB phones.

3. Quantization Precision: Q4_K_M, Q6_K, Q8_0 & FP16

Most local users start with Q4_K_M (~4.5 bits/weight average), which cuts memory footprints by roughly 70% compared to FP16 while preserving strong reasoning and coding quality. Stepping up to Q6_K or Q8_0 adds ~40–95% memory overhead for marginal gains in strict JSON formatting and tool-calling reliability. You can inspect exact per-tier tradeoffs in our quantization reference.

4. Dense vs. Sparse Mixture-of-Experts (Gemma 4 26B-A4B)

Sparse MoE checkpoints such as Gemma 4 26B-A4B IT still require enough total memory (~15.8 GB at Q4_K_M) to hold all 26 billion weights, yet only activate ~4B parameters per token. This lets 24 GB GPUs and 32–36 GB Macs capture near-30B reasoning quality at 4B-class generation speeds.

Frequently Asked Questions

Gemma 4 Local Hardware, Quantization & Runtime FAQs

How much VRAM or RAM do I need to run Gemma 4 locally?

At the standard Q4_K_M 4-bit quantization with an 8K context window, Gemma 4 E2B requires ~4.5–6 GB of active memory, Gemma 4 E4B requires ~6.5–8 GB, Gemma 4 12B IT requires ~9.5–12 GB, Gemma 4 26B-A4B IT (MoE) requires ~18.5–22 GB, and Gemma 4 31B requires ~22.5–24 GB of dedicated GPU VRAM or Apple unified memory.

Which Gemma 4 GGUF quantization should I download first?

Start with Q4_K_M (~4.5 bits per weight). It shrinks memory requirements by roughly 70% compared with FP16 while retaining near-lossless chat, coding, and multimodal quality. Only upgrade to Q6_K or Q8_0 when your GPU or Mac has 35–90% spare memory headroom and you need strict structured JSON or tool-calling accuracy.

Can a 16GB Apple Silicon MacBook Air or Mac mini run Gemma 4?

Yes. macOS typically permits ~11.5–12 GB of wired Metal GPU allocation on a 16GB Mac, making Gemma 4 E4B IT (Q4_K_M or Q6_K) and Gemma 4 12B IT (Q4_K_M at 8K–16K context) smooth daily drivers at 25–45 tokens/second. Avoid Gemma 4 26B-A4B or 31B on 16GB RAM.

Why does Gemma 4 run slowly (2–4 tok/s) on my Windows PC even though it loads?

Windows WDDM automatically spills overflow layers into Shared GPU Memory (system DDR4/DDR5 RAM over PCIe) whenever a GGUF checkpoint plus KV cache exceeds dedicated GPU VRAM. Check Task Manager -> Performance -> GPU -> Dedicated GPU Memory, and switch to a smaller quant or shorten context to stay 100% inside dedicated VRAM.

What is the difference between Gemma 4 26B-A4B IT and Gemma 4 31B?

Gemma 4 26B-A4B IT uses a Sparse Mixture-of-Experts (MoE) architecture: all 26B parameters stay in memory (~15.8 GB Q4_K_M), but only ~4B parameters activate per token, delivering fast 4B-class generation speeds with near-30B reasoning. Gemma 4 31B is a dense model that activates every parameter on every token, needing 24GB+ VRAM and higher memory bandwidth.

Can I run Gemma 4 offline on an Android phone or APK without Google Play?

Yes. On 8GB+ Android phones, install Google AI Edge Gallery from Google Play or download the official signed APK from Google AI Edge GitHub Releases (or open-source PocketPal AI), then load Gemma 4 E2B IT for 100% offline inference. Never download third-party "Gemma 4 mod APKs."

Which local runtime is best for Gemma 4: LM Studio, Ollama, or llama.cpp?

Use LM Studio for the fastest desktop GUI setup on Mac and Windows, Ollama when you need a local OpenAI-compatible HTTP API for coding agents, llama.cpp when you want granular control over GPU layer offload (-ngl) and quantized KV cache (-ctk q8_0), and Google AI Edge Gallery on Android and iOS.

Does Gemma 4 support multimodal image input locally?

Yes. Gemma 4 instruct builds incorporate native multimodal support, allowing local image understanding, OCR, and visual Q&A in updated versions of LM Studio, llama.cpp, Ollama, and Google AI Edge Gallery—just budget an extra 1–2 GB of memory headroom for vision tokens.

Is Gemma 4 free for commercial and private offline use?

Yes. Google released the Gemma 4 open-weights model family under the permissive Apache 2.0 license, allowing free local execution, fine-tuning, and commercial deployment without API token fees or cloud telemetry.