Phones, tablets, old laptops, and the fastest local proof.
Can I Run Gemma 4 locally?
Find out which Gemma 4 models your machine can actually run — or jump to Download Gemma 4 · Android APK Guide · Compare Models & Hardware Fit.
Better phones, Apple silicon laptops, and mainstream desktops.
16GB+ unified-memory Macs and desktops with room for context.
The instruct 12B when you want chat, vision, and a 256K window on 16GB+ machines.
Gemma-family MoE when you want the newer diffusion decode path and multimodal prompts.
Strong desktop GPUs and disciplined 24GB+ Apple silicon setups.
High-end workstations where local quality matters more than convenience.
Check other local AI models
Evaluate the same hardware profile across open-weight LLM, reasoning, coding, and multimodal models beyond Gemma 4.
Microsoft's efficient small model with long context
Lightweight reasoning model
Compact Qwen 3 for general tasks
Multimodal Gemma with 128K context
Small multimodal Qwen 3.5
Gemma 4 efficient base model (official)
High-quality 7B with sliding window attention
Strong multilingual and coding capabilities
Dedicated coding model
R1 reasoning distilled into Qwen 7B
Compact dedicated vision-language model — OCR & image chat on edge
Lightweight Llama for mobile and edge
Lightweight multilingual reasoning
Compact enterprise model for edge and constrained environments
Current-gen tiny Ministral — edge chat with 256K context
Ultra-lightweight coding model
Tiny reasoning model distilled from R1
Compact multilingual Qwen 3
Small multimodal Qwen 3.5
Google's compact open model
Ultra-tiny model for embedded and edge
Meta's smallest Llama for edge devices
Google's tiny Gemma for on-device
Ultralight model for constrained devices
Gemma 4 balanced base model (official)
Meta's versatile 8B — great quality/speed ratio
Qwen 3 with thinking mode support
Mistral's efficient 8B model
Balanced general-purpose enterprise model
Ultra-light Qwen 3 model for constrained devices
Gemma 4 efficient instruct model (official)
The community-favourite local VLM — superb OCR, receipts & captioning
Google's best mid-size open model
Multilingual model supporting 26 languages with 128K context
Hybrid Mamba2 architecture for reasoning
Self-improving agentic coding model optimized for terminal and software engineering tasks
Multimodal vision and text model
Gemma 4 mid-size base model (official)
Multimodal Gemma with 128K context
Multilingual 12B with 128K context
Gemma 4 mid-size instruct — multimodal any-to-any
Gemma 4 balanced instruct model (official)
Excellent quality for its size class
Microsoft's reasoning-focused model
Strong all-rounder with thinking mode
R1 reasoning distilled into Qwen 14B
Current-gen Ministral mid-size — local assistant with 256K context
Multimodal Qwen 3.5 mid-size
OpenAI's open-weight MoE with configurable reasoning
Hybrid MoE with convolution+attention layers — 2.3B active
Coding-focused model with 256K context — 68% SWE-bench
Multimodal Mistral with vision support
Discrete diffusion MoE — 1100+ tok/s on H100, multimodal (text/image/video)
Google's largest Gemma 2 model
Google's flagship Gemma 3 model
Gemma 4 MoE base model (official)
Flagship dense Qwen 3.8 — native multimodal all-rounder with video understanding
Flagship dense Qwen 3.6 — native multimodal all-rounder
MoE with only 3.3B active — extremely efficient
MoE with 1M context and 3B active
High-capacity enterprise model for complex reasoning and tool use
Open agentic coding MoE with 3B active — built for software engineering and terminal tasks
Efficient agentic coding MoE — 3B active, 256K context
Open agentic 30B distilled from Muse Spark — tool use, vision and local recovery on a single GPU
Efficient vision MoE — 3B active, strong temporal & document understanding
High-quality reasoning and multilingual
Best open-source coding model at release
Qwen 3 flagship dense model
R1 reasoning distilled into Qwen 32B — sweet spot
Hybrid reasoning, multilingual
Fully open research model by Allen AI
Gemma 4 flagship base model (official)
Optimized for retrieval-augmented generation
Efficient multimodal agentic MoE for long-horizon search, engineering and scientific research
Agentic coding MoE with a 3B active working set and self-improving training
Big-model quality at 3B-active speed — the mid-hardware sweet spot
Gemma 4 MoE instruct model (official)
Flagship native multimodal Qwen 3.5
MoE with 12.9B active params
Efficient multimodal MoE with 3B active
Best open model at 70B class
Alibaba's flagship open model
Run Nemotron 3.5 Bolt 74B-A9B locally. Check hardware compatibility, VRAM requirements, and performance.
High-sparsity MoE — extreme low activation ratio for fast inference at 80B scale
Ultra-efficient agentic coding MoE optimized for tool-calling coding agents
MoE with 16 experts, 17B active params
Gemma 4 flagship instruct model (official)
OpenAI's flagship open-weight MoE — 52.6% SWE-bench
Consumer-friendly GLM MoE — 12B active, strong agentic & tool use
Dense 123B coding model — 72.2% SWE-bench Verified
Large multimodal MoE
Large MoE with 39B active params
Multimodal MoE with 128 experts — 17B active, 1M context
Massive MoE with 22B active — frontier quality
Largest multimodal Qwen 3.5 MoE
Largest open-weight dense model by Meta
Largest open coding MoE — 35B active
Massive MoE reasoning model — 37B active
Improved V3 with hybrid thinking and tool use
State-of-the-art MoE — 37B active params
1T-param MoE with 384 experts — 32B active, strong agentic coding
Flagship vision-language MoE — frontier multimodal reasoning and agentic GUI control
Production-focused agentic MoE with strong coding, tool use and long-context reasoning
Large GLM MoE with strong coding and 200K context
MoE with 256 experts, 40B active — frontier-class agentic coding
Improved agentic coding — SOTA SWE-bench Pro, long-horizon tasks
Frontier open-weight coder — top SWE-bench, 1M context
Same 753B / 40B-active base as GLM-5.2 — post-training lifts coding and long-horizon agents, 1M context
Frontier-scale agentic and coding MoE with sparse attention and native 1M context
Efficient long-context V4 — 13B active, 1M context
Flagship V4 MoE — 49B active, 1M context
Natively multimodal 1T MoE — 32B active, frontier agentic
Frontier 2.8T multimodal MoE — 104B active, native video understanding, 1M context
Frontier Qwen 3.8 MoE — 95B active, 1M context
Native multimodal MoE — understands text, image and long video with 1M context
Sparse Mistral Small 4 — 6.5B active, strong local all-rounder
125B sparse MoE (6B active) — runs on 12GB VRAM gaming PCs with Strata engine up to 96 tok/s
Gemma 4 in the wild
Real developers sharing what they’re running — on the exact models and hardware this checker helps you evaluate.
MoE architecture activating only 3.8B active parameters hits its absolute peak efficiency on Apple's high-bandwidth unified memory.
For users prioritizing dense reasoning quality, 31B fits comfortably in consumer 24GB cards without offloading to slower system RAM.
Proves that budget desktop hardware can host an ultra-fast, 24/7 private coding assistant and semantic daemon.
AMD's large unified memory APUs combined with multi-token prediction offer a genuine high-speed alternative to discrete GPUs.
Demonstrates that quantized Gemma 4 MoE allows true autonomous software building completely offline on affordable laptops.
MTP speculative decoding makes larger Gemma 4 variants practical on modest hardware with almost no quality loss.
GGUF Quantization Cheat Sheet
Gemma 4 GGUF VRAM & Unified Memory Quick Reference (8K Context)
Active memory figures include GGUF weights plus an 8,192-token KV cache and runtime compute buffers. On Windows/Linux GPUs, match against Dedicated VRAM; on Apple Silicon Macs, stay within ~75% of unified RAM.
| Model Checkpoint | Architecture | Q4_K_M Disk | Safe Q4_K_M VRAM | Q6_K VRAM | FP16 / Q8_0 | Recommended Hardware |
|---|---|---|---|---|---|---|
| Gemma 4 E2B IT | Dense (~5B) | ~3.1–3.4 GB | 4.5–6 GB | ~5.5 GB | ~10.8 GB | 8GB+ Android/iPhone, 8GB Mac, 6GB RTX Laptop |
| Gemma 4 E4B IT | Dense (~8B) | ~4.6–5.3 GB | 6.5–8 GB | ~8.2 GB | ~16.9 GB | 16GB Apple Silicon Mac, 8GB+ RTX GPU, 12GB+ Phone |
| Gemma 4 12B IT | Dense (12B · 256K) | ~7.1 GB | 9.5–12 GB | ~11.8 GB | ~25.1 GB | RTX 3060 12GB / 4070, 16GB–24GB M2/M3/M4 Mac |
| Gemma 4 26B-A4B IT | Sparse MoE (4B active) | ~15.8 GB | 18.5–22 GB | ~21.5 GB | ~28.1 GB (Q8_0) | 24GB RTX 3090/4090, 32GB–36GB MacBook Pro / Studio |
| DiffusionGemma 26B-A4B | Diffusion MoE (4B active) | ~13.8 GB | 16.5–20 GB | ~22.0 GB | ~53.8 GB | 24GB Desktop GPU, 32GB+ Unified Memory Mac |
| Gemma 4 31B | Dense Flagship (~31B) | ~19.0 GB | 22.5–24 GB | ~29.5 GB | ~68.1 GB | 24GB–32GB RTX 4090/5090, 48GB–64GB Mac Studio |
Engineering Methodology
How Local Gemma 4 & LLM Memory Sizing Works
Choosing a local model is rarely about total disk space alone. A quantized GGUF file that looks small in a file manager still needs contiguous high-speed memory for model weights, runtime compute scratchpads, and a dynamically growing Key-Value (KV) cache as your conversation lengthens. Our hardware checker estimates real usability using the total active budget: Total Memory = Quantized Weights + KV Cache + Runtime Buffers.
1. Why Memory Bandwidth Governs Tokens per Second
Autoregressive token generation is memory-bandwidth bound on almost every consumer laptop and desktop GPU. Each generated token requires reading active model parameters from VRAM or unified memory. As a rule of thumb, sustainable speed scales roughly with (Bandwidth GB/s / Active Weights GB) × efficiency. That is why an RTX 4090 (1,008 GB/s GDDR6X) or Apple M4 Max (546 GB/s unified memory) generates replies dramatically faster than a midrange card with the same VRAM capacity but a narrower 128-bit memory bus.
2. Discrete VRAM vs. Apple Unified Memory vs. Mobile LMK
On Windows and Linux desktops with NVIDIA, AMD, or Intel GPUs, exceeding dedicated VRAM by even 1 GB triggers silent layer spilling across the PCIe bus into system DDR5 (~50 GB/s), cutting throughput by 5–10×. Apple Silicon Macs avoid PCIe spillover via shared unified memory, though macOS typically caps safe GPU wired allocation near ~75% of total RAM. On Android and iOS, OS memory watchdogs (LMK and Jetsam) reserve 45–50% of physical RAM, making compact E2B and E4B builds essential on 8 GB phones.
3. Quantization Precision: Q4_K_M, Q6_K, Q8_0 & FP16
Most local users start with Q4_K_M (~4.5 bits/weight average), which cuts memory footprints by roughly 70% compared to FP16 while preserving strong reasoning and coding quality. Stepping up to Q6_K or Q8_0 adds ~40–95% memory overhead for marginal gains in strict JSON formatting and tool-calling reliability. You can inspect exact per-tier tradeoffs in our quantization reference.
4. Dense vs. Sparse Mixture-of-Experts (Gemma 4 26B-A4B)
Sparse MoE checkpoints such as Gemma 4 26B-A4B IT still require enough total memory (~15.8 GB at Q4_K_M) to hold all 26 billion weights, yet only activate ~4B parameters per token. This lets 24 GB GPUs and 32–36 GB Macs capture near-30B reasoning quality at 4B-class generation speeds.
Frequently Asked Questions
Gemma 4 Local Hardware, Quantization & Runtime FAQs
How much VRAM or RAM do I need to run Gemma 4 locally?
At the standard Q4_K_M 4-bit quantization with an 8K context window, Gemma 4 E2B requires ~4.5–6 GB of active memory, Gemma 4 E4B requires ~6.5–8 GB, Gemma 4 12B IT requires ~9.5–12 GB, Gemma 4 26B-A4B IT (MoE) requires ~18.5–22 GB, and Gemma 4 31B requires ~22.5–24 GB of dedicated GPU VRAM or Apple unified memory.
Which Gemma 4 GGUF quantization should I download first?
Start with Q4_K_M (~4.5 bits per weight). It shrinks memory requirements by roughly 70% compared with FP16 while retaining near-lossless chat, coding, and multimodal quality. Only upgrade to Q6_K or Q8_0 when your GPU or Mac has 35–90% spare memory headroom and you need strict structured JSON or tool-calling accuracy.
Can a 16GB Apple Silicon MacBook Air or Mac mini run Gemma 4?
Yes. macOS typically permits ~11.5–12 GB of wired Metal GPU allocation on a 16GB Mac, making Gemma 4 E4B IT (Q4_K_M or Q6_K) and Gemma 4 12B IT (Q4_K_M at 8K–16K context) smooth daily drivers at 25–45 tokens/second. Avoid Gemma 4 26B-A4B or 31B on 16GB RAM.
Why does Gemma 4 run slowly (2–4 tok/s) on my Windows PC even though it loads?
Windows WDDM automatically spills overflow layers into Shared GPU Memory (system DDR4/DDR5 RAM over PCIe) whenever a GGUF checkpoint plus KV cache exceeds dedicated GPU VRAM. Check Task Manager -> Performance -> GPU -> Dedicated GPU Memory, and switch to a smaller quant or shorten context to stay 100% inside dedicated VRAM.
What is the difference between Gemma 4 26B-A4B IT and Gemma 4 31B?
Gemma 4 26B-A4B IT uses a Sparse Mixture-of-Experts (MoE) architecture: all 26B parameters stay in memory (~15.8 GB Q4_K_M), but only ~4B parameters activate per token, delivering fast 4B-class generation speeds with near-30B reasoning. Gemma 4 31B is a dense model that activates every parameter on every token, needing 24GB+ VRAM and higher memory bandwidth.
Can I run Gemma 4 offline on an Android phone or APK without Google Play?
Yes. On 8GB+ Android phones, install Google AI Edge Gallery from Google Play or download the official signed APK from Google AI Edge GitHub Releases (or open-source PocketPal AI), then load Gemma 4 E2B IT for 100% offline inference. Never download third-party "Gemma 4 mod APKs."
Which local runtime is best for Gemma 4: LM Studio, Ollama, or llama.cpp?
Use LM Studio for the fastest desktop GUI setup on Mac and Windows, Ollama when you need a local OpenAI-compatible HTTP API for coding agents, llama.cpp when you want granular control over GPU layer offload (-ngl) and quantized KV cache (-ctk q8_0), and Google AI Edge Gallery on Android and iOS.
Does Gemma 4 support multimodal image input locally?
Yes. Gemma 4 instruct builds incorporate native multimodal support, allowing local image understanding, OCR, and visual Q&A in updated versions of LM Studio, llama.cpp, Ollama, and Google AI Edge Gallery—just budget an extra 1–2 GB of memory headroom for vision tokens.
Is Gemma 4 free for commercial and private offline use?
Yes. Google released the Gemma 4 open-weights model family under the permissive Apache 2.0 license, allowing free local execution, fine-tuning, and commercial deployment without API token fees or cloud telemetry.