Android
Gemma 4 on Android
Complete guide to running Gemma 4 locally on Android phones and tablets: chipset NPU/GPU support, LMK memory limits, AI Edge Gallery setup, and realistic quants.
For most Android users, the cleanest way to run Gemma 4 locally without root access or Termux build toolchains is Google AI Edge Gallery, backed by LiteRT and hardware GPU/NPU delegates.
Mobile inference fails most often not because the phone chip lacks FLOPS, but because Android’s Low Memory Killer (LMK) terminates any foreground process that pushes total free system RAM below safe OS reserves.
Recommended model sizes
- Gemma 4 E2B: safest default for 8 GB and 12 GB Android phones with fast cold loads and reliable battery behavior.
- Gemma 4 E4B: realistic on 12 GB–16 GB Snapdragon 8 Gen 3 / 8 Elite or Dimensity 9300/9400 flagships once E2B is verified.
- 12B, 26B-A4B, and 31B: desktop/workstation territory; avoid loading multi-gigabyte desktop GGUFs onto phones.
Android chipset acceleration & memory ceilings
Actual throughput on Android depends heavily on whether your app offloads matrix multiplications to the Qualcomm Hexagon NPU / Adreno GPU or falls back to ARM CPU cores.
| Android SoC Tier | Physical RAM | Safe Model Ceiling | Recommended Path | Practical Expectation |
|---|---|---|---|---|
| Snapdragon 8 Gen 3 / 8 Elite | 12–16 GB | ~6.0–8.0 GB | E2B / E4B via AI Edge Gallery | 16–24 tok/s before thermal step-down |
| Google Tensor G4 / G5 | 12–16 GB | ~5.5–7.5 GB | E2B first, E4B short ctx | 12–16 tok/s; close camera/games first |
| Dimensity 9300 / 9400 | 12–16 GB | ~6.0–8.0 GB | E2B / E4B GPU/NPU delegate | 15–22 tok/s in short offline bursts |
| Mid-range 8 GB (Snapdragon 7 / A-series) | 8 GB | ~3.5–4.0 GB | E2B or Qwen 3 0.6B/1.7B | 4–8 tok/s; keep context <= 2K |
Setup order
- Install Google AI Edge Gallery from Google Play or the verified official GitHub release.
- Confirm at least 4–6 GB of free storage and close background camera, browser, or game apps before loading.
- Start with Gemma 4 E2B and keep the first test prompt short (text only, 2K context).
- Move to E4B only after the smaller model loads cleanly, answers reliably, and avoids thermal reloads.
What people usually get wrong
- They jump straight to desktop GGUF files on an 8 GB phone, triggering instant Android LMK termination.
- They assume NPU/GPU delegates activate automatically without checking runtime backend settings.
- They run 15-minute continuous benchmarks in warm ambient rooms and mistake thermal clock throttling for a runtime bug.
Thermal throttling, battery drain, and quantization sizing on phones
Unlike laptops or desktops with active cooling fans, smartphones dissipate heat passively through the display glass and aluminum frame. Continuous 100% NPU/GPU inference draws roughly 5 W to 9 W—enough to raise SoC skin temperatures within 8 to 15 minutes and cut sustained token generation speeds by 30% to 50%.
For reliable daily utility on Android, keep prompts focused, favor Q4_K_M or mobile LiteRT int4 bundles over larger 8-bit quants, and unload idle chat sessions before launching camera or navigation apps.
Run an LLM locally on Android in AI Edge Gallery
- Install AI Edge Gallery on the Android phone or tablet.
- Open the local model picker and choose a small model first.
- Download or import Gemma 4 E2B IT, Qwen 3 0.6B, or another phone-sized GGUF.
- Keep the first prompt short and use text only.
- Close background apps if the model reloads or Android kills the process.
- Move up to Gemma 4 E4B IT only after the smaller model stays loaded.
Test Gemma Android fit before a bigger download
- Run the hardware checker from this site on the same Android device.
- Pick the smallest recommended model instead of the largest visible model.
- Confirm free storage before copying any GGUF file.
- Open AI Edge Gallery and run one short chat.
- Stop the test if the phone heats up, reloads, or answers too slowly to use.
Recommended models
Frequently asked questions
- Can I run an LLM locally on Android?
- Yes, you can run an LLM locally on Android with AI Edge Gallery or another local runtime. Start with a small model such as Qwen 3 0.6B or Gemma 4 E2B IT before testing larger downloads.
- What is the best local LLM app for Android?
- AI Edge Gallery is the best first local LLM app Android users should try for Gemma 4. It bundles optimized LiteRT models and keeps setup straightforward on consumer phones.
- Can LM Studio run on Android?
- LM Studio is not the standard Android route; use AI Edge Gallery or PocketPal / Termux llama.cpp builds on Android instead. LM Studio is designed for Mac, Windows, and Linux desktops.
- Which Gemma model should I run on Android?
- Gemma 4 E2B IT is the safest Android starting point. Try E4B only after E2B loads cleanly, stays in memory, and answers without heavy thermal throttling.
- Why does my Android phone reload the local model?
- Android reloads or closes local model apps when Android LMK memory pressure rises above ~55% of physical RAM. Close other apps, lower context length, or step down to E2B / Gemma 3 1B.
- Can I run a 30B model locally on Android?
- No practical Android phone can hold a 26B–31B model comfortably in active RAM. Keep 26B and 31B checkpoints on 24 GB+ desktops or high-RAM Apple Silicon Macs.
Official Android App & APK Download
Download Google AI Edge Gallery from Google Play, review APK security advisories, and get recommended mobile quants.