Runtime

AI Edge Gallery for Gemma 4

Complete mobile guide to Google AI Edge Gallery: LiteRT .task bundles, Snapdragon NPU vs GPU delegates, Android LMK memory limits, and safe APK sourcing.

AI Edge Gallery is Google’s open-source on-device AI showcase app built on LiteRT (formerly TensorFlow Lite) and MediaPipe GenAI tasks. For Android users, it removes the friction of compiling llama.cpp via Termux or manually tweaking NDK flags.

Instead of loading generic desktop GGUF files, AI Edge Gallery downloads pre-compiled `.task` bundles tuned specifically for mobile ARM CPUs, Adreno/Mali OpenCL GPUs, and Qualcomm Hexagon NPUs.

Backend delegates: NPU vs. GPU vs. CPU

Understanding which compute accelerator AI Edge Gallery targets on your phone explains most differences in speed, battery drain, and heat.

Delegate BackendSupported HardwareRelative SpeedThermal & Memory Profile
NPU / HTP DelegateSnapdragon 8 Gen 3 / 8 Elite, flagship DimensityFastest & lowest powerBest sustained efficiency; strict tensor shape support
GPU (OpenCL / Vulkan)Adreno 7xx/8xx, Immortalis, Tensor MaliFast (12–22 tok/s E2B)Higher battery draw; shares unified memory buffers
CPU (ARMv9 KleidiAI)All 64-bit Android devicesSlowest (4–8 tok/s)Safest fallback if GPU shaders crash or glitch

Android LMK memory rules & safe downloads

Android’s Low Memory Killer (LMK) begins terminating background processes and then large foreground heaps once uncommitted system RAM falls below ~800 MB. On a 12 GB phone, OS services and display buffers already consume 4–6 GB, leaving roughly 6 GB of safe headroom.

Always install AI Edge Gallery strictly from official Google Play listings or the verified `google-ai-edge/gallery` GitHub Releases page. Never sideload third-party "mod APK" packages claiming to unlock 31B desktop models on mobile—those binaries are fake or malicious.

  • Start with Gemma 4 E2B `.task` (~1.6–2.0 GB download) before attempting E4B.
  • Close heavy camera, social, and game apps in Recents prior to loading a model.
  • Keep multimodal image uploads modest and cap multi-turn chat history before switching tasks.

Troubleshooting LiteRT compilation stalls and mobile storage limits

When opening a checkpoint inside AI Edge Gallery for the first time, Android compiles device-specific OpenCL or Hexagon NPU execution kernels and maps weight pages into memory. That initial warmup can take 15 to 45 seconds depending on flash storage UFS speed (UFS 3.1/4.0 vs older UFS 2.2) and thermal state. Do not force-close the app during first-load kernel compilation.

If a model still fails to initialize on an 8 GB phone, check two common blockers: first, verify you have at least 3–5 GB of free internal flash storage beyond the model download itself so Android does not starve `/data/cache`; second, switch the inference delegate from NPU to GPU or CPU in the app prompt settings to isolate vendor driver bugs on newly released Android OS updates.

AI Edge Gallery vs. PocketPal and Termux llama.cpp

Choose AI Edge Gallery whenever your goal is fast, zero-boilerplate offline validation of official Google Edge models, multimodal photo prompts, or NPU acceleration benchmarks. Switch to PocketPal or a Termux `llama.cpp` CLI build only if you need raw Hugging Face GGUF support, custom system prompt templates, or specialized quantizations like IQ4_XS and Q5_K_M across third-party Qwen and Llama checkpoints.

Before running multi-turn vision or voice experiments on Android, monitor battery temperature in Device Care or CPU-Z: keeping sessions under 10 minutes on 8 GB phones avoids aggressive thermal clock throttling and preserves consistent response speed.

Deploy Gemma 4 E2B cleanly in AI Edge Gallery

  1. Install AI Edge Gallery from Google Play or the verified google-ai-edge GitHub repository.
  2. Open the LLM Chat / Prompt Lab tab and choose Gemma 4 E2B.
  3. Select GPU or NPU accelerator backend in model configuration settings.
  4. Wait for weight compilation on first launch (10–30 seconds), then send a short text prompt.
  5. Switch backend to CPU only if the GPU delegate fails shader verification on your specific SoC.

Recommended models

Frequently asked questions

Is AI Edge Gallery completely offline once models download?
Yes. After downloading the model bundle over Wi-Fi, all token inference runs locally on your Android CPU, GPU, or NPU with zero cloud API calls.
Can I load arbitrary Hugging Face GGUF files into AI Edge Gallery?
AI Edge Gallery primarily targets LiteRT / MediaPipe `.task` bundles. If you want to import custom Hugging Face GGUF files on Android, use PocketPal AI or Termux with llama.cpp.
Why is the first response slower than the second prompt?
Mobile OpenCL and NPU delegates compile compute graphs and initialize memory buffers on the first run. Subsequent turns stay warm unless Android unloads the app.
Is there an official iOS version of AI Edge Gallery?
Check official google-ai-edge release notes for current iOS TestFlight/App Store status; on iPhone and iPad today, PocketPal and MLX/llama.cpp Swift apps remain the most reliable local routes.
Where can I safely get the AI Edge Gallery APK without malware?
Download exclusively from Google Play or the official github.com/google-ai-edge/gallery releases page. Avoid unofficial APK mirror domains.