Troubleshooting
Gemma 4 Out of Memory
Diagnose and fix Gemma 4 OOM crashes, CUDA allocation errors, Windows PCIe VRAM spillover, macOS swap thrashing, and iOS/Android memory kills.
An out-of-memory failure rarely means your GGUF file is corrupt. Almost every OOM crash happens because total allocated runtime memory—weights plus KV cache plus prompt batch buffers plus OS reserves—crossed your device’s real usable ceiling.
Use this systematic diagnostic matrix to isolate whether you are hitting hard VRAM allocation failure, context-growth overflow mid-conversation, or OS watchdog kills.
OS-specific OOM signatures and root causes
Different operating systems fail in completely different ways when local model memory runs dry:
| Platform | Failure Symptom | Underlying Memory Mechanism | Fastest Concrete Fix |
|---|---|---|---|
| Windows (NVIDIA/AMD) | Model loads, then drops to 2 tok/s or CUDA OOM | WDDM spills excess layers into Shared GPU Memory over PCIe | Cap context, enable Q8_0 KV cache, or step down 1 model/quant |
| macOS Apple Silicon | Beachball, yellow/red Memory Pressure, 1–3 tok/s | Metal wired allocation exceeds ~75% RAM and pages to SSD | Uncheck Keep Model in Memory, reduce -c to 4K–8K, drop to E4B/12B |
| Android Phone | App reloads model or closes silently | Android Low Memory Killer (LMK) kills >55% RAM heap | Close background apps, switch E4B -> E2B |
| iPhone / iPad | App vanishes instantly mid-generation | iOS Jetsam hard kill (0x8badf00d) above ~4.8 GB on 8 GB RAM | Cap context at 2048 tokens, use Q4_K_M E2B |
Six recovery levers ranked by effectiveness
Apply these six changes in exact order rather than randomly reinstalling runtimes or deleting working drivers:
- 1. Step down one checkpoint size (31B -> 26B-A4B -> 12B -> E4B -> E2B): immediately isolates hardware ceiling from software bugs.
- 2. Enable Flash Attention and Q8_0 KV cache (`-fa on -ctk q8_0 -ctv q8_0` in llama.cpp or `OLLAMA_KV_CACHE_TYPE=q8_0` in Ollama): cuts long-context overhead up to 50%.
- 3. Trim context window (`-c 4096` or `8192`): Gemma 4 supports 128K–256K context, which pre-allocates huge KV buffers if left unchecked.
- 4. Reduce prompt batch sizes (`-b 512 -ub 256` or `-b 256 -ub 128`): shrinks temporary compute scratchpad spikes during initial prompt ingestion.
- 5. Drop quantization one notch (`Q6_K` -> `Q4_K_M`, or `Q4_K_M` -> `IQ4_XS` / `Q3_K_M`): saves 1.5 GB to 5 GB of weight memory.
- 6. Close GPU-hungry background apps: Chrome hardware acceleration, Docker containers, Electron apps, and games often consume 1–3 GB of hidden VRAM.
# Recover a borderline 16 GB machine running llama.cpp
llama-server -m gemma-4-12b-it-Q4_K_M.gguf \
-ngl 99 -fa on -ctk q8_0 -ctv q8_0 -c 4096 -b 256 -ub 128Preventing multi-model conflicts and browser VRAM theft
Desktop machines rarely hand 100% of physical VRAM to your inference engine. On Windows and Linux workstations driving high-refresh 4K monitors, desktop compositors, Chromium browsers, Slack/Discord Electron windows, and container runtimes quietly lock 1.2 GB to 2.5 GB of dedicated GPU memory before you even start Ollama or LM Studio.
Similarly, running an embedding model alongside a chat model (`OLLAMA_MAX_LOADED_MODELS` > 1) or forgetting to unload an earlier LM Studio session can silently halve usable VRAM. Before downgrading a checkpoint size, run `nvidia-smi` or open Task Manager -> Performance -> GPU to confirm no stale background process is holding VRAM hostage.
On Linux workstations with headless GPUs, driving displays via motherboard integrated graphics frees nearly 100% of dedicated RTX VRAM for model weights and KV caches.
Diagnose and cure a Gemma 4 OOM crash in 4 checks
- Check whether the crash happens immediately on model load (weights > RAM) or after several long turns (KV cache growth).
- Turn on Flash Attention and Q8_0 KV cache quantization in LM Studio, Ollama, or llama.cpp.
- Lower context length to 4096 tokens and close Chrome, games, or Docker containers consuming VRAM.
- Step down one model tier or quant level until Dedicated VRAM / Memory Pressure stays under 80%.
Recommended models
Frequently asked questions
- Why does Gemma 4 run fine on prompt 1 and crash on prompt 10?
- Each turn appends tokens to the active KV cache until total memory crosses your VRAM or OS limit. Enable Flash Attention + Q8_0 KV cache and lower maximum context length.
- Does lowering batch size (`-b`) reduce quality?
- No. Smaller physical/ubatch sizes (`-b 256 -ub 128`) only slice prompt ingestion into smaller chunks, lowering peak temporary buffer RAM without changing generated tokens.
- Why does my 16 GB RTX 4060 Ti fail to fit a 15.8 GB Q4_K_M model cleanly?
- Because Windows WDDM, your display monitor, and runtime CUDA buffers consume 1–2 GB before any LLM loads, leaving ~14 GB clean VRAM. Use MoE `-ot` expert CPU offloading or step down to 12B.
- Is Q3_K_M safe to avoid out-of-memory errors?
- On 12B–31B models, Q3_K_M can help fit borderline memory budgets, though 3-bit compression shows noticeable degradation on strict code syntax and math compared to Q4_K_M.
- Should I increase Windows pagefile size to fix GPU OOM?
- Increasing SSD pagefile prevents hard app crashes on CPU loads, but running LLM layers out of SSD swap or PCIe shared RAM makes generation unbearably slow. Fit weights cleanly in RAM/VRAM instead.