Runtime

Ollama for Gemma 4

Production guide to running Gemma 4 with Ollama: verified model tags, OLLAMA_FLASH_ATTENTION, Q8_0 KV cache environment flags, Modelfile context tuning, and ollama ps diagnostics.

Ollama excels once you already know your hardware ceiling and want a clean background daemon exposing an OpenAI- and Ollama-native API across macOS, Windows, and Linux.

Rather than acting as a manual quant explorer, Ollama wraps verified weights, Jinja chat templates, and runtime orchestration behind simple commands (`ollama run gemma4:e4b`) and environment variables that govern KV cache memory.

Verified Gemma 4 tags and Modelfile defaults

Always note that Ollama defaults to a conservative context window (typically 4K–8K depending on release and system RAM) unless overridden via Modelfile `PARAMETER num_ctx` or API `options.num_ctx`. Raising `num_ctx` silently increases KV cache footprint.

Ollama TagModel VariantApprox DownloadRecommended GPU / RAM
gemma4:e2bGemma 4 E2B IT (Q4)~3.1 GB6–8 GB GPU or 8 GB Mac
gemma4:e4bGemma 4 E4B IT (Q4)~4.6 GB8 GB GPU or 16 GB Mac
gemma4:12bGemma 4 12B IT (Q4)~7.1 GB12 GB GPU or 16 GB Mac
gemma4:26bGemma 4 26B-A4B MoE~15.8 GB24 GB GPU or 24–36 GB Mac
gemma4:31bGemma 4 31B Dense~19.0 GB24–32 GB GPU or 36 GB+ Mac

Essential Ollama environment variables for low-VRAM stability

Two environment flags drastically reduce OOM crashes and PCIe CPU spillover when running 12B–26B models on 12–16 GB hardware:

# Enable Flash Attention and Q8_0 quantized KV cache before starting ollama serve
export OLLAMA_FLASH_ATTENTION=1
export OLLAMA_KV_CACHE_TYPE=q8_0
export OLLAMA_KEEP_ALIVE=5m
ollama serve

# In another terminal, run Gemma 4 and verify 100% GPU split:
ollama run gemma4:12b
ollama ps

Reading `ollama ps` to spot silent CPU spillover

Whenever Ollama feels unexpectedly slow, run `ollama ps` in a second terminal while a chat is loaded. The `PROCESSOR` column tells you immediately whether weights landed `100% GPU` or split across GPU and slow system RAM (for example `65%/35% CPU/GPU`).

If you see CPU split on a desktop card, lower `num_ctx`, enable `OLLAMA_KV_CACHE_TYPE=q8_0`, or step down to the next smaller Gemma 4 checkpoint.

Custom Modelfiles, system prompts, and IDE integrations

Because Ollama resets context parameters per model tag unless specified, creating a dedicated Modelfile for coding or document review keeps your memory budget predictable across sessions. For instance, setting `PARAMETER num_ctx 16384` and `PARAMETER temperature 0.2` in a custom Modelfile prevents external clients from accidentally allocating massive 128K KV caches that trigger out-of-memory spills.

Once your custom tag is registered (`ollama create gemma4-code -f Modelfile`), point Open WebUI, Continue, Aider, or Python OpenAI SDK scripts to `http://localhost:11434/v1`. Remember to keep `OLLAMA_MAX_LOADED_MODELS=1` on 8 GB to 16 GB machines so Ollama unloads an idle embedding or chat model automatically before loading a larger checkpoint.

On Windows and Linux desktops with multiple GPUs or integrated graphics alongside a discrete RTX card, setting `CUDA_VISIBLE_DEVICES=0` (matching your primary GPU index from `nvidia-smi`) ensures Ollama allocates model tensors strictly onto high-bandwidth GDDR VRAM rather than inadvertently touching slower secondary adapters or saturating shared bus links across the motherboard.

You can also track GPU utilization live in another terminal while running agent workflows so background embedding models do not silently evict your primary Gemma 4 chat model from memory.

# Example Modelfile tuned for 16 GB local coding workflows
FROM gemma4:12b
PARAMETER num_ctx 16384
PARAMETER temperature 0.2
SYSTEM "You are a concise systems engineering assistant."

Run Gemma 4 efficiently in Ollama

  1. Install or update Ollama to the latest release so Gemma 4 model architectures and templates are recognized.
  2. Set `OLLAMA_FLASH_ATTENTION=1` and `OLLAMA_KV_CACHE_TYPE=q8_0` in your shell or Windows environment variables.
  3. Run `ollama run gemma4:e4b` (or `gemma4:26b` on 24 GB hardware) to download and verify chat output.
  4. Execute `ollama ps` to confirm `100% GPU` residency without CPU spillover.
  5. Connect external apps to `http://localhost:11434/v1` using any OpenAI-compatible SDK.

Recommended models

Frequently asked questions

Why does `ollama ps` show `CPU/GPU` split instead of `100% GPU`?
Ollama automatically offloads layers to system RAM when model weights plus the configured `num_ctx` KV cache exceed free VRAM. Enable `OLLAMA_KV_CACHE_TYPE=q8_0`, reduce `num_ctx`, or run a smaller model.
How do I increase context length in Ollama without editing global code?
In an interactive session run `/set parameter num_ctx 16384`, or create a custom Modelfile (`FROM gemma4:e4b` + `PARAMETER num_ctx 16384`) and run `ollama create`.
Can Ollama run Gemma 4 completely offline?
Yes. After the initial `ollama pull` completes, the local daemon serves models on localhost with zero internet connection required.
Does Ollama support multimodal image inputs for Gemma 4?
Yes, recent Ollama releases support image inputs on multimodal Gemma 4 tags (such as E4B, 12B, and 26B) via both the CLI and `/api/chat` or `/v1/chat/completions`.
Why should beginners try LM Studio before Ollama if they hit memory errors?
LM Studio surfaces exact layer counts, visual VRAM bars, and quantization picker toggles directly in a GUI, making borderline 16 GB setups easier to diagnose than headless CLI errors.