Windows
Gemma 4 on Windows
Check Windows NVIDIA RTX, AMD Radeon, Intel Arc, and system RAM sizing for Gemma 4: avoiding PCIe Shared GPU Memory slowdown, CUDA vs Vulkan, and Ollama/LM Studio setup.
On Windows PCs, local LLM performance hinges on one critical boundary: whether every layer of your chosen checkpoint fits inside dedicated GPU VRAM, or overflows across the PCIe bus into system RAM.
Because Windows WDDM automatically maps "Shared GPU Memory" when dedicated VRAM fills up, an oversized model often loads silently without crashing—and then crawls at 2–4 tokens per second instead of 40–70 tok/s.
Windows GPU VRAM & runtime matrix
Use your dedicated GPU VRAM (not Windows Total Available Graphics Memory) when choosing a Gemma 4 checkpoint.
| Windows GPU Tier | Dedicated VRAM | Best Full-VRAM Fit | Backend Recommendation | Bandwidth Reality |
|---|---|---|---|---|
| RTX 3050 / 4050 Laptop | 6–8 GB | Gemma 4 E2B / E4B (Q4_K_M) | CUDA via LM Studio / Ollama | 35–55 tok/s when kept in VRAM |
| RTX 3060 12GB / RTX 4070 | 12 GB | Gemma 4 12B (Q4_K_M, 8K ctx) | CUDA + Flash Attention | 35–60 tok/s; avoid 26B full load |
| RTX 4060 Ti 16GB / 4080 / 5080 | 16 GB | Gemma 4 12B (Q6_K) / 26B (offload) | CUDA (-ctk q8_0) | 40–90+ tok/s depending on bus |
| RTX 3090 / RTX 4090 / 5090 | 24–32 GB | Gemma 4 26B-A4B & 31B (Q4/Q6) | CUDA + llama.cpp / LM Studio | 70–120+ tok/s full offload |
| AMD RX 7800 XT / 7900 XTX / Arc | 16–24 GB | E4B / 12B / 26B-A4B | Vulkan or ROCm (7900/9070) | Verify GPU compute in Task Mgr |
Avoiding the Windows Shared GPU Memory trap
In NVIDIA Control Panel or Windows Task Manager, exceeding physical VRAM by even 300 MB pushes weights into shared host RAM. Keep GPU utilization clean by closing Chrome hardware-accelerated tabs, OBS, and games, or enabling KV cache quantization (-ctk q8_0 -ctv q8_0).
Best runtime choices
- LM Studio: easiest Windows GUI with clear GPU Offload sliders and visible VRAM allocation bars.
- Ollama for Windows: ideal if you want a Windows tray daemon exposing localhost:11434 to coding extensions.
- llama.cpp (CUDA / Vulkan): best when pairing a 12–16 GB GPU with 32–64 GB DDR5 RAM for MoE expert CPU offloading (-ot).
Dual-GPU setups, laptop Optimus switching, and PCIe lane checks
Windows gaming laptops frequently ship with hybrid graphics (integrated Intel/AMD iGPU paired with a discrete NVIDIA RTX GPU). Always confirm your runtime binds to the discrete RTX card rather than allocating shared system memory on the integrated adapter.
For desktop builders pairing two used 12 GB or 24 GB GPUs, `--split-mode layer` in llama.cpp or automatic multi-GPU splitting in LM Studio lets you run 31B Q8_0 or 70B+ checkpoints cleanly inside combined GDDR VRAM, even across x8/x4 secondary PCIe slots, because generation splits sequentially rather than hammering inter-GPU bandwidth on every tensor.
Keep NVIDIA Studio or Game Ready drivers current, verify Windows Hardware-Accelerated GPU Scheduling does not starve CUDA allocations under heavy desktop compositing, and reserve at least 1.5 GB of VRAM headroom on the primary display-connected GPU for Windows DWM and browser tabs.
Run an LLM locally on Windows with LM Studio
- Install LM Studio for Windows and ensure CUDA or Vulkan runtime extensions are updated in Settings.
- Search for Gemma 4 E4B IT or Gemma 4 12B IT in Discover.
- Download Q4_K_M first and watch the GPU Offload slider before loading.
- Enable Flash Attention and Q8_0 KV Cache if VRAM is 8 GB to 12 GB.
- Open Task Manager -> Performance -> GPU and confirm Dedicated GPU Memory rises without Shared GPU Memory spiking.
Install a private LLM on Windows with Ollama
- Install Ollama for Windows and open PowerShell or Windows Terminal.
- Run `ollama run gemma4:e4b` (or `gemma4:12b` on 12 GB+ GPUs).
- Open a second terminal and run `ollama ps` to confirm `100% GPU` residency.
- If `ollama ps` shows CPU split (e.g. 60%/40% GPU/CPU), drop context or pick a smaller model.
Recommended models
Frequently asked questions
- Can I run an LLM locally on Windows?
- Yes. LM Studio, Ollama, and llama.cpp all ship native Windows builds supporting NVIDIA CUDA, AMD Vulkan/ROCm, Intel Arc, and CPU fallback.
- How do I check if Gemma 4 is actually using my NVIDIA or AMD GPU on Windows?
- Run `ollama ps` to verify 100% GPU offload, or check Windows Task Manager -> GPU -> Dedicated GPU Memory usage and 3D/Compute activity while generating tokens.
- Why does a model load on Windows RTX 4060 8GB but run at 2 tokens/sec?
- Windows WDDM spills excess layers into Shared GPU Memory across the PCIe bus once 8 GB dedicated VRAM fills up. Drop one model size or reduce context to keep 100% of weights in dedicated VRAM.
- Can I run a private LLM on Windows completely offline?
- Yes. Once weights are downloaded via LM Studio, Ollama, or Hugging Face, local inference runs offline with zero telemetry or cloud prompt transmission.
- Which GPU is best value for running Gemma 4 26B-A4B on Windows?
- Used RTX 3090 (24 GB) or RTX 4090 (24 GB) cards hold the entire 26B-A4B Q4_K_M checkpoint plus generous context inside fast GDDR6X VRAM.
Official Windows Download & Setup Guide
Get official download links for Windows (LM Studio, Ollama, llama.cpp) or read our deep setup walkthrough.