Troubleshooting

Gemma 4 Slow Loading & Crawling Speeds

Fix slow Gemma 4 cold model loads and sluggish token generation: NVMe mmap bottlenecks, silent PCIe offload spillover, CPU thread saturation, and Metal shader warmup.

Local AI slowness usually hides two completely different bottlenecks: slow **cold loading** (moving gigabytes of GGUF tensors from disk into RAM/VRAM) and slow **generation throughput** (computing tokens once loaded).

If a model takes 60+ seconds to load or streams under 5 tokens per second on desktop hardware, pinpoint which stage is stalling before changing runtimes.

Diagnosing slow cold loads vs. slow token generation

Match your symptom in the diagnostic table below to target the true bottleneck:

Observed SymptomLikely BottleneckDiagnostic CheckImmediate Remediation
60–180s load, then fast tok/sExternal USB drive, HDD, or cold OS page cacheCheck drive speed / Task Mgr Disk %Move GGUF folder onto internal NVMe SSD
Fast load, 1–4 tok/s generationPartial CPU offload over PCIe or CPU fallbackRun `ollama ps` or check Dedicated VRAMDrop quant/ctx so 100% of weights stay in VRAM
Fast text gen, slow initial reply (TTFT)Large prompt ingestion or missing Flash AttnMeasure prompt eval (`pp`) speedEnable `-fa on`, lower batch only if OOM
Slow only on the very first run on Mac/AndroidOne-time Metal or OpenCL shader compilationCompare prompt 1 vs prompt 2Keep runtime resident (`OLLAMA_KEEP_ALIVE`)

Understanding `mmap`, `--no-mmap`, and CPU thread counts

By default, `llama.cpp` memory-maps (`mmap`) GGUF files from disk so the OS loads tensors lazily into page cache. On fast NVMe SSDs with ample RAM, `mmap` makes repeated reloads near-instant. On low-RAM systems where weights barely fit, `mmap` page faults during generation can cause stuttering—testing `--no-mmap` forces sequential RAM loading upfront.

For CPU-offloaded workloads, setting thread count (`-t`) higher than your **physical performance cores** (for example setting 16 or 24 threads on an 8P+8E hybrid CPU) creates thread contention on efficiency cores and drops speed by 20–40%. Set `-t` equal to physical performance cores only.

# Measure exact prompt processing (pp) vs token generation (tg) speed with llama-bench
llama-bench -m ./gemma-4-e4b-it-Q4_K_M.gguf -ngl 99 -fa 1

Preventing cold-start reload penalties in server and agent setups

If you connect coding assistants or local scripts to Ollama, note that Ollama unloads idle models from VRAM after 5 minutes by default (`OLLAMA_KEEP_ALIVE=5m`). Every time a cron job or editor plugin waits 6 minutes between prompts, the engine must reload gigabytes from disk and re-allocate KV cache buffers from scratch.

On dedicated workstations with enough RAM/VRAM, setting `OLLAMA_KEEP_ALIVE=30m` or running a persistent `llama-server` process eliminates repeated cold loads completely. Conversely, on tight 16 GB laptops where other applications compete for memory, keep `OLLAMA_KEEP_ALIVE` short (or toggle off Keep Model in Memory in LM Studio) so macOS or Windows can reclaim memory promptly.

Checking thermal throttling and laptop power profiles

Unplugged laptops frequently drop GPU power limits and unified memory clock frequencies by 40% to 60% under battery-saver profiles. Always test baseline throughput on AC power with High Performance mode enabled before concluding a model is permanently too slow for your GPU.

In addition, verify that laptop cooling vents are unobstructed during sustained batch runs: thermal saturation on thin ultrabooks and passively cooled tablets causes dynamic clock drops after the first 2–3 minutes of continuous decoding, especially on multimodal vision workloads or high-batch prompt evaluations.

If cold load times jump suddenly after a Windows or macOS OS update, verify that cloud-sync folders (iCloud Drive, OneDrive, or Dropbox) are not continuously syncing or evicting your multi-gigabyte `.gguf` directory to remote storage.

Isolate and eliminate Gemma 4 slowness

  1. Verify the model folder lives on an internal NVMe SSD rather than a spinning HDD, SD card, or network share.
  2. Check `ollama ps` or LM Studio memory telemetry to confirm zero layers spilled into host RAM.
  3. Set CPU threads `-t` strictly to physical performance cores if running CPU or hybrid offload.
  4. Enable Flash Attention (`-fa on`) to accelerate long-context prompt ingestion.

Recommended models

Frequently asked questions

Why does Gemma 4 load instantaneously the second time I open it?
Your operating system caches memory-mapped (`mmap`) GGUF weight pages in free system RAM after the first load until other programs reclaim that RAM.
Why is prompt processing slow when pasting a 10-page document?
Prompt ingestion (prefill) is compute-bound rather than memory-bandwidth bound. Enabling Flash Attention and avoiding CPU fallback keeps prompt eval speed above several hundred tokens/sec.
Does setting CPU threads to max logical threads make llama.cpp faster?
Almost never. Hyper-threading and efficiency cores saturate shared memory bandwidth; matching `-t` to physical performance core count consistently runs faster.
Can antivirus software slow down GGUF loading on Windows?
Yes. Real-time Windows Defender scanning of 5–20 GB `.gguf` files on every read can add 20–40 seconds to cold loads. Excluding your local `.ollama` or LM Studio models directory helps significantly.
When should I give up tuning a slow model and switch checkpoints?
If tuning stays below 6–8 tokens/sec after verifying 100% GPU offload and NVMe storage, your hardware bandwidth ceiling requires a smaller model such as E4B or 12B (or MoE 26B-A4B over dense 31B).