Mac
Gemma 4 on Mac
Apple Silicon unified memory sizing for Gemma 4 across M1–M6 Macs: E4B vs 12B vs 26B-A4B MoE, LM Studio vs llama.cpp, Metal wired caps, and KV cache flags.
Apple Silicon Macs are uniquely effective for local LLMs because unified memory lets the Metal GPU read weights directly out of system RAM without copying layers across a slow PCIe bus.
However, macOS reserves part of unified memory for the compositor, Spotlight, browser tabs, and kernel buffers—typically capping comfortable Metal wired memory around 75% of physical RAM.
Apple Silicon unified memory sizing table
Match your Mac’s installed unified RAM and memory bandwidth to the checkpoint that delivers responsive generation without pushing macOS into SSD swap.
| Mac Unified Memory | Practical GPU Budget | Daily Sweet Spot | Stretch Checkpoint | Notes |
|---|---|---|---|---|
| 8 GB (Base M1/M2/M3) | ~5.5–6.0 GB | Gemma 4 E2B (Q4/Q6) | Gemma 4 E4B (2K ctx) | Close heavy browser tabs before running |
| 16 GB (Air / Pro / Mini) | ~12.0 GB | Gemma 4 E4B / 12B | 26B-A4B (tight 4K–8K) | Disable Keep Model in Memory if swapping |
| 24 GB – 36 GB (Pro / Max) | ~18–27 GB | Gemma 4 26B-A4B (Q4/Q6) | Gemma 4 31B (Q4_K_M) | Ideal balance for 26B MoE + 32K ctx |
| 48 GB – 128 GB (Max / Ultra) | 36–96 GB+ | Gemma 4 31B (Q6/Q8) | 70B–120B+ models | High bandwidth enables fast long context |
Recommended runtimes
- LM Studio: cleanest visual desktop workbench with native Apple Metal / MLX support, easy GGUF management, and instant OpenAI local server.
- Ollama: best when you want a background macOS daemon serving localhost:11434 to Cursor, Continue, or custom Python scripts.
- llama.cpp (Homebrew / source): gives fine-grained control over Flash Attention (-fa), KV cache quantization (-ctk q8_0), and wired memory testing.
Managing macOS Metal wired memory and swap
When a model fits on disk (15.8 GB) on a 16 GB Mac, macOS defaults limit GPU wired allocation near 10.6–12 GB. Pushing past that boundary forces macOS to page weights to the internal SSD—slowing generation below 3 tok/s and spiking kernel_task CPU usage.
Before forcing oversized models on 16 GB machines, drop context to 4K–8K, enable Q8_0 KV cache, or run Gemma 4 12B / E4B for consistently crisp 25–60 tok/s responses.
Power users on 24 GB to 64 GB Macs running `llama.cpp` or LM Studio often inspect memory pressure in Activity Monitor: green memory pressure with zero Swap Used confirms the entire model and KV cache are pinned cleanly inside high-speed unified LPDDR memory.
For developers building automated coding pipelines on 32 GB+ Macs, pairing Flash Attention with Q8_0 KV cache lets you sustain 32K–64K context windows across Gemma 4 26B-A4B without triggering disk paging or thermal fan spikes on MacBook Pro and Mac Studio hardware.
Run an LLM locally on Mac with LM Studio
- Install LM Studio on the Mac and open the Discover tab.
- Search for Gemma 4 E4B IT, Gemma 4 12B IT, or Gemma 4 26B A4B IT.
- Download the Q4_K_M build first instead of a larger quant.
- Open Chat, select the model, enable Flash Attention, and start with 4K or 8K context.
- Watch macOS Activity Monitor Memory Pressure during the first answer.
- Move up one model size only after the smaller model stays green and responsive.
Run a local LLM on Mac with llama.cpp
- Install llama.cpp via Homebrew (`brew install llama.cpp`) or build with `GGML_METAL=ON`.
- Pull a verified GGUF matching your unified memory tier.
- Launch with `-ngl 99 -fa -ctk q8_0 -ctv q8_0 -c 8192`.
- Run a one-prompt smoke test before long document work.
- Lower context or model size if Activity Monitor Memory Pressure turns yellow/red.
Recommended models
Frequently asked questions
- Can I run an LLM locally on Mac?
- Yes, Apple Silicon Macs run local LLMs natively on Metal via LM Studio, Ollama, or llama.cpp. Start with Gemma 4 E4B or 12B on 16 GB Macs, and 26B-A4B MoE on 24 GB+ Macs.
- What is the best Mac to run a local LLM?
- Unified memory capacity and memory bandwidth determine Mac AI performance: Pro, Max, and Ultra chips (with 24 GB to 128 GB unified RAM and 273–819+ GB/s bandwidth) outperform higher-clocked base chips with smaller RAM pools.
- Can a 16 GB Mac mini M4 run Gemma 4 26B A4B?
- It can load Q4_K_M tightly if you close background apps and cap context around 4K–8K with Q8_0 KV cache, though Gemma 4 E4B and 12B provide far smoother everyday responsiveness on 16 GB.
- Should I use LM Studio on an Intel Mac?
- Intel Macs lack unified Apple Silicon memory and fast Metal neural bandwidth; stick to smaller Q4_K_M models (E2B/E4B) and short context windows if running on CPU or older discrete Radeon GPUs.
- Why is my local LLM slow on Mac?
- When model weights plus KV cache exceed ~75% of physical unified memory, macOS swaps memory pages to SSD. Dropping one quant tier or capping context immediately restores full Metal bandwidth.
- Does MLX run faster than GGUF on Apple Silicon?
- MLX-quantized checkpoints inside LM Studio often match or slightly beat llama.cpp Metal throughput on M2–M4 Macs while simplifying memory management, though GGUF remains the most portable cross-platform format.
Official Mac & Apple Silicon Download
Get native Apple silicon installers (LM Studio, Ollama) and optimize memory bandwidth for M1–M6.