back to local model checker

Qwen 3.8 Flash 128B

MoEToolsThinkingVisionPopular

Alibaba · 125B (6B active) · Mixture of Experts

125B sparse MoE (6B active) — runs on 12GB VRAM gaming PCs with Strata engine up to 96 tok/s

Apache 2.0
Downloads

0

Likes

0

Released

2026-08

Context

128K tokens

Use cases

chatcodereasoningvision
Total experts512
Active experts8
Active params6.0B

Your hardware

16 GB Reference Desktop

Quant
Q4_K_M
Memory
64.5 GB
Speed
--
Fit
Too heavy

Quantization options

What is quantization?
QuantBitsVRAMDiskQualityStatus
Q2_K240.5 GB38.2 GBlowToo heavy · ~1 t/s
Q3_K_M356.5 GB53.5 GBmoderateToo heavy · ~1 t/s
Q4_K_M464.5 GB61.1 GBgoodToo heavy · ~1 t/s
Q5_K_M580.5 GB76.4 GBgoodToo heavy · ~1 t/s
Q6_K696.5 GB91.7 GBexcellentToo heavy · ~1 t/s
Q8_08128.6 GB122.2 GBexcellentToo heavy · ~1 t/s
F1616256.6 GB244.5 GBlosslessToo heavy · ~1 t/s

About this model

Qwen 3.8 Flash 128B (Qwen3.8-Flash-Next) is Alibaba’s flagship open-weight sparse Mixture-of-Experts (MoE) foundation model, featuring 125 billion total parameters with only 6 billion activated per token. By combining Gated DeltaNet linear attention with Qwen Sparse Attention (QSA) and off-accelerator n-gram embedding tables, it delivers frontier reasoning and coding capabilities with an extraordinarily lightweight active memory footprint.

With modern local inference engines like Strata (github.com/Niko1221/Strata), Qwen 3.8 Flash 128B can run directly on consumer gaming PCs equipped with 12 GB VRAM graphics cards (such as NVIDIA GeForce RTX 5070, RTX 4070, or RTX 3060) and 32 GB to 64 GB of system RAM. In benchmark testing on an RTX 5070 (12 GB), the Q2_0 quantization achieves output generation speeds of up to 94 to 96 tokens per second, with prompt reading reaching 2,650 tokens/s.

Highlights

  • 12GB VRAM Consumer GPU FitSparse MoE architecture (6B active per token) allows full 125B intelligence to fit on consumer gaming hardware via tiered VRAM/RAM offloading.
  • Up to 96 tok/s Generation SpeedAchieves real-time interactive generation (94–96 tokens/sec on RTX 5070; 100–140 tokens/sec on RTX 3090) faster than human reading speed.
  • Frontier Hybrid ArchitectureEmploys Gated DeltaNet, QSA sparse attention, and a 128K context window for fast document processing and deep reasoning.

System requirements by device

Memory planning targets 32 GB minimum RAM/VRAM and 64 GB recommended memory. The baseline quant is Q4_K_M, estimated at 64.5 GB memory and 61.1 GB disk. Actual runtime usage shifts with context length, KV cache precision, and OS background overhead.

DeviceNotes
Mac (Apple silicon)Requires a 48 GB–192 GB Mac Studio or MacBook Pro Max/Ultra with enough unified wired budget for 64.5 GB.
Windows / Linux GPUUse dual/multi-GPU CUDA splitting (--tensor-split) or MoE CPU expert offloading (-ot) paired with 32–128 GB DDR5 system RAM.
Phones & Compact Edge BoardsNot practical on phones (32 GB min RAM exceeds Android LMK and iOS Jetsam limits). Pick a 2B–4B model on mobile.

Start with a short session and leave memory for the operating system. A file fitting on disk does not guarantee that it will load in memory.

Estimated memory by quantization

These are rough estimates, not tested minimum requirements. Check the actual file and runtime documentation; context, offloading and operating-system overhead can change the memory needed.

Quant TierEstimated memoryDisk SizeQuality
Q2_K40.5 GB38.2 GBlow
Q3_K_M56.5 GB53.5 GBmoderate
Q4_K_M64.5 GB61.1 GBgood
Q5_K_M80.5 GB76.4 GBgood
Q6_K96.5 GB91.7 GBexcellent
Q8_0128.6 GB122.2 GBexcellent
F16256.6 GB244.5 GBlossless

How to run Qwen 3.8 Flash 128B on your device

Start with a short prompt and the smallest quant that still gives the quality you need. A clean first run is more useful than loading a larger file with no memory headroom.

Windows

Run Qwen 3.8 Flash 128B on Windows with Strata

  1. Download Strata from github.com/Niko1221/Strata and unzip to an SSD with 80GB+ free space.
  2. Double-click START-HERE.bat. The installer detects your NVIDIA or AMD GPU and recommends the best model quant.
  3. Select Q2_0 or IQ2_XS (for 64GB RAM) or Coder (for 32GB RAM), and let Strata download model weights.
  4. Once started, access the local chat UI at http://127.0.0.1:8080 or connect coding agents via localhost OpenAI-compatible API.
Device how-to guide
Mac

Run Qwen 3.8 Flash on Apple Silicon

  1. For Apple Silicon Macs with unified memory (64GB or 128GB RAM), download the GGUF weights from Hugging Face (AtomicChat/Qwen3.8-Flash-Next-GGUF).
  2. Launch via llama.cpp or LM Studio with Metal acceleration enabled.
  3. Set context size to 32K or 128K according to available unified memory headroom.
Device how-to guide

Qwen 3.8 Flash 128B vs alternatives

ModelParamsContextMin RAMLicense
Qwen 3.8 Flash 128B125B (6B active)128K32 GBApache 2.0
Qwen 3.8 27B27B256K15.1 GBApache 2.0
Qwen 3.5 122B-A10B122B (10B active)256K68.2 GBApache 2.0
Qwen 3 Coder 480B480B (35B active)256K268.2 GBApache 2.0
DeepSeek R1 Distill 14B14B64K7.8 GBMIT

Use this table to choose the smallest nearby model that still matches the job. A related model with lower RAM can be the better local choice when the current page fits only at the edge of available memory.

Frequently asked questions

Can Qwen 3.8 Flash 128B really run on a 12GB VRAM graphics card?
Yes. Through the Strata inference engine (or modular MoE offloading), only the active experts (6B active parameters) and KV cache are pinned to GPU VRAM, while inactive experts and off-accelerator embedding tables are streamed from system RAM. On an NVIDIA RTX 5070 (12GB) paired with 64GB RAM, the Q2_0 quantization generates responses at 94–96 tokens per second.
What are the minimum hardware requirements for Qwen 3.8 Flash?
You need an NVIDIA RTX 20/30/40/50 series GPU (or AMD RX 7000/9000 series) with at least 12GB VRAM, 32GB to 64GB of system RAM, and approximately 80GB of free SSD storage. A 32GB RAM PC can run the Coder (IQ1_M) version at 55 tok/s, while 64GB RAM runs all quantization sizes.
How fast is Qwen 3.8 Flash on local consumer hardware?
On an RTX 5070 (12GB) with 64GB RAM: Q2_0 generates 94 tokens/s (2,650 tokens/s prompt read); IQ2_XS generates 79 tokens/s; and the Coder edition generates 55 tokens/s. On a 24GB RTX 3090, write speeds reach 100–140 tokens/s. On an AMD RX 9070 XT (16GB), Q2_0 generates around 60 tokens/s.
How do I download and install Qwen 3.8 Flash locally?
You can install it with one click using the open-source Strata engine (github.com/Niko1221/Strata) on Windows or Linux, or download the GGUF weights from Hugging Face (Qwen/Qwen3.8-Flash-Next). For automated AI setup, you can paste the Strata setup prompt into Claude Code, Codex, or Cursor.
How do I download and install Qwen 3.8 flash?
Download a GGUF build for Qwen 3.8 Flash 128B, start with Q4_K_M, and confirm the model fits locally before increasing context or quant quality.