Google Gemma 4 26B-A4B

Gemma 4 26B-A4B is the mixture-of-experts variant in the Gemma 4 family, released on April 3, 2026, under Apache 2.0. It uses a fine-grained MoE architecture with 128 small experts, of which 8 plus 1 shared expert are active per token — meaning only 3.8B parameters fire during each forward pass, despite having 26B total parameters. This design delivers 97% of the dense 31B model's quality while dramatically reducing inference compute.

The key trade-off to understand is memory versus compute. All 26B parameters must reside in memory (it loads like a large model), but it runs like a small one during inference. This means you need the same VRAM as a full 26B model, but you get throughput closer to a 4B model. For API providers and cloud deployments, this is an excellent deal — OpenRouter offers it at $0.13 per million input tokens, making it the cheapest frontier-quality model available.

Benchmark performance validates the 97% claim. AIME 2026 comes in at 88.3% versus the 31B's 89.2%, GPQA Diamond at 82.3% versus 84.3%, and LiveCodeBench v6 at 77.1% versus 80%. The Arena ELO of 1441 places it firmly in the competitive tier, and MMLU Pro at 82.6% shows broad general knowledge. For most practical tasks, the quality difference from the full 31B is imperceptible.

Like its 31B sibling, it benefits from full Apache 2.0 licensing and day-1 ecosystem support across llama.cpp, Ollama, vLLM, and other popular frameworks. The MoE architecture makes it particularly well-suited for high-throughput serving scenarios where you need to handle many concurrent requests efficiently — the lower per-token compute means more requests per GPU-second.