A post in the Reddit LocalLLaMA community recently drew 821 upvotes with a blunt assessment: Gemma 4 had casually knocked almost every model off the local leaderboard, leaving only Claude Opus 4.6 and GPT-5.2 ahead.
That reaction followed Google DeepMind’s April 2, 2026 release of Gemma 4, a family that spans four models—from the phone-friendly E2B to the server-oriented 31B Dense version. All are released under the Apache 2.0 license, with no commercial-use restrictions.
The excitement is not based on one isolated feature. Gemma 4 combines unusually strong benchmark results with a mixture-of-experts model that can deliver roughly 26B-class capability while activating only about 4B parameters during inference. Its smaller models also use a different embedding design intended to make better use of limited parameters.
The benchmark jump is hard to ignore
<table> <thead> <tr> <th>Benchmark</th> <th>Gemma 4 31B</th> <th>Gemma 4 26B MoE</th> <th>Gemma 4 E4B</th> <th>Gemma 3 27B, previous generation</th> </tr> </thead> <tbody> <tr> <td>MMLU Pro</td> <td>85.2%</td> <td>82.6%</td> <td>69.4%</td> <td>67.6%</td> </tr> <tr> <td>AIME 2026</td> <td>89.2%</td> <td>88.3%</td> <td>42.5%</td> <td>20.8%</td> </tr> <tr> <td>GPQA Diamond</td> <td>84.3%</td> <td>82.3%</td> <td>58.6%</td> <td>42.4%</td> </tr> <tr> <td>Codeforces ELO</td> <td>2150</td> <td>1718</td> <td>940</td> <td>110</td> </tr> </tbody> </table>The comparison with the previous generation is particularly striking. Gemma 3 27B scored 20.8% on AIME, while Gemma 4 31B reached 89.2%. These models are in a similar broad parameter class, yet the new model’s mathematics score is more than four times higher.
A Codeforces ELO of 2150 is also substantial: it is described as roughly corresponding to the top 0.3% of human competitive programmers. On the Arena AI open-model ranking, Gemma 4 31B placed third and the 26B MoE version placed sixth.
Against Qwen 3.5, Gemma 4 31B has a modest edge on MMLU Pro—85.2% versus approximately 82%. The larger point is that both models now occupy the same tier. Open models are no longer merely cheaper substitutes that trail proprietary systems by an obvious margin; they are competing near the top of the field.
Why the 26B model behaves like a 4B model
The 26B model is officially called Gemma 4 26B A4B. The A4B designation refers to its active parameter count: approximately 3.8B parameters are activated during inference, even though the complete model contains 26B parameters.
This is made possible by a Mixture-of-Experts (MoE) architecture. Instead of sending every token through the entire network, the model uses a routing mechanism to select the expert networks most relevant to that token. Many of the model’s parameters remain inactive for any individual inference step.
In practical terms, the 26B MoE model can approach the speed and computational cost of a 4B model while producing results closer to a much larger model. It is not literally identical to a dense 26B model, but it offers a remarkably favorable compromise between capability, memory use, and throughput.
<table> <thead> <tr> <th>Comparison</th> <th>26B MoE, A4B</th> <th>Standard 26B Dense</th> </tr> </thead> <tbody> <tr> <td>Total parameters</td> <td>26B</td> <td>26B</td> </tr> <tr> <td>Parameters activated during inference</td> <td>3.8B</td> <td>26B</td> </tr> <tr> <td>Inference speed</td> <td>Approximately a 4B model</td> <td>Around 6–7 times slower</td> </tr> <tr> <td>Capability</td> <td>Close to 26B Dense</td> <td>26B Dense</td> </tr> <tr> <td>VRAM requirement</td> <td>Lower</td> <td>Higher</td> </tr> </tbody> </table>
This efficiency helps explain why the 26B MoE model ranks sixth rather than falling far behind the 31B Dense version. The dense model still leads on absolute performance, but the MoE version makes up part of the difference by running much more economically.
The design behind the smaller models: Per-Layer Embeddings
MoE explains the efficiency of the 26B model, but it does not explain why the E2B and E4B models outperform the previous generation so convincingly. One of the key ideas there is Per-Layer Embeddings (PLE).
In a conventional Transformer, each token receives an embedding vector when it enters the model. That same representation then travels through the layers, accumulating contextual information along the way. The initial vector therefore has to carry information that may be useful at many different depths of the network, even though a particular layer may need only a small part of it.
PLE adds another embedding table that supplies each token with a separate, smaller vector at every layer. These per-layer vectors are 256-dimensional, compared with 1536 dimensions for the main embedding, and can be stored in flash memory and loaded at the beginning of inference.
The result is that a token can carry different layer-specific signals as it moves through the network. The token cat, for example, might be treated as a noun at one layer and as a small animal at another. Each layer receives a signal about what that token means at that stage instead of relying entirely on one static representation created at the input.
The broader idea is simple: smaller models do not always need more parameters. They can also benefit from using existing parameters more precisely. That may help explain why the 4B E4B model reached 42.5% on AIME 2026, while the previous 27B model managed only 20.8% despite having roughly seven times as many parameters.
Four models, four different use cases
<table> <thead> <tr> <th>Model</th> <th>Effective parameters</th> <th>Main characteristics</th> <th>Minimum hardware</th> <th>Suggested hardware</th> </tr> </thead> <tbody> <tr> <td>E2B</td> <td>Approximately 2B active</td> <td>Phone and embedded use; video and audio support</td> <td>4GB RAM</td> <td>Phone or Raspberry Pi</td> </tr> <tr> <td>E4B</td> <td>Approximately 4B active</td> <td>Lightweight multimodal agents; audio support</td> <td>8GB RAM</td> <td>MacBook with an M-series chip</td> </tr> <tr> <td>26B MoE, A4B</td> <td>3.8B active</td> <td>Strongest efficiency-to-performance ratio; speed close to 4B</td> <td>16GB VRAM</td> <td>RTX 4090 or RTX 4080</td> </tr> <tr> <td>31B Dense</td> <td>31B</td> <td>Highest performance; suitable for fine-tuning</td> <td>24GB VRAM</td> <td>H100 or A100</td> </tr> </tbody> </table>For most users, the choice is fairly straightforward:
- A laptop or M-series Mac is best paired with E4B: it is fast, supports multimodal workloads, and is light enough for everyday use.
- A desktop GPU with 16–24GB of VRAM can run the 26B MoE model and offers the strongest overall local option without requiring datacenter hardware.
- Research workflows that require fine-tuning are better suited to 31B Dense.
Running Gemma 4 locally
Ollama
Ollama is the simplest starting point because it handles model downloads and quantization automatically. The commands are:
# 运行 Gemma 4 E4B（轻量版，大多数人的起点）
ollama run gemma4:4b
# 运行 26B MoE（需要 16GB+ VRAM）
ollama run gemma4:27b
Once running, Ollama exposes a local API compatible with the OpenAI format. That makes it possible to connect Gemma 4 to Cursor, Open WebUI, or other tools that support OpenAI-style endpoints.
llama.cpp
Users looking for more control or potentially higher performance can use llama.cpp with a GGUF build:
# 直接用 Hugging Face GGUF 格式
llama-server -hf ggml-org/gemma-4-26b-a4b-it-GGUF:Q4_K_M
Q4_K_M is a 4-bit quantization format. The 26B MoE model reportedly needs around 10–12GB of VRAM in Q4, which puts it within comfortable reach of an RTX 4080.
LM Studio
LM Studio provides a graphical alternative. Search for gemma-4, download the model, and load it through the interface. It can also provide a local API service, making it useful for people who do not want to work from the command line.
What can it replace?
Gemma 4 is not a universal replacement for the strongest cloud models. A realistic comparison with Claude Sonnet 4.6 looks like this:
<table> <thead> <tr> <th>Capability</th> <th>Gemma 4 26B, local</th> <th>Claude Sonnet 4.6, cloud</th> </tr> </thead> <tbody> <tr> <td>Code generation for complex projects</td> <td>★★★★☆</td> <td>★★★★★</td> </tr> <tr> <td>Mathematical reasoning</td> <td>★★★★★</td> <td>★★★★★</td> </tr> <tr> <td>Instruction following</td> <td>★★★★☆</td> <td>★★★★★</td> </tr> <tr> <td>Long context</td> <td>256K ✅</td> <td>200K ✅</td> </tr> <tr> <td>Multimodal input, including images and video</td> <td>✅ Native support</td> <td>✅</td> </tr> <tr> <td>Data privacy</td> <td>✅ Fully local</td> <td>❌ Data is sent to the cloud</td> </tr> <tr> <td>Latency</td> <td>No network delay</td> <td>Dependent on the connection</td> </tr> <tr> <td>Cost</td> <td>Electricity, approaching zero for some workloads</td> <td>Charged by token</td> </tr> </tbody> </table>Gemma 4 is especially compelling for local code review, mathematical and logical reasoning, multimodal document or screenshot analysis, and high-volume batch processing. Code can stay on the machine, routine reasoning tasks can run without an internet connection, and large offline workloads do not incur per-token API fees.
Cloud models such as Claude still have an advantage for complicated multi-step engineering tasks, especially workflows that depend on agent capabilities such as Claude Code. They are also preferable when a task requires live information, extensive tool use, or exceptionally polished writing.
Why this release matters
The significance of Gemma 4 is not limited to its benchmark scores. The larger signal is that the gap between open and proprietary models is narrowing faster than many developers expected.
Only a short time ago, local models were often described as usable but compromised. Gemma 4 changes the conversation. Its 31B model’s 89.2% result on AIME 2026 is already beyond the public results reported for most commercial models, while the 26B MoE version makes that level of capability much more accessible on consumer hardware.
For developers, three consequences stand out:
- Local AI workflows are becoming practical. The use case is expanding beyond code completion toward genuine task automation.
- Privacy-sensitive industries have more credible options. Medical, legal, and internal enterprise data can be processed without leaving the local environment.
- The economics of high-throughput inference are changing. When a model can run on existing hardware without per-token charges, the cost advantage becomes significant at scale.
Gemma 4 is only one point on that trajectory, but Google’s Apache 2.0 licensing and unrestricted commercial use send a clear signal about how seriously it is approaching open models.
Before installing, check the available hardware: 16GB or more of VRAM is a sensible target for the 26B MoE model, while E4B is more suitable for systems with around 8GB of memory. A practical first test is to launch the 4B version through Ollama or LM Studio, connect it to an existing local interface such as Open WebUI, and compare it against the coding and reasoning tasks used most often.