Gemma 4's 12B: Why Dropping Encoders Beats Bigger Models

The encoder-free 12B is the one to run

Google DeepMind's Gemma 4 technical report, published 19 June 2026, contains a counterintuitive finding: the 12B model performs like models several times its size, and it does so by removing the vision and audio encoders entirely. For creators running models locally, this changes the practical calculus of which size to download.

Encoders are the preprocessing front-ends that convert raw pixels or waveforms into tokens the model can understand. In the 12B, the 550-million-parameter vision encoder was replaced by a single matrix multiplication of 35 million parameters operating on raw 48x48x3 image patches. The 305-million-parameter audio encoder was discarded altogether; raw 16kHz audio is cut into 40-millisecond chunks and fed straight into the embedding space. The result: accuracy did not drop. On the FLEURS speech benchmark, the encoder-free 12B hits a 0.063 word error rate on English, versus 0.065 for the E4B, which carries a dedicated 305M audio encoder. Lower is better, so the stripped-down model is actually slightly more accurate.

Why does this matter for local inference? Memory and speed. The 12B needs about 7.65GB of memory when quantised, which fits comfortably on a consumer GPU with 8GB or more. The 31B, by contrast, needs about 19.2GB, pushing it into workstation or cloud territory. And because the 12B has no encoder layers, it avoids the compute and memory overhead those layers add. Three combined changes in the report reduce the model's global key-value cache by 37.5 percent, which directly translates to faster generation and lower memory pressure during long video or audio sessions.

But the size choice is not just about memory. It is about capability. The 12B is the only encoder-free model in the five-size family (E2B, E4B, 12B, 26B-A4B, and 31B). The 26B-A4B and 31B both carry a 550M-parameter vision encoder and have no audio capability at all. That means if your work involves voice — transcribing dialogue, generating speech, or syncing audio to video — the 12B is the only local option that handles raw audio natively. The larger models simply cannot process sound. For a video creator who needs both vision and voice, the 12B is the sweet spot: it sees images via the lightweight matrix multiplication and hears audio via raw chunks, all within a memory footprint that fits on a single consumer card.

The 31B's strong leaderboard position — third among open models on LMArena's text leaderboard, 27th overall — might tempt you to run it anyway. But the report is explicit: the 31B did not beat models twenty times its size. It is statistically tied with Kimi K2.5 Thinking, a one-point Elo gap with overlapping confidence intervals. And while Kimi K2.5 activates 32 billion of a trillion total parameters, the 31B is dense, so all 31 billion are active. The two activate a comparable number of parameters. The 31B's advantage is not raw power; it is the efficiency of a dense model that does not need to route through a mixture of experts.

For local creators, the practical takeaway is clear. If you have a GPU with 8-10GB of VRAM, the 12B is the model to use, and it gives you both vision and audio. If you have 20GB or more and only need text or vision, the 31B is a strong choice, but you lose audio entirely. The 26B-A4B sits in between, but it also lacks audio. There is no single model that does everything at every size; the 12B is the only one that does video and voice together.

One more note for those who have seen conflicting information online: several widely-circulated pages describe Gemma 4 as a 1B/4B/12B/27B family released in April 2025. That describes Gemma 3, and the wrong year. Gemma 4 is dated 19 June 2026, ships in the five sizes listed above, and is released under the Apache 2.0 licence — a change from Gemma versions 1 through 3, which used Google's own custom licence terms. That permissive licence is another reason to run it locally without legal headaches.

Full analysis, with every figure mapped to the table it came from: The Gemma 4 File.

undefined: The Fable Brief — .