DiffusionGemma: DeepMind Launches Open-Source Model That Generates Text 4x Faster
Google DeepMind has just released DiffusionGemma, an experimental open-source model that promises to revolutionize text generation speed. Unlike traditional LLMs that process words one at a time (autoregressively), DiffusionGemma generates entire blocks of text simultaneously, delivering up to 4x faster generation on dedicated GPUs.
How it works
While most language models act like a digital "typewriter" — generating one token at a time from left to right — DiffusionGemma works more like a printing press: it drafts an entire 256-token block at once and iteratively refines it.
DiffusionGemma reverses the inefficiency of traditional models. Instead of predicting words sequentially, it drafts an entire block of 256 tokens simultaneously — explain researchers Brendan O'Donoghue and Sebastian Flennerhag.
The model uses bi-directional attention, allowing every token to attend to all others during generation — a significant advantage for tasks like inline editing, code infilling, and non-linear text structures.
Technical specifications
- 26B parameters (Mixture of Experts), activating only 3.8B per inference
- Fits within 18GB VRAM when quantized (high-end consumer GPUs)
- 1000+ tokens/second on a single NVIDIA H100
- 700+ tokens/second on an NVIDIA GeForce RTX 5090
- Licensed under Apache 2.0
Experimental status
DiffusionGemma is an experimental model. Its overall output quality is lower than standard Gemma 4. The model is ideal for scenarios prioritizing speed over maximum quality — such as real-time editing tools, rapid prototyping, and interactive local workflows.
The team has demonstrated successful fine-tuning for specific tasks, like solving Sudoku — something traditional autoregressive models struggle with due to future token dependencies.
Availability
Model weights are available on Hugging Face, with support for MLX, vLLM, and Hugging Face Transformers. Google also released a fine-tuning tutorial using Hackable Diffusion, a modular JAX toolbox.
Implications: DiffusionGemma paves the way for a new generation of AI applications requiring real-time response on local hardware — a persistent Achilles' heel for large autoregressive models.