谷歌的DiffusionGemma证明构建文本扩散模型无需从头训练

Instead of training a new model from scratch, Google DeepMind retrofitted Gemma 4 into a diffusion model. The newly published report explains how it works and where the tradeoffs are. Google DeepMind released DiffusionGemma as a model in mid-June and has now followed up with the technical report. Unlike standard language models that generate text one token at a time, DiffusionGemma refines blocks of 256 tokens in parallel, similar to how image AIs pull a picture out of noise. On an Nvidia H100 accelerator, the model hits about 1,500 tokens per second. Building a new model from scratch wasn't necessary. The team started with the existing Gemma-4-26B-A4B and converted it into a diffusion model using less than ten percent of the original training token budget, according to the report . In the first of two steps, the model learns to reconstruct noisy text blocks from example data. A combined phase of reinforcement learning and sampler distillation follows, which Google calls SD·RL. Reinforcement learning typically boosts answer quality, while sampler distillation lets the model get by with fewer compute steps. Google merges both into a single process. According to the report, this combined approach raises quality on reasoning benchmarks by an average of ten points while nearly quadrupling the number of tokens per compute step. As a side effect, DiffusionGemma's answers run about 50 percent shorter, which further boosts speed. Standard language models have to commit to the first digit of an answer before they've worked through the reasoning. In a math problem from the report, Gemma 4 starts its response with "-1," realizes during its derivation that "-25" is correct, and tacks on a correction afterward. DiffusionGemma develops the answer and reasoning in parallel, so it can fix mistakes before the output is finalized. Sudoku solving works on the same principle, since every entry depends on entries that come later. After minimal fine-tuning, DiffusionGemma solves close to 85 percent of puzzles correctly, while the base model fails at the task entirely. Structured outputs like JSON or code repairs finish after just two to three refinement steps, according to the report, because the input already determines most tokens. DiffusionGemma also keeps its original ability to generate text word by word, letting users switch between both modes depending on the task. Absolute performance falls short of the autoregressive base model. Google points to several reasons for this. DiffusionGemma wasn't trained as a diffusion model from the start but was retrofitted after the fact. The subsequent training phase was relatively short, and the second step, SD·RL, prioritized speed over peak quality. The architecture, training data, and other settings were also carried over from the original Gemma 4 model, which aren't necessarily ideal for diffusion. The model occasionally gets stuck in repetition loops, producing individual words multiple times in a row. This is an artifact of the aggressively reduced compute steps. On multimodal tasks, DiffusionGemma sometimes forgets to close its reasoning section properly, which artificially drags down benchmark scores. The speed advantage also holds mainly for single-user scenarios. Once about 32 concurrent requests hit the model, standard language models catch up on throughput. Google explicitly calls DiffusionGemma an experimental model and says the release is meant to speed up research on text diffusion while giving the community a foundation for specialized, resource-efficient adaptations.