Why text diffusion?
Although the AI research community has been exploring diffusion-based text generation for years, applying it to huge models remains a challenge. DiffusionGemma changes this by changing the way models operate hardware.
A compromise with classic models
Most language models work like a typewriter, generating one token at a time from left to right. In the cloud, this is proficient because servers can pool thousands of user requests to share the hardware load. However, when run locally for a single user, this word-by-word process does not make full operate of the dedicated GPU or TPU – most of the time is spent simply waiting for the next “keystroke”.
DiffusionGemma reverses this inefficiency. Instead of predicting words sequentially, it simultaneously drafts an entire paragraph of 256 characters. By giving your computer’s processor more of the work at once, DiffusionGemma harnesses its full potential. It upgrades model inference from a single, sequential typewriter to a massive printing press that stamps an entire block of text at once.