Google's DeepMind has released DiffusionGemma, a new open-source language model designed to run faster on local hardware like GPUs. Unlike other models that generate text one token at a time, DiffusionGemma uses a parallel approach, denoising text canvas multiple times to create outputs.
This results in significantly faster generation speeds. Tests show it can produce up to 700 tokens per second on an RTX 5090 GPU and up to 1,000+ tokens per second with a single Nvidia H100 AI accelerator.
DiffusionGemma boasts around 4x the output speed compared to similar-sized autoregressive models. This significant improvement makes it a valuable tool for developers interested in faster text generation locally.
Comments