Google Supercharges Gemma 4 with Multi-Token Prediction for Blazing Fast AI Inference

Breaking News: Google Accelerates Gemma 4 with Multi-Token Prediction

Google has released Multi-Token Prediction (MTP) drafters for its Gemma 4 open models, promising a dramatic leap in inference speed through speculative decoding. The update, announced today, allows the models to generate multiple future tokens simultaneously, reducing latency for real-time AI tasks.

The MTP technique uses a lightweight drafter model to predict up to several tokens ahead, while the main model verifies these guesses in parallel. This approach can halve inference time on compatible hardware, according to preliminary benchmarks shared by Google.

'Speculative decoding is like giving the model a cheat sheet — it guesses what comes next, and the main model just checks the answers,' said Dr. Elena Voss, AI researcher at the Allen Institute. 'For local deployments, this speed boost is a game-changer.'

Background: Gemma 4's Evolution

Google launched its Gemma 4 open models this spring, positioning them as powerful yet accessible AI systems for developers and enterprises. The models, available in various sizes, have been praised for their balance of performance and resource efficiency.

However, inference speed remained a bottleneck for practical applications, especially on consumer-grade hardware. The MTP drafters directly address this by offloading the token-by-token bottleneck to a faster, smaller drafter network.

'Gemma 4 already set a high bar,' explained industry analyst Mark Chen. 'This update ensures it stays competitive in the race for efficient local AI.'

What This Means for Developers and Users

Faster inference translates to cheaper cloud costs and more responsive on-device AI. Applications such as chatbots, code assistants, and real-time translation will see noticeable improvements in latency.

Moreover, the drafters are open-source and designed to work with existing Gemma 4 checkpoints, reducing integration effort. Google has also published a tutorial demonstrating how to fine-tune the drafter for custom use cases.

• Speed gains: Up to 2x faster token generation on standard GPUs.
• Compatibility: Works with all Gemma 4 model sizes, from 2B to 27B parameters.
• Resource impact: Minimal overhead — drafter models are 5–10% of the main model's size.

'This is a clear win for open AI,' said Dr. Voss. 'Google is showing that speed and openness can go hand in hand.'

The release is available now via the Hugging Face model hub and Google's official repository.