Gemma Translator: Offline AI Translation on a Pi 5

A Complete Interpreter That Never Touches the Cloud

Three Google engineers have published Gemma Translator, an open-source handheld interpreter that runs its entire speech-to-speech pipeline locally on a Raspberry Pi 5. Speech recognition, translation, and speech synthesis all happen on the board. After initial setup there is no network call, no API key, and no audio leaving the device — which is exactly the property that makes this project more than a demo.

• Runs on a Raspberry Pi 5 with 8 GB RAM, with a 480x320 touchscreen, microphone, speaker, push-to-talk buttons, rotary encoder, and a 3D-printed enclosure
• Gemma 4 E2B handles translation at roughly 2.3B effective parameters (about 5.1B total with per-layer embeddings)
• Measured throughput on the Pi 5 is 99 tokens/sec prefill and 9 tokens/sec decode, with a peak memory footprint of 1,432 MB
• Moonshine performs speech recognition and Kokoro generates speech, with Gemma 4 E2B running through the LiteRT-LM runtime

What Makes the Performance Numbers Interesting

Nine tokens per second of decode sounds modest until you convert it to what a listener experiences: roughly 27.3 characters per second, or about 300 words per minute. Conversational speech runs somewhere around 130 to 150 words per minute, so the model generates translated text meaningfully faster than a person speaks it. The bottleneck in a live interpretation loop is therefore not the language model — it is the turn-taking.

The memory figure is the other number worth sitting with. A peak footprint of 1,432 MB means the whole translation stage fits comfortably inside a fraction of an 8 GB Pi 5, leaving headroom for the audio pipeline, the display, and the operating system. This is a very different situation from a year ago, when running any capable language model on a single-board computer meant aggressive quantization and constant swap pressure. Readers who followed our piece on an ESP32-S3 running a local LLM at 9 tokens per second will recognize how quickly the floor has moved.

Why Does LiteRT Matter for Single-Board Computers?

LiteRT is Google's runtime for on-device inference, and LiteRT-LM is the language-model-specific layer. The reason it shows up in a Raspberry Pi project rather than a phone project is that the same runtime targets ARM CPUs across form factors. Gemma 4 E2B was built for tight edge environments specifically — it uses memory-mapped per-layer embeddings so the full parameter count never has to sit resident in RAM at once, which is how a nominally 5.1B-parameter model fits in under 1.5 GB.

That combination — a model designed for edge memory constraints plus a runtime that maps cleanly onto ARM — is the reason this project reads as reproducible rather than heroic. The build is documented, the source is on GitHub, and the parts list is ordinary maker hardware.

Where an Offline Interpreter Actually Earns Its Keep

The privacy property is not incidental. In healthcare settings, a translation device that keeps patient conversations entirely on-device sidesteps a category of data-handling questions that cloud translation raises. In classrooms, language learners get real-time pronunciation and comprehension practice without a subscription or connectivity requirement. And for travel, an interpreter that works in a basement, on a plane, or in a country where your data plan does not, is simply more useful than one that does not.

The Broader Local AI Trend on Pi Hardware

Gemma Translator lands in a year when the Raspberry Pi 5 has become a credible local AI host rather than a curiosity. The pattern is consistent: smaller purpose-built models, runtimes that respect ARM memory limits, and projects that publish real throughput numbers instead of vague claims. Our full mini computer coverage tracks where that line keeps moving.

Should You Build One?

If you already own a Pi 5 with 8 GB of RAM, the additional bill of materials is a screen, an audio in/out pair, a couple of controls, and a printed shell. The project is open source and the throughput figures come from Google's own developer writeup, so expectations are set honestly before you order parts. For anyone who has wanted a genuinely private translation device, this is the most complete published reference build available right now.

Sources: CNX Software — August 14, 2026; Google Developers Blog — August 2026; gemma-translator on GitHub — accessed August 14, 2026.