Hy-SimulMT: Tencent's Open Real-Time Speech Translator

Simultaneous interpretation is one of the hardest jobs in language work: you have to start speaking before the speaker has finished the thought. Tencent Hunyuan has just released an open model built for exactly that task. Hy-SimulMT, announced on October 26, 2026, translates Chinese and English speech in both directions as the audio streams in, producing translated text and speech, and it can clone the original speaker's voice in real time. It is a notable step for open real-time speech translation, and the weights are downloadable today.

• What it is: a simultaneous translation model for bidirectional Chinese-English speech-to-speech and speech-to-text, with real-time voice cloning
• Size: six 30B-A3B mixture-of-experts checkpoints (about 30.4 billion parameters), in BF16, FP8 and GGUF formats
• Speed: a StreamLAAL lag of roughly 3 seconds on Tencent's RealSI long-form evaluation, the lowest of the systems in its comparison
• Tooling: ships with SimulInfer (web UI, WebSocket API, one-command vLLM deployment) and SimulTrain for further training

How Does Hy-SimulMT Translate While People Are Still Talking?

Traditional speech translation waits for a full sentence, then translates. That makes listeners wait, which is a real problem in a meeting or a live interview. Hy-SimulMT instead translates a continuous audio stream incrementally. According to Tencent's project page, it starts speaking early and corrects itself afterward when later words change the meaning, much like a human interpreter's self-corrections.

Tencent measures that trade-off with StreamLAAL, a metric for lag across the translation stream (lower is better), paired with GEMBA, an LLM-judged quality score. On long-form RealSI audio, Hy-SimulMT came in at about 3 seconds of StreamLAAL, and Tencent reports its GEMBA score beat the other open-source models it compared and four of the five commercial APIs. The team is careful to note that StreamLAAL is not the same as time to first spoken output.

How Does It Score on Speech Translation Benchmarks?

On four public short-form benchmarks, Tencent's model card lists these Hy-SimulMT scores:

• FLEURS: 90.3
• CoVoST2: 75.5
• ACL6060: 81.2
• RealSI-Sentence: 84.7, the highest of every system in Tencent's table

Results are mixed in an honest way: other systems lead on CoVoST2 and FLEURS, and Tencent says the benchmarks helped it find areas for improvement. In a small study with 18 newly recorded six-minute speeches, interviews and meetings across ten domains, Tencent compared the model with six certified professional interpreters and reports quality on par with the best of them, often at lower latency. Tencent stresses that this applies only to its test samples, and that professional interpreters remain essential where accountability or high-stakes decisions are involved.

What Do You Need to Run Hy-SimulMT Locally?

This is a server-class open-weight model, not a phone app. The BF16 checkpoint is roughly 61 GB, and the SimulInfer guide calls for a Linux machine with Python 3.12 or newer, an NVIDIA GPU with enough memory for the chosen checkpoint, and vLLM 0.31 or later. The FP8 and GGUF versions cut that footprint, and a llama.cpp bridge is included, though Tencent says official llama.cpp support has not been merged yet. Spoken output and voice cloning also need a separately configured TTS service. For everyone else, Tencent offers the model in its Hy Translate app.

The Hugging Face listing shows an Apache 2.0 license tag, while the GitHub repository ships its own license file, so builders should read those terms before deploying.

Why Open Simultaneous Translation Matters

Real-time interpretation has mostly lived behind commercial APIs. An open model with downloadable weights and a training toolkit lets universities, conference organizers and accessibility projects experiment, fine-tune and self-host. It extends Tencent's recent run of open releases such as Hy4's 770B open weights, and pairs naturally with smaller on-device projects like the offline Gemma translator on a Raspberry Pi 5. For more model releases, follow our AI news coverage.

Sources: Tencent Hunyuan Hy-SimulMT on GitHub — October 26, 2026; Hy-SimulMT-30B-A3B model card on Hugging Face — October 2026; Samon AI Brief — October 26, 2026.