Natural and expressive voices in multiple languages. For voice agents and brand ambassadors.
Overview
The model is an end-to-end multilingual neural text-to-speech model that generates speech in twelve different languages (English-US, Spanish-US, German-German, French-France, Italian, Vietnamese, Mandarin-Chinese, Hindi, Japanese, Modern Standard Arabic, Brazilian Portuguese, and Korean) by predicting discrete audio codec tokens autoregressively using a transformer encoder-decoder architecture. It supports at least one male and one female speaker for all the languages. It employs multi-codebook prediction (typically 8 codebooks) with local transformer refinement and frame stacking (factor = 2) for high-quality audio generation, and leverages techniques like attention priors, classifier-free guidance (CFG), and Group Relative Policy Optimization (GRPO) for improved alignment and robustness. This release also includes a zero-shot inference fix, IPA grapheme-to-phoneme (G2P) support for custom dictionaries and code-switching, and updated G2P support for English-to-Katakana code-switching. The model supports both batch inference for complete utterances and long-form inference for very long text inputs with sliding window mechanisms. The generated codecs are then converted to a speech waveform using a frozen pretrained audio codec model.
This model is ready for commercial use.
GOVERNING TERMS: The NIM container is governed by the NVIDIA Software License Agreement and the Product-Specific Terms for NVIDIA AI Products; and the use of this model is governed by the NVIDIA Open Model License Agreement.
Global
For streaming voice agent use-cases, For Offline speech generation from text, In multiple languages.
Build.Nvidia.com [06/22/2026] via [https://build.nvidia.com/nvidia/magpie-tts-multilingual]
NGC [06/22/2026] via [https://registry.ngc.nvidia.com/orgs/nim/teams/nvidia/containers/magpie-tts-multilingual]
TTS model papers:
Audio codec paper:
Architecture Type: Transformer Encoder, Transformer Decoder, Local Transformer, and feedforward layers
Network Architecture:
Causal Transformer Encoder with 6 layers, learnable positional encoder of length 2048, and 1 Layer Normalization output layer.
Causal Transformer Decoder with 12 layers, learnable positional encoder of length 2048, and 1 Layer Normalization output layer.
Local Transformer for multi-codebook refinement, operating on stacked frames with a frame stacking factor of 2 to improve audio quality and reduce sequence length.
** Number of model parameters: 3.64 × 10^8 (364 M parameters)
Inference Configuration:
decoder-ce-context with GRPO-aligned weightscfg_scale = 2.5temperature = 0.6, top_k = 80apply_attention_prior = True, prior_epsilon = 0.1, estimate_alignment_from_layers = [4, 5, 8, 9], apply_prior_to_layers = [2, 3, 4, 5, 6, 7, 8, 9, 10], start_prior_after_n_audio_steps = 0 Input Type(s): Text, Audio (optional reference utterance for zero-shot voice cloning)
Input Format(s):
Input Parameters:
Other Properties Related to Input: Text input is UTF-8 encoded; text normalization is required. Audio reference (when provided for voice cloning) is mono, PCM-encoded 16-bit WAV at a sampling rate of 22.05 kHz; recommended reference duration is between 3 and 10 seconds.
Output Type(s): Audio
Output Format(s):
Output Parameters:
Other Properties Related to Output: Mono, PCM-encoded 16 bit audio; sampling rate of 22.05 kHz; Audio output with dimensions (B x T), where B is batch size and T is time dimension.
Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.
Runtime Engine(s):
Supported Hardware Microarchitecture Compatibility:
Preferred/Supported Operating System(s):
The integration of foundation and fine-tuned models into AI systems requires additional testing using use-case-specific data to ensure safe and effective deployment. Following the V-model methodology, iterative testing and validation at both unit and system levels are essential to mitigate risks, meet technical and functional requirements, and ensure compliance with safety and ethical standards before deployment.
RivaTTS_MagpieTTS_Multilingual v4.0
The following datasets were used to train the model, including additional datasets focused on speech and ASR.
English (En):
German (De):
Spanish (Es):
French (Fr):
Italian (It):
Hindi (Hi):
Japanese (Ja):
Mandarin Chinese (Zh):
Vietnamese (Vi):
Brazilian Portuguese (pt-BR) — NEW:
Arabic — Modern Standard Arabic (ar_MSA) — NEW; also includes Saudi Arabia (ar_SA), UAE (ar_AE), and Damascene (ar_SY) accents:
Korean (Ko) — NEW:
Data Modality
Audio Training Data Size
Data Collection Method by dataset
Labeling Method by dataset
Properties:
Number of data items in training set: ~54.3k hours
Modality: Audio (speech signal)
Nature of the content: Audio books, conversational speech, and read speech
Language: Multilingual (En, Es, De, Fr, Vi, It, Zh, Hi, Ja, Ar, pt-BR, Ko)
Sensor Type: Microphones
Data Collection Method by dataset:
Labeling Method by dataset:
Properties:
Modality: Audio (speech signal)
Nature of the content: Audio books, conversational speech, and newspaper passages
Language: Multilingual (En, Es, De, Fr, Vi, It, Zh, Hi, Ja, Ar, pt-BR, Ko)
Sensor Type: Microphones
| CER (%) | SV-SSIM (%) | |
|---|---|---|
| English (LibriTTS) | 0.34 | 82.79 |
| English Accents | 0.33 | 76.31 |
| Spanish CML | 0.51 | 78.18 |
| German CML | 0.86 | 75.05 |
| French CML | 1.07 | 74.30 |
| Italian | 1.78 | 78.50 |
| Vietnamese | 0.41 | 75.34 |
| Mandarin | 2.49 | 83.13 |
| Hindi | 1.23 | 78.71 |
| Japanese | 1.17 | 77.57 |
| Arabic (MSA) | 1.63 | 80.32 |
| Korean | 2.57 | 81.23 |
| Brazilian Portuguese | 3.46 | 76.79 |
Lower CER is better; higher SV-SSIM is better. MagpieTTS 4.0 (May 2026) shows CER and SV-SSIM improvements over MagpieTTS 3.1 across all previously supported languages, with the largest CER gains on Mandarin, French, and Italian. The English hard-set evaluation shows a substantial improvement on short prompts (CER reduced from 4.0% to 1.2%) while remaining comparable on Money, Digits, and Letters categories.
Acceleration Engine: Triton
Test Hardware:
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their supporting model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
For more detailed information on ethical considerations for this model, please see the Model Card++ Bias, Explainability, Safety & Security, and Privacy Subcards.
Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns here.