--- title: "magpie-tts-flow" publisher: "nvidia" type: "endpoint" updated: "2025-07-10T02:02:24.864Z" description: "Expressive and engaging text-to-speech, generated from a short audio sample." canonical: "https://build.nvidia.com/nvidia/magpie-tts-flow" --- # Speech Synthesis: Magpie TTS Flow Model Overview ## Description: With only a prompt of 5 seconds or less, the Magpie TTS Flow model can analyze a speaker’s voice and replicate voice qualities such as pitch, timbre and speech rate to achieve a speaker similarity of over 70%, and an MOS score of 4.40. Maintaining the original characteristics that capture unique voice audio signature, it can create high-quality audio (speech) when used in combination with a vocoder model like BigVGAN [1]. Magpie TTS Flow [2] is an alignment-aware pre-training method that builds upon E2TTS’s [3] training framework to learn alignment between unit sequences and speech frames. By using de-duplicated units that retain only phonetic content, Magpie TTS Flow effectively learns alignment without relying on a phoneme duration predictor. This allows for direct application to zero-shot voice conversion, where phonetic content can be transferred to the target speaker’s voice without additional fine-tuning. This model is packaged with BigVGAN, a universal vocoder that generalizes well for various out-of-distribution scenarios without fine-tuning. This model is ready for commercial use. **You are responsible for ensuring that your use of NVIDIA AI Foundation Models complies with all applicable laws.** ## License/Terms of Use: [NVIDIA AI Foundation Models Community License Agreement](https://docs.nvidia.com/ai-foundation-models-community-license.pdf) ## References: [1] [BigVGAN: A Universal Neural Vocoder with Large-Scale Training](https://arxiv.org/abs/2206.04658)
[2] [Magpie-TTS-Flow Paper](https://openreview.net/forum?id=e2p1BWR3vq)
[3] [E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS](https://arxiv.org/abs/2406.18009)
[4] [Flow Matching for Generative Modeling](https://arxiv.org/abs/2210.02747)
## Model Architecture: Architecture Type: Flow Matching
Network Architecture: Optimal Transport Conditional Flow Matching (OT-CFM)-based Masked Speech Modeling Flow Matching [4] (FM) is a simulation-free approach for training Continuous Normalizing Flows (CNFs) based on regressing vector fields of fixed conditional probability paths. It is compatible with a general family of Gaussian probability paths for transforming between noise and data samples — which subsumes existing diffusion paths as specific instances. Furthermore, Flow Matching opens the door to training CNFs with other, non-diffusion probability paths. An instance of particular interest is using Optimal Transport (OT) displacement interpolation to define the conditional probability paths. These paths are more efficient than diffusion paths, provide faster training and sampling, and result in better generalization. ## Input: *Input Type:* Text + Audio
*Input Format:*
For Text: Strings (Graphemes in US English)
For Audio: wav file
*Input Parameters:*
For text: One-Dimensional (1D)
For audio prompt: Two-Dimensional (batch x time)
*Other Properties related to Input:*
For Audio: Recommended format for prompt: Mono, PCM-encoded 16 bit audio; sampling rate of 22.05 kHz; between 3 and 5 second duration.
## Output: *Output Type:* Audio
*Output Format:* Audio of shape (batch x time) in wav format
*Output Parameters:* Two-Dimensional (batch x time)
*Other Properties related to Output:* Mono, PCM-encoded 16 bit audio; sampling rate of 22.05 kHz; 20 Second Maximum Length.
**Supported Operating System(s):**
* Linux
## Model Version(s): Magpie-TTS-Flow_v1
## Inference: **Engine:** Triton
**Test Hardware:**
* NVIDIA A100 GPU
* NVIDIA A30 GPU
* NVIDIA A10 GPU
* NVIDIA H100 GPU
* NVIDIA L4 GPU
* NVIDIA L40 GPU
## Ethical Considerations: NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their supporting model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. For more detailed information on ethical considerations for this model, please see the Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards. Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/). ## Bias Field | Response :------|:---------- What is the language balance of the model validation data? | US English: 100% What is the geographic origin language balance of the model validation data? | United States: 100% What is the accent balance of the model validation data? | US English: 100% Measures taken to mitigate against unwanted bias: | This model was trained on a variety of languages and accents, including: US English, French, German, Italian, European Spanish, Dutch, Polish, and Brazilian Portuguese. Participation considerations from adversely impacted groups [protected classes](https://www.senate.ca.gov/content/protected-classes) in model design and testing: | Evaluators of several genders, age groups, language backgrounds and geographic locations were recruited when assessing the quality of this model. ## Explainability Field | Response :------|:---------- Intended Application & Domain: | Speech Synthesis Model Task | Speech Synthesis and Voice Characterization Intended Users | This model is intended for developers building interactive call centers, virtual assistants, and language learning assistants to improve pronunciation, automatically generate voice-overs, narrate or comment on videos, and provide audio alternatives for visually impaired users or people with light sensitivity. Model Output | Audio of shape (batch x time) in wav format Describe how the model works | Model takes input text and outputs an audio representation of the text. It can be used as a zero-shot voice characterization model. When given a reference audio sample to replicate along with an input text, the produced synthetic audio will be similar to this reference. Name the adversely impacted groups this has been tested to deliver comparable outcomes regardless of | Gender, Age (including people in older age brackets) Technical Limitations | Model only has the capacity to produce a voice in the languages, dialects and gender(s) in which it is trained. This model makes no effort to moderate or modify input text. Languages that are underrepresented may not sound as natural. Verified to have met prescribed NVIDIA quality standards | Yes Performance Metrics | % preference when compared with available alternatives
word error rate (wer)
character error rate (cer)
mean opinion score (MOS) Potential Known Risks | This model has the ability to replicate the characteristics of an individual's voice but may unnaturally synthesize vocabulary not included in the pronunciation dictionary or omit phonetic symbols not used in training. Licensing: | [https://docs.nvidia.com/ai-foundation-models-community-license.pdf](https://docs.nvidia.com/ai-foundation-models-community-license.pdf) ## Privacy Field | Response :------|:--------- Generatable or reverse engineerable personal information? | None Personal data used to create this model? | Yes - Voice Was consent obtained for any personal data used? | Yes How often is the dataset reviewed? | Before release (during training) Is a mechanism in place to honor data subject right of access or deletion of personal data? | Yes If personal data was collected for the development of the model, was it collected directly by NVIDIA? | Yes If personal data was collected for the development of the model by NVIDIA, do you maintain or have access to disclosures made to data subjects? | Yes If personal data was collected for the development of this AI model, was it minimized to only what was required? | Yes Is there provenance for all datasets used in training? | Yes Does data labeling (annotation, metadata) comply with privacy laws? | Yes Is data compliant with data subject requests for data correction or removal, if such a request was made? | Yes ## Safety & Security Field | Response :------|:--------- Model Application(s): | Speech synthesis and Voice Characterization Describe the life critical impact (if present). | Not Applicable Model and dataset restrictions: | The Principle of least privilege (PoLP) is applied limiting access for dataset generation and model development. Restrictions enforce dataset access during training, and dataset license constraints adhered to. Use case restrictions for the model. | Abide by [https://docs.nvidia.com/ai-foundation-models-community-license.pdf](https://docs.nvidia.com/ai-foundation-models-community-license.pdf)