4c:I[510592,["/_next/static/chunks/27gl9k2703ugd.js","/_next/static/chunks/3pd35vh8xvq70.js","/_next/static/chunks/268l5ccisytc0.js","/_next/static/chunks/2b4d1e9ztz65u.js","/_next/static/chunks/24mqrfe34h1_g.js","/_next/static/chunks/1la1pzs9j8idc.js","/_next/static/chunks/3-axx3uk1c0-h.js","/_next/static/chunks/14bn3mp7hfp6x.js","/_next/static/chunks/1_xj2c5utin8v.js","/_next/static/chunks/2tdv_pqf9hlyu.js","/_next/static/chunks/44n8gpf9x1nn2.js","/_next/static/chunks/1e7h6z92_w2-c.js","/_next/static/chunks/2g-r6rokjl78a.js","/_next/static/chunks/1geiruffics9x.js","/_next/static/chunks/1lfygqrocy8iw.js","/_next/static/chunks/0q47y720ck5-9.js","/_next/static/chunks/03uy_tk5mi4gg.js","/_next/static/chunks/3mi_myijg06x1.js","/_next/static/chunks/2_c926k09c1th.js","/_next/static/chunks/0bvim9_g0hu68.js","/_next/static/chunks/3xav1ddquf5gq.js","/_next/static/chunks/0ayh-bmdnfery.js","/_next/static/chunks/34kl4tv6yifcg.js","/_next/static/chunks/0nm6za7z70maq.js","/_next/static/chunks/44e22u2t839t0.js","/_next/static/chunks/0czpc-f8wjri8.js","/_next/static/chunks/2aq42x9vabhfb.js","/_next/static/chunks/24_gg9g_4g22i.js","/_next/static/chunks/00yjb-caoi49j.js","/_next/static/chunks/1l0z2zsen9vs2.js","/_next/static/chunks/0irw7vnh9fzo0.js","/_next/static/chunks/01auidv8wbc03.js","/_next/static/chunks/434amnen8yqdi.js","/_next/static/chunks/1yd7p-515l2jc.js","/_next/static/chunks/2pqhypz2vo8mu.js","/_next/static/chunks/1i60vi-4s7c4l.js","/_next/static/chunks/2-18150gleij4.js","/_next/static/chunks/295ffwyuqa4b6.js"],"PlaygroundContextProvider"] 4d:T3188,# Sarvam-m Overview ### Description Sarvam-m generates human-like text for a seamless chatting experience, providing a smooth and accessible multilingual conversation experience, and is intended for general-purpose conversation and text generation tasks. This model is ready for commercial/non-commercial use. ### Third-Party Community Consideration This model is not owned or developed by NVIDIA. This model has been developed and built to a third-party’s requirements for this application and use case; see link to Non-NVIDIA [sarvamai/sarvamai-m](https://huggingface.co/sarvamai/sarvam-m). ### License/Terms of Use **GOVERNING TERMS:** This trial service is governed by the [NVIDIA API Trial Terms of Service](https://assets.ngc.nvidia.com/products/api-catalog/legal/NVIDIA%20API%20Trial%20Terms%20of%20Service.pdf). The model is governed by the [NVIDIA Community Model License Agreement](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-community-models-license/); ADDITIONAL INFORMATION: [Apache License Version 2.0](https://huggingface.co/datasets/choosealicense/licenses/blob/main/markdown/apache-2.0.md). You are responsible for ensuring that your use of NVIDIA AI Foundation Models complies with all applicable laws. ### Deployment Geography Global ### Use Case This model is intended for users seeking a seamless multilingual chatting experience, particularly those interested in Indian languages and cultures. It can be used by developers, researchers, and individuals looking to leverage its advanced reasoning capabilities for coding, math, and general conversation purposes. ### Release Date **Hugging Face:** 05/23/2025 via [sarvamai/sarvam-m](https://huggingface.co/sarvamai/sarvam-m).
**Build.NVIDIA.com:** 07/25/2025 via [link](https://build.nvidia.com/sarvamai/sarvam-m) ## Model Architecture **Architecture Type:** Hybrid-reasoning Transformer **Network Architecture:** Mistral-Small This model was developed based on Mistral-Small-3.1-24B-Base-2503 [sarvamai/sarvamai-m](https://www.sarvam.ai/blogs/sarvam-m). Number of model parameters: 23.6B ## Input **Input Type(s):** Text
**Input Format(s):** Strings
**Input Parameters:** One-Dimensional (1D)
**Input Range:** [0, 1] (float32) or [0, 255] (uint8, auto-converted)
**Other Properties Related to Input:** Supports up to 8,192-token context length (example shown in Quickstart using max_new_tokens=8192)
## Output **Output Type(s)**:Text
**Output Format(s)**: Strings
**Output parameters:** One-Dimensional (1D)
**Other Properties Related to Output:** Context Length: 32,768 tokens and sliding window attention of 4096 tokens. Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA’s hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions. ## Software Integration Runtime Engine: vLLM ### Supported Hardware Microarchitecture Compatibility * NVIDIA Ampere * NVIDIA Blackwell * NVIDIA Hopper * NVIDIA Lovelace ### Preferred Operating System(s) Linux ### Model Version(s) sarvam-m-v1.0 ### Quickstart The following code snippet demonstrates how to use sarvam-m using Transformers. ```bash from transformers import AutoModelForCausalLM, AutoTokenizer model_name = "sarvamai/sarvam-m" # load the tokenizer and the model tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype="auto", device_map="auto" ) # prepare the model input prompt = "Who are you and what is your purpose on this planet?" messages = [{"role": "user", "content": prompt}] text = tokenizer.apply_chat_template( messages, tokenize=False, enable_thinking=True, # Switches between thinking and non-thinking modes. Default is True. ) model_inputs = tokenizer([text], return_tensors="pt").to(model.device) # conduct text completion generated_ids = model.generate(**model_inputs, max_new_tokens=8192) output_ids = generated_ids[0][len(model_inputs.input_ids[0]) :].tolist() output_text = tokenizer.decode(output_ids) if "" in output_text: reasoning_content = output_text.split("")[0].rstrip("\n") content = output_text.split("")[-1].lstrip("\n").rstrip("") else: reasoning_content = "" content = output_text.rstrip("") print("reasoning content:", reasoning_content) print("content:", content) ``` For thinking mode, we recommend temperature=0.5; for no-think mode, temperature=0.2. ### With Sarvam APIs ```bash from openai import OpenAI base_url = "https://api.sarvam.ai/v1" model_name = "sarvam-m" api_key = "Your-API-Key" # get it from https://dashboard.sarvam.ai/ client = OpenAI( base_url=base_url, api_key=api_key, ).with_options(max_retries=1) messages = [ {"role": "system", "content": "You're a helpful AI assistant"}, {"role": "user", "content": "Explain quantum computing in simple terms"}, ] response1 = client.chat.completions.create( model=model_name, messages=messages, reasoning_effort="medium", # Enable thinking mode. `None` for disable. max_completion_tokens=4096, ) print("First response:", response1.choices[0].message.content) # Building messages for the second turn (using previous response as context) messages.extend( [ { "role": "assistant", "content": response1.choices[0].message.content, }, {"role": "user", "content": "Can you give an analogy for superposition?"}, ] ) response2 = client.chat.completions.create( model=model_name, messages=messages, reasoning_effort="medium", max_completion_tokens=8192, ) print("Follow-up response:", response2.choices[0].message.content) ``` Refer to API docs here: [sarvam Chat Completions API docs](https://docs.sarvam.ai/api-reference-docs/chat/chat-completions) **reasoning_effort** can take three possible values: low, medium, and high to be consistent with the OpenAI API spec. Setting any of the three values just enables the thinking mode of sarvam-m. ### vLLM Deployment For easy deployment, we can use vllm>=0.8.5 and create an OpenAI-compatible API endpoint with vllm serve sarvamai/sarvam-m. If you want to use vLLM with python, you can do the following. ```bash from openai import OpenAI # Modify OpenAI's API key and API base to use vLLM's API server. openai_api_key = "EMPTY" openai_api_base = "http://localhost:8000/v1" client = OpenAI( api_key=openai_api_key, base_url=openai_api_base, ) models = client.models.list() model = models.data[0].id messages = [{"role": "user", "content": "Why is 42 the best number?"}] # By default, thinking mode is enabled. # If you want to disable thinking, add: # extra_body={"chat_template_kwargs": {"enable_thinking": False}} response = client.chat.completions.create(model=model, messages=messages) output_text = response.choices[0].message.content if "" in output_text: reasoning_content = output_text.split("")[0].rstrip("\n") content = output_text.split("")[-1].lstrip("\n") else: reasoning_content = "" content = output_text print("reasoning content:", reasoning_content) print("content:", content) # For the next round, add the model's response directly as assistant turn. messages.append( {"role": "assistant", "content": output_text} ) ``` ## Training, Testing, and Evaluation Datasets ### Training Dataset: * **Link:** Undisclosed * **Data Collection Method by dataset:** Hybrid: Human, Synthetic, Automated * **Labeling Method by dataset:** Hybrid: Human, Automated * **Properties:** The model was trained on a significant volume of data, with a substantial portion dedicated to Indian languages. Approximately one-third of the training samples were in 11 Indic languages (Hindi, Bengali, Gujarati, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil, and Telugu). Specifically, 30% of coding, math, and reasoning prompts, and 50% of the remaining prompts were translated into these languages. The datasets were designed to improve the model's capabilities in logical reasoning, mathematical problem-solving, and multilingual conversational tasks. No sensors were used for data collection. ### Testing Dataset * **Link:** Undisclosed * **Data Collection Method by dataset:** Hybrid: Human, Synthetic, Automated * **Labeling Method by dataset:** Human * **Properties:** The testing datasets comprise a wide range of tasks and quantities: - **IFEval:** Over 500 prompts to test instruction-following capabilities. - **GSM8K:** 8,500 high-quality, linguistically diverse grade-school math word problems. - **MATH:** 12,500 challenging math competition problems. - **Big Bench Hard (BBH):** A set of 23 difficult tasks designed to test reasoning. - **MMLU:** A comprehensive benchmark with multiple-choice questions covering 57 different subjects. - **HellaSwag:** A dataset of 70,000 multiple-choice questions for commonsense natural language inference. - **WinoGrande:** A collection of 44,000 problems for commonsense reasoning. - **ARC (AI2 Reasoning Challenge):** A set of 7,787 science questions from grade-school examinations. - **TruthfulQA:** A benchmark consisting of 817 questions across 38 categories to measure a model's truthfulness. - **Indic Language Benchmarks:** Various datasets were used to evaluate performance in 11 Indian languages, including translations of standard benchmarks and Indic-specific evaluations. ### Evaluation Dataset * **Link:** Undisclosed * **Data Collection Method by dataset:** Hybrid: Human, Synthetic, Automated * **Labeling Method by dataset:** Hybrid: Human, Automated * **Properties:** * **Data Collection Method by dataset:** Hybrid: Automated, Human, Synthetic * **Labeling Method by dataset:** Hybrid: Automated, Human, Synthetic * **Properties:** - **IFEval:** Contains over 500 prompts designed to test a model's ability to adhere to complex instructions - **GSM8K:** A dataset of 8,500 high-quality, linguistically diverse grade-school math word problems requiring multi-step reasoning. - **MATH:** A challenging dataset of 12,500 problems from mathematics competitions. - **Big Bench Hard (BBH):** A collection of 23 difficult tasks that are beyond the capabilities of most current language models. - **MMLU:** A massive multitask benchmark consisting of multiple-choice questions across 57 subjects, designed to test a model's general knowledge and problem-solving abilities. - **TruthfulQA:** Comprises 817 questions across 38 categories, designed to measure a model's tendency to produce truthful answers. - **Indic Language Benchmarks:** The model was also evaluated on its performance in 11 Indian languages, showing significant improvement over baseline models on various language tasks. #### Benchmark Results Results on multilingual benchmarks for 21 European languages with instruction-tuned models:
| Model | Avg. | EU21-ARC | EU21-HeSw | EU21-TQA | EU21-MMLU | |---------------------------------------|-------|----------|-----------|----------|-----------| | Meta-Llama-3.1-8B-Instruct | 0.563 | 0.563 | 0.579 | 0.532 | 0.576 | | Mistral-7B-Instruct-v0.3 | 0.527 | 0.530 | 0.538 | 0.548 | 0.491 | | Salamandra-7B-Instruct | 0.543 | 0.595 | 0.637 | 0.482 | 0.459 | | Aya-23-8B | 0.485 | 0.475 | 0.535 | 0.476 | 0.455 | | Occiglot-7B-eu5-Instruct | 0.475 | 0.484 | 0.519 | 0.471 | 0.428 | | Pharia-1-LLM-7B-C-A | 0.417 | 0.396 | 0.438 | 0.469 | 0.366 | | Bloomz-7B1 | 0.358 | 0.316 | 0.354 | 0.461 | 0.302 | | Teuken-7B-instruct-commercial-v0.4 | 0.531 | 0.569 | 0.620 | 0.503 | 0.430 | ### Inference Acceleration Engine: vLLM Test Hardware: * L40s x2 ## Ethical Considerations NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).4e:T4c5,from openai import OpenAI client = OpenAI( base_url = "https://integrate.api.nvidia.com/v1", api_key = "$NVIDIA_API_KEY" ) <% if (request.tools) { %> completion = client.chat.completions.create( model="<%- request.model %>", messages=<%- JSON.stringify(request.messages) %>, temperature=<%- request.temperature %>, top_p=<%- request.top_p %>, max_tokens=<%- request.max_tokens %>, stream=<%- request.stream?.toString()[0].toUpperCase() + request.stream?.toString().slice(1) %>, tools=<%- JSON.stringify(request.tools) %>, <% if (request.tool_choice) { %>tool_choice=<%- JSON.stringify(request.tool_choice) %><% } %> )<% } else { %> completion = client.chat.completions.create( model="<%- request.model %>", messages=<%- JSON.stringify(request.messages) %>, temperature=<%- request.temperature %>, top_p=<%- request.top_p %>, max_tokens=<%- request.max_tokens %>, stream=<%- request.stream?.toString()[0].toUpperCase() + request.stream?.toString().slice(1) %> )<% } %> <% if (request.stream) { %> for chunk in completion: if chunk.choices[0].delta.content is not None: print(chunk.choices[0].delta.content, end="") <% } else { %> print(completion.choices[0].message.content) <% } %> 4f:T504,import OpenAI from 'openai'; const openai = new OpenAI({ apiKey: '$NVIDIA_API_KEY', baseURL: 'https://integrate.api.nvidia.com/v1', }) <% if (request.tools) { %> async function main() { const completion = await openai.chat.completions.create({ model: "<%- request.model %>", messages: <%- JSON.stringify(request.messages) %>, temperature: <%- request.temperature %>, top_p: <%- request.top_p %>, max_tokens: <%- request.max_tokens %>, stream: <%- request.stream %>, <% if (request.tools) { %>tools: <%- JSON.stringify(request.tools) %>,<% } %> <% if (request.tool_choice) { %>tool_choice: <%- JSON.stringify(request.tool_choice) %>,<% } %> })<% } else { %> async function main() { const completion = await openai.chat.completions.create({ model: "<%- request.model %>", messages: <%- JSON.stringify(request.messages) %>, temperature: <%- request.temperature %>, top_p: <%- request.top_p %>, max_tokens: <%- request.max_tokens %>, stream: <%- request.stream %> })<% } %> <% if (request.stream) { %> for await (const chunk of completion) { process.stdout.write(chunk.choices[0]?.delta?.content || '') } <% } else { %> process.stdout.write(completion.choices[0]?.message?.content); <% } %> } main();50:T65b,<% if (request.tools) { %> "curl https://integrate.api.nvidia.com/v1/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -H \"Authorization: Bearer $NVIDIA_API_KEY\" \\\n -d '{\n \"model\": \"sarvamai/sarvam-m\",\n \"messages\": <%- JSON.stringify(request.messages).replaceAll(\"'\", \"'\\\"'\\\"'\") %>,\n \"temperature\": <%- request.temperature %>,\n \"top_p\": <%- request.top_p %>,\n \"max_tokens\": <%- request.max_tokens %>,\n \"stream\": <%- request.stream %> <% if (request.tools) { %>,\n \"tools\": <%- JSON.stringify(request.tools).replaceAll(\"'\", \"'\\\"'\\\"'\") %><% } %> <% if (request.tool_choice) { %>,\n \"tool_choice\": <%- JSON.stringify(request.tool_choice).replaceAll(\"'\", \"'\\\"'\\\"'\") %><% } %> }'\n"<% } else { %> "curl https://integrate.api.nvidia.com/v1/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -H \"Authorization: Bearer $NVIDIA_API_KEY\" \\\n -d '{\n \"model\": \"sarvamai/sarvam-m\",\n \"messages\": <%- JSON.stringify(request.messages).replaceAll(\"'\", \"'\\\"'\\\"'\") %>,\n \"temperature\": <%- request.temperature %>,\n \"top_p\": <%- request.top_p %>,\n \"max_tokens\": <%- request.max_tokens %>,\n \"stream\": <%- request.stream %> <% if (request.tools) { %>,\n \"tools\": <%- JSON.stringify(request.tools).replaceAll(\"'\", \"'\\\"'\\\"'\") %><% } %> <% if (request.tool_choice) { %>,\n \"tool_choice\": <%- JSON.stringify(request.tool_choice).replaceAll(\"'\", \"'\\\"'\\\"'\") %><% } %> }'\n"<% } %>4b:["$","$L4c",null,{"initialState":{"artifact":{"artifactType":"ENDPOINT","name":"sarvam-m","displayName":"sarvam-m","publisher":"sarvamai","shortDescription":"Multilingual, hybrid-reasoning model optimized for Indian language tasks, programming, mathematical reasoning capabilities.","logo":"https://assets.ngc.nvidia.com/products/api-catalog/images/sarvam-m.jpg","labels":["cloudPartnerType:endpoint:cloud_partner_type_together","coding","hybrid","indic languages","math","multilingual","nimType:endpoint:nim_type_preview","nimType:endpoint:nim_type_run_anywhere","nimType:endpoint:nim_type_upgrade_available","playgroundType:endpoint:playgroundtype_chat","reasoning"],"attributes":[{"key":"AVAILABLE","value":"true"},{"key":"DEPRECATION","value":"07/27/2026"},{"key":"PREVIEW","value":"true"},{"key":"last_month_api_invocation_count","value":"436455"},{"key":"last_month_download_count","value":"0"}],"updatedDate":"2025-07-25T17:29:05.725Z","canGuestDownload":true,"createdDate":"2025-07-25T17:29:05.725Z","description":"$4d","isPublic":true,"isReadOnly":true,"orgName":"qc69jvmznzxy"},"specAttributes":"$44:props:spec:attributes","modelPath":"/qc69jvmznzxy/sarvam-m","openApiSpec":{"openapi":"3.1.0","info":{"title":"NVIDIA NIM API for sarvamai/sarvam-m","description":"The NVIDIA NIM REST API. Please see https://docs.api.nvidia.com/nim/reference/sarvamai-sarvam-m for more details.","version":"1.0.0","termsOfService":"https://www.nvidia.com/en-us/data-center/products/nvidia-ai-enterprise/eula/","contact":{"name":"NVIDIA Enterprise Support","url":"https://www.nvidia.com/en-us/support/enterprise/"},"license":{"name":"Apache 2.0 License","url":"https://choosealicense.com/licenses/apache-2.0/"}},"servers":[{"url":"https://integrate.api.nvidia.com/v1/"}],"paths":{"/chat/completions":{"post":{"operationId":"create_chat_completion_v1_chat_completions_post","tags":["Chat"],"summary":"Creates a model response for the given chat conversation.","description":"Given a list of messages comprising a conversation, the model will return a response. Compatible with OpenAI. See https://platform.openai.com/docs/api-reference/chat/create","requestBody":{"content":{"application/json":{"schema":{"additionalProperties":false,"properties":{"model":{"type":"string","title":"Model","default":"sarvamai/sarvam-m"},"messages":{"description":"A list of messages comprising the conversation so far. The roles of the messages must be alternating between `user` and `assistant`. The last input message should have role `user`. A message with the the `system` role is optional, and must be the very first message if it is present; `context` is also optional, but must come before a user question.","examples":[[{"content":"I am going to Paris, what should I see?","role":"user"}]],"items":{"additionalProperties":false,"properties":{"role":{"description":"The role of the message author.","enum":["user","assistant"],"title":"Role","type":"string"},"content":{"description":"The contents of the message.","title":"Content","anyOf":[{"type":"string"},{"type":"null"}]}},"required":["role","content"],"title":"Message","type":"object"},"title":"Messages","type":"array"},"temperature":{"default":0.5,"description":"The sampling temperature to use for text generation. The higher the temperature value is, the less deterministic the output text will be. It is not recommended to modify both temperature and top_p in the same call.","maximum":1,"minimum":0,"title":"Temperature","type":"number"},"top_p":{"default":1,"description":"The top-p sampling mass used for text generation. The top-p value determines the probability mass that is sampled at sampling time. For example, if top_p = 0.2, only the most likely tokens (summing to 0.2 cumulative probability) will be sampled. It is not recommended to modify both temperature and top_p in the same call.","maximum":1,"exclusiveMinimum":0,"title":"Top P","type":"number"},"frequency_penalty":{"type":"number","maximum":2,"minimum":-2,"default":0,"title":"Frequency Penalty","description":"Indicates how much to penalize new tokens based on their existing frequency in the text so far, decreasing model likelihood to repeat the same line verbatim."},"presence_penalty":{"type":"number","maximum":2,"minimum":-2,"default":0,"title":"Presence Penalty","description":"Positive values penalize new tokens based on whether they appear in the text so far, increasing model likelihood to talk about new topics."},"max_tokens":{"default":16384,"description":"The maximum number of tokens to generate in any given call. Note that the model is not aware of this value, and generation will simply stop at the number of tokens specified.","maximum":16384,"minimum":1,"title":"Max Tokens","type":"integer"},"stream":{"default":false,"description":"If set, partial message deltas will be sent. Tokens will be sent as data-only server-sent events (SSE) as they become available (JSON responses are prefixed by `data: `), with the stream terminated by a `data: [DONE]` message.","title":"Stream","type":"boolean"},"stop":{"anyOf":[{"items":{"type":"string"},"type":"array"},{"type":"string"},{"type":"null"}],"title":"Stop","description":"A string or a list of strings where the API will stop generating further tokens. The returned text will not contain the stop sequence."}},"required":["messages"],"title":"ChatRequest","type":"object"}}},"required":true},"responses":{"200":{"description":"Invocation is fulfilled","content":{"application/json":{"schema":{"properties":{"id":{"description":"A unique identifier for the completion.","format":"uuid","title":"Id","type":"string"},"choices":{"description":"The list of completion choices the model generated for the input prompt.","items":{"properties":{"index":{"description":"The index of the choice in the list of choices (always 0).","title":"Index","type":"integer"},"message":{"allOf":[{"additionalProperties":false,"properties":{"role":{"description":"The role of the message author.","enum":["user","assistant"],"title":"Role","type":"string"},"content":{"description":"The contents of the message.","title":"Content","anyOf":[{"type":"string"},{"type":"null"}]}},"required":["role","content"],"title":"Message","type":"object"}],"description":"A chat completion message generated by the model.","examples":[{"content":"Ah, Paris, the City of Light! There are so many amazing things to see and do in this beautiful city ...","role":"assistant"}]},"finish_reason":{"anyOf":[{"enum":["stop","length"],"type":"string"},{"type":"null"}],"default":null,"description":"The reason the model stopped generating tokens. This will be `stop` if the model hit a natural stop point or a provided stop sequence, or `length` if the maximum number of tokens specified in the request was reached.","examples":["stop"],"title":"Finish Reason"}},"required":["index","message"],"title":"Choice","type":"object"},"title":"Choices","type":"array"},"usage":{"allOf":[{"properties":{"completion_tokens":{"description":"Number of tokens in the generated completion.","examples":[25],"title":"Completion Tokens","type":"integer"},"prompt_tokens":{"description":"Number of tokens in the prompt.","examples":[9],"title":"Prompt Tokens","type":"integer"},"total_tokens":{"description":"Total number of tokens used in the request (prompt + completion).","examples":[34],"title":"Total Tokens","type":"integer"}},"required":["completion_tokens","prompt_tokens","total_tokens"],"title":"Usage","type":"object"}],"description":"Usage statistics for the completion request."}},"required":["id","choices","usage"],"title":"ChatCompletion","type":"object"}},"text/event-stream":{"schema":{"properties":{"id":{"description":"A unique identifier for the completion.","format":"uuid","title":"Id","type":"string"},"choices":{"description":"The list of completion choices the model generated for the input prompt.","items":{"properties":{"index":{"description":"The index of the choice in the list of choices (always 0).","title":"Index","type":"integer"},"delta":{"allOf":[{"additionalProperties":false,"properties":{"role":{"description":"The role of the message author.","enum":["user","assistant"],"title":"Role","type":"string"},"content":{"description":"The contents of the message.","title":"Content","anyOf":[{"type":"string"},{"type":"null"}]}},"required":["role","content"],"title":"Message","type":"object"}],"description":"A chat completion delta generated by streamed model responses.","examples":[{"content":"Ah,","role":"assistant"}]},"finish_reason":{"anyOf":[{"enum":["stop","length"],"type":"string"},{"type":"null"}],"default":null,"description":"The reason the model stopped generating tokens. This will be `stop` if the model hit a natural stop point or a provided stop sequence, or `length` if the maximum number of tokens specified in the request was reached. Will be `null` if the model has not finished generating.","title":"Finish Reason"}},"required":["index","delta"],"title":"ChoiceChunk","type":"object"},"title":"Choices","type":"array"}},"required":["id","choices"],"title":"ChatCompletionChunk","type":"object"}}}},"202":{"description":"Result is pending. Client should poll using the requestId.\n","content":{"application/json":{"example":{},"schema":{}}},"headers":{"NVCF-REQID":{"description":"requestId required for pooling","schema":{"type":"string","format":"uuid"}},"NVCF-STATUS":{"description":"Invocation status","schema":{"type":"string"}}}},"422":{"description":"Validation failed, provided entity could not be processed.","content":{"application/json":{"schema":{"properties":{"type":{"type":"string","description":"Error type"},"title":{"type":"string","description":"Error title"},"status":{"type":"integer","description":"Error status code"},"detail":{"type":"string","description":"Detailed information about the error"},"instance":{"type":"string","description":"Function instance used to invoke the request"},"requestId":{"type":"string","format":"uuid","description":"UUID of the request"}},"type":"object","required":["type","title","status","detail","instance","requestId"],"title":"InvokeError"},"example":{"type":"urn:nvcf-worker-service:problem-details:unprocessable-entity","title":"Unprocessable Entity","status":422,"detail":"string","instance":"/v2/nvcf/pexec/functions/4a58c6cb-a9b4-4014-99de-3e704d4ae687","requestId":"3fa85f64-5717-4562-b3fc-2c963f66afa6"}}}},"500":{"description":"The invocation ended with an error.","content":{"application/json":{"schema":{"properties":{"type":{"type":"string","description":"Error type"},"title":{"type":"string","description":"Error title"},"status":{"type":"integer","description":"Error status code"},"detail":{"type":"string","description":"Detailed information about the error"},"instance":{"type":"string","description":"Function instance used to invoke the request"},"requestId":{"type":"string","format":"uuid","description":"UUID of the request"}},"type":"object","required":["type","title","status","detail","instance","requestId"],"title":"InvokeError"},"example":{"type":"urn:nvcf-worker-service:problem-details:internal-server-error","title":"Internal Server Error","status":500,"detail":"string","instance":"/v2/nvcf/pexec/functions/4a58c6cb-a9b4-4014-99de-3e704d4ae687","requestId":"3fa85f64-5717-4562-b3fc-2c963f66afa6"}}}}},"x-nvai-meta":{"name":"Create chat completion","returns":"Returns a [chat completion](/docs/api-reference/chat/object) object, or a streamed sequence of [chat completion chunk](/docs/api-reference/chat/streaming) objects if the request is streamed.\n","path":"create","examples":[{"name":"Which number is larger, 9.11 or 9.8?","requestJson":"{\n \"model\": \"sarvamai/sarvam-m\",\n \"messages\": [\n {\n \"role\": \"user\",\n \"content\": \"Which number is larger, 9.11 or 9.8?\"\n }\n ],\n \"temperature\": 0.5,\n \"top_p\": 1.0,\n \"max_tokens\": 16384,\n \"seed\": 42,\n \"stream\": true\n}\n","responseJson":"{\n \"id\": \"id-123\",\n \"object\": \"chat.completion\",\n \"created\": 1677652288,\n \"model\": \"sarvamai/sarvam-m\",\n \"system_fingerprint\": \"fp_44709d6fcb\",\n \"choices\": [{\n \"index\": 0,\n \"message\": {\n \"role\": \"assistant\",\n \"content\": \"The number 9.11 is...\"\n },\n \"finish_reason\": \"stop\"\n }],\n \"usage\": {\n \"prompt_tokens\": 9,\n \"completion_tokens\": 12,\n \"total_tokens\": 21\n }\n}\n"},{"name":"How many 'r's are in the word 'strawberry'?","requestJson":"{\n \"model\": \"sarvamai/sarvam-m\",\n \"messages\": [\n {\n \"role\": \"user\",\n \"content\": \"How many 'r's are in the word 'strawberry'?\"\n }\n ],\n \"temperature\": 0.5,\n \"top_p\": 1.0,\n \"max_tokens\": 16384,\n \"seed\": 42,\n \"stream\": true\n}\n","responseJson":"{\n \"id\": \"id-123\",\n \"object\": \"chat.completion\",\n \"created\": 1677652288,\n \"model\": \"sarvamai/sarvam-m\",\n \"system_fingerprint\": \"fp_44709d6fcb\",\n \"choices\": [{\n \"index\": 0,\n \"message\": {\n \"role\": \"assistant\",\n \"content\": \"There are 3 'r's in the word 'strawberry'.\"\n },\n \"finish_reason\": \"stop\"\n }],\n \"usage\": {\n \"prompt_tokens\": 9,\n \"completion_tokens\": 12,\n \"total_tokens\": 21\n }\n}\n"}],"templates":[{"title":"No Streaming","requestEjs":{"python":"$4e","langChain":"from langchain_nvidia_ai_endpoints import ChatNVIDIA\n\nclient = ChatNVIDIA(\n model=\"<%- request.model %>\",\n api_key=\"$NVIDIA_API_KEY\", \n temperature=<%- request.temperature %>,\n top_p=<%- request.top_p %>,\n max_tokens=<%- request.max_tokens %>,\n)\n<% if (request.stream) { %>\nfor chunk in client.stream(<%- JSON.stringify(request.messages) %>):\n print(chunk.content, end=\"\")\n<% } else { %>\nresponse = client.invoke(<%- JSON.stringify(request.messages) %>)\nprint(response.content)\n<% } %>\n","node.js":"$4f","curl":"$50"},"response":"{\n \"id\": \"chatcmpl-123\",\n \"object\": \"chat.completion\",\n \"created\": 1677652288,\n \"model\": \"sarvamai/sarvam-m\",\n \"system_fingerprint\": \"fp_44709d6fcb\",\n \"choices\": [{\n \"index\": 0,\n \"message\": {\n \"role\": \"assistant\",\n \"content\": \"\\n\\nHello there, how may I assist you today?\",\n },\n \"finish_reason\": \"stop\"\n }],\n \"usage\": {\n \"prompt_tokens\": 9,\n \"completion_tokens\": 12,\n \"total_tokens\": 21\n }\n}\n"}]}}}},"security":[{"Token":[]}],"components":{"securitySchemes":{"Token":{"type":"http","scheme":"bearer"}},"schemas":{"Errors":{"properties":{"type":{"type":"string","description":"Error type"},"title":{"type":"string","description":"Error title"},"status":{"type":"integer","description":"Error status code"},"detail":{"type":"string","description":"Detailed information about the error"},"instance":{"type":"string","description":"Function instance used to invoke the request"},"requestId":{"type":"string","format":"uuid","description":"UUID of the request"}},"type":"object","required":["type","title","status","detail","instance","requestId"],"title":"InvokeError"},"ChatCompletion":{"properties":{"id":{"description":"A unique identifier for the completion.","format":"uuid","title":"Id","type":"string"},"choices":{"description":"The list of completion choices the model generated for the input prompt.","items":{"properties":{"index":{"description":"The index of the choice in the list of choices (always 0).","title":"Index","type":"integer"},"message":{"allOf":[{"additionalProperties":false,"properties":{"role":{"description":"The role of the message author.","enum":["user","assistant"],"title":"Role","type":"string"},"content":{"description":"The contents of the message.","title":"Content","anyOf":[{"type":"string"},{"type":"null"}]}},"required":["role","content"],"title":"Message","type":"object"}],"description":"A chat completion message generated by the model.","examples":[{"content":"Ah, Paris, the City of Light! There are so many amazing things to see and do in this beautiful city ...","role":"assistant"}]},"finish_reason":{"anyOf":[{"enum":["stop","length"],"type":"string"},{"type":"null"}],"default":null,"description":"The reason the model stopped generating tokens. This will be `stop` if the model hit a natural stop point or a provided stop sequence, or `length` if the maximum number of tokens specified in the request was reached.","examples":["stop"],"title":"Finish Reason"}},"required":["index","message"],"title":"Choice","type":"object"},"title":"Choices","type":"array"},"usage":{"allOf":[{"properties":{"completion_tokens":{"description":"Number of tokens in the generated completion.","examples":[25],"title":"Completion Tokens","type":"integer"},"prompt_tokens":{"description":"Number of tokens in the prompt.","examples":[9],"title":"Prompt Tokens","type":"integer"},"total_tokens":{"description":"Total number of tokens used in the request (prompt + completion).","examples":[34],"title":"Total Tokens","type":"integer"}},"required":["completion_tokens","prompt_tokens","total_tokens"],"title":"Usage","type":"object"}],"description":"Usage statistics for the completion request."}},"required":["id","choices","usage"],"title":"ChatCompletion","type":"object"},"ChatCompletionChunk":{"properties":{"id":{"description":"A unique identifier for the completion.","format":"uuid","title":"Id","type":"string"},"choices":{"description":"The list of completion choices the model generated for the input prompt.","items":{"properties":{"index":{"description":"The index of the choice in the list of choices (always 0).","title":"Index","type":"integer"},"delta":{"allOf":[{"additionalProperties":false,"properties":{"role":{"description":"The role of the message author.","enum":["user","assistant"],"title":"Role","type":"string"},"content":{"description":"The contents of the message.","title":"Content","anyOf":[{"type":"string"},{"type":"null"}]}},"required":["role","content"],"title":"Message","type":"object"}],"description":"A chat completion delta generated by streamed model responses.","examples":[{"content":"Ah,","role":"assistant"}]},"finish_reason":{"anyOf":[{"enum":["stop","length"],"type":"string"},{"type":"null"}],"default":null,"description":"The reason the model stopped generating tokens. This will be `stop` if the model hit a natural stop point or a provided stop sequence, or `length` if the maximum number of tokens specified in the request was reached. Will be `null` if the model has not finished generating.","title":"Finish Reason"}},"required":["index","delta"],"title":"ChoiceChunk","type":"object"},"title":"Choices","type":"array"}},"required":["id","choices"],"title":"ChatCompletionChunk","type":"object"},"ChatRequest":{"additionalProperties":false,"properties":{"model":{"type":"string","title":"Model","default":"sarvamai/sarvam-m"},"messages":{"description":"A list of messages comprising the conversation so far. The roles of the messages must be alternating between `user` and `assistant`. The last input message should have role `user`. A message with the the `system` role is optional, and must be the very first message if it is present; `context` is also optional, but must come before a user question.","examples":[[{"content":"I am going to Paris, what should I see?","role":"user"}]],"items":{"additionalProperties":false,"properties":{"role":{"description":"The role of the message author.","enum":["user","assistant"],"title":"Role","type":"string"},"content":{"description":"The contents of the message.","title":"Content","anyOf":[{"type":"string"},{"type":"null"}]}},"required":["role","content"],"title":"Message","type":"object"},"title":"Messages","type":"array"},"temperature":{"default":0.5,"description":"The sampling temperature to use for text generation. The higher the temperature value is, the less deterministic the output text will be. It is not recommended to modify both temperature and top_p in the same call.","maximum":1,"minimum":0,"title":"Temperature","type":"number"},"top_p":{"default":1,"description":"The top-p sampling mass used for text generation. The top-p value determines the probability mass that is sampled at sampling time. For example, if top_p = 0.2, only the most likely tokens (summing to 0.2 cumulative probability) will be sampled. It is not recommended to modify both temperature and top_p in the same call.","maximum":1,"exclusiveMinimum":0,"title":"Top P","type":"number"},"frequency_penalty":{"type":"number","maximum":2,"minimum":-2,"default":0,"title":"Frequency Penalty","description":"Indicates how much to penalize new tokens based on their existing frequency in the text so far, decreasing model likelihood to repeat the same line verbatim."},"presence_penalty":{"type":"number","maximum":2,"minimum":-2,"default":0,"title":"Presence Penalty","description":"Positive values penalize new tokens based on whether they appear in the text so far, increasing model likelihood to talk about new topics."},"max_tokens":{"default":16384,"description":"The maximum number of tokens to generate in any given call. Note that the model is not aware of this value, and generation will simply stop at the number of tokens specified.","maximum":16384,"minimum":1,"title":"Max Tokens","type":"integer"},"stream":{"default":false,"description":"If set, partial message deltas will be sent. Tokens will be sent as data-only server-sent events (SSE) as they become available (JSON responses are prefixed by `data: `), with the stream terminated by a `data: [DONE]` message.","title":"Stream","type":"boolean"},"stop":{"anyOf":[{"items":{"type":"string"},"type":"array"},{"type":"string"},{"type":"null"}],"title":"Stop","description":"A string or a list of strings where the API will stop generating further tokens. The returned text will not contain the stop sequence."}},"required":["messages"],"title":"ChatRequest","type":"object"},"Choice":{"properties":{"index":{"description":"The index of the choice in the list of choices (always 0).","title":"Index","type":"integer"},"message":{"allOf":[{"additionalProperties":false,"properties":{"role":{"description":"The role of the message author.","enum":["user","assistant"],"title":"Role","type":"string"},"content":{"description":"The contents of the message.","title":"Content","anyOf":[{"type":"string"},{"type":"null"}]}},"required":["role","content"],"title":"Message","type":"object"}],"description":"A chat completion message generated by the model.","examples":[{"content":"Ah, Paris, the City of Light! There are so many amazing things to see and do in this beautiful city ...","role":"assistant"}]},"finish_reason":{"anyOf":[{"enum":["stop","length"],"type":"string"},{"type":"null"}],"default":null,"description":"The reason the model stopped generating tokens. This will be `stop` if the model hit a natural stop point or a provided stop sequence, or `length` if the maximum number of tokens specified in the request was reached.","examples":["stop"],"title":"Finish Reason"}},"required":["index","message"],"title":"Choice","type":"object"},"ChoiceChunk":{"properties":{"index":{"description":"The index of the choice in the list of choices (always 0).","title":"Index","type":"integer"},"delta":{"allOf":[{"additionalProperties":false,"properties":{"role":{"description":"The role of the message author.","enum":["user","assistant"],"title":"Role","type":"string"},"content":{"description":"The contents of the message.","title":"Content","anyOf":[{"type":"string"},{"type":"null"}]}},"required":["role","content"],"title":"Message","type":"object"}],"description":"A chat completion delta generated by streamed model responses.","examples":[{"content":"Ah,","role":"assistant"}]},"finish_reason":{"anyOf":[{"enum":["stop","length"],"type":"string"},{"type":"null"}],"default":null,"description":"The reason the model stopped generating tokens. This will be `stop` if the model hit a natural stop point or a provided stop sequence, or `length` if the maximum number of tokens specified in the request was reached. Will be `null` if the model has not finished generating.","title":"Finish Reason"}},"required":["index","delta"],"title":"ChoiceChunk","type":"object"},"Message":{"additionalProperties":false,"properties":{"role":{"description":"The role of the message author.","enum":["user","assistant"],"title":"Role","type":"string"},"content":{"description":"The contents of the message.","title":"Content","anyOf":[{"type":"string"},{"type":"null"}]}},"required":["role","content"],"title":"Message","type":"object"},"Usage":{"properties":{"completion_tokens":{"description":"Number of tokens in the generated completion.","examples":[25],"title":"Completion Tokens","type":"integer"},"prompt_tokens":{"description":"Number of tokens in the prompt.","examples":[9],"title":"Prompt Tokens","type":"integer"},"total_tokens":{"description":"Total number of tokens used in the request (prompt + completion).","examples":[34],"title":"Total Tokens","type":"integer"}},"required":["completion_tokens","prompt_tokens","total_tokens"],"title":"Usage","type":"object"}}}},"pageLayout":"v3"},"children":["$L51","$L52","$L53"]}]