---
title: "solar-10.7b-instruct"
publisher: "upstage"
type: "endpoint"
updated: "2025-04-10T01:11:57.895Z"
description: "Excels in NLP tasks, particularly in instruction-following, reasoning, and mathematics."
canonical: "https://build.nvidia.com/upstage/solar-10_7b-instruct"
---

# Model Overview

## Description

SOLAR-10.7B, is an advanced large language model (LLM) with 10.7 billion parameters, that demonstrates superior performance in various natural language processing (NLP) tasks. It's compact, yet remarkably powerful, and demonstrates unparalleled state-of-the-art performance in models with parameters under 30B.

It uses a methodology for scaling LLMs called depth up-scaling (DUS), which encompasses architectural modifications and continued pretraining. In other words, it integrates Mistral 7B weights into the upscaled layers, and finally, continues pre-training for the entire model. It outperforms models with up to 30B parameters, even surpassing the Mixtral 8X7B model.

We at NVIDIA have optimized SOLAR-10.7B using TensorRT-LLM to run optimally on latest NVIDIA GPUs.

## Third-Party Community Consideration

This model is not owned or developed by NVIDIA. This model has been developed and built to a third-party’s requirements for this application and use case; see link to the [SOLAR-10.7B-Instruct-v1.0  Model Card](https://huggingface.co/upstage/SOLAR-10.7B-Instruct-v1.0).

## License and Terms of use
<b>GOVERNING TERMS</b>: Your use of this API is governed by the <a href="https://assets.ngc.nvidia.com/products/api-catalog/legal/NVIDIA%20API%20Trial%20Terms%20of%20Service.pdf" rel="noreferrer" target="_blank">NVIDIA API Trial Service Terms of Use</a>; and the use of this model is governed by the <a href="https://docs.nvidia.com/ai-foundation-models-community-license.pdf" rel="noreferrer" target="_blank">NVIDIA AI Foundation Models Community License</a> and <a href="https://creativecommons.org/licenses/by-nc/4.0/legalcode.en" rel="noreferrer" target="_blank">CC BY-NC 4.0 License</a>.

**Model Developer:** Upstage <br> 
**Model Release Date:** December 13, 2023

**Model Architecture** 
* Architecture Type: Transformer <br> 
* Network Architecture: Llama

**Input** 
* Input Type: Text
* Input Format: String
* Input Parameters: max_tokens, temperature, top_p, stop, frequency_penalty, presence_penalty, seed

**Output** 
* Output Type: Text
* Output Format: String

## Software Integration:
* Supported Hardware Platform(s): NVIDIA Lovelace <br>

**[Preferred/Supported] Operating System(s):** 
* Linux <br>

## Inference

**Engine:** TensorRT-LLM <br>
**Test Hardware:** L40S <br>

## Usage Instructions

This model has been fine-tuned primarily for single-turn conversation, making it less suitable for multi-turn conversations such as chat.

## Ethical Considerations

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).

## Prototype

```python
from openai import OpenAI

client = OpenAI(
base_url = "https://integrate.api.nvidia.com/v1",
api_key = "$NVIDIA_API_KEY"
)

completion = client.chat.completions.create(
model="",
messages=[{"role":"user","content":""}],
temperature=,
top_p=,
max_tokens=,
stream=NaN
)

print(completion.choices[0].message)
```

```python
from langchain_nvidia_ai_endpoints import ChatNVIDIA

client = ChatNVIDIA(
model="",
api_key="$NVIDIA_API_KEY", 
temperature=,
top_p=,
max_tokens=,
)

response = client.invoke([{"role":"user","content":""}])
print(response.content)
```

```javascript
import OpenAI from 'openai';

const openai = new OpenAI({
apiKey: '$NVIDIA_API_KEY',
baseURL: 'https://integrate.api.nvidia.com/v1',
})

async function main() {
const completion = await openai.chat.completions.create({
model: "",
messages: [{"role":"user","content":""}],
temperature: ,
top_p: ,
max_tokens: ,
stream: ,
})

process.stdout.write(completion.choices[0]?.message?.content);

}

main();
```

```bash
curl https://integrate.api.nvidia.com/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $NVIDIA_API_KEY" \
-d '{
"model": "upstage/solar-10.7b-instruct",
"messages": [{"role":"user","content":""}],
"temperature": ,   
"top_p": ,
"max_tokens": ,
"stream":                 
}'
```