---
title: "falcon3-7b-instruct"
publisher: "tiiuae"
type: "endpoint"
updated: "2025-05-22T19:22:18.825Z"
description: "Instruction tuned LLM achieving SoTA performance on reasoning, math and general knowledge capabilities"
canonical: "https://build.nvidia.com/tiiuae/falcon3-7b-instruct"
---

**Model Overview**

## Description:
The **Falcon3-7B-Instruct** is an open foundation model designed for state-of-the-art performance in reasoning, language understanding, instruction following, code generation, and mathematics. It supports long-context tasks with a token limit of up to **32K** and multilingual capabilities in **English, French, Spanish, and Portuguese**.  
This model is for **research and development purposes only**.  

## Third-Party Community Consideration
This model is not owned or developed by NVIDIA. This model has been developed and built to a third-party’s requirements for this application and use case; see link to Non-NVIDIA [TII Model Card](https://huggingface.co/tiiuae/Falcon3-7B-Instruct).

### License/Terms of Use
GOVERNING TERMS: This trial service is governed by the [NVIDIA API Trial Terms of Service](https://assets.ngc.nvidia.com/products/api-catalog/legal/NVIDIA%20API%20Trial%20Terms%20of%20Service.pdf). Use of this model is governed by the [NVIDIA Community Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-community-models-license/). Additional Information: [Falcon 3 TII Falcon License](https://falconllm.tii.ae/falcon-terms-and-conditions.html). The NVIDIA-optimized Falcon3-7B-Instruct is built using artificial intelligence technology from the Technology Innovation Institute.

## References:
* [Falcon3-7B-Instruct Model Card](https://huggingface.co/tiiuae/Falcon3-7B-Instruct)
* [TII Website](https://www.tii.ae/)
* [Falcon 3 Family Release Blog](https://huggingface.co/blog/falcon3)

## Model Architecture:
**Architecture Type:** Transformer <br>
**Network Architecture:** Transformer Decoder Only Architecture <br>
**Model Details:** <br>
* Transformer-based causal decoder-only design. <br>
* Composed of 28 decoder blocks, leveraging Grouped Query Attention (GQA) for faster inference with 12 query heads and 4 key-value heads. <br>
* Wider head dimension of 256 for enhanced performance. <br>
* High RoPE (Rotary Positional Embedding) value of 1000042, enabling extended context understanding up to 32,000 tokens. <br>
* Incorporates SwiGLU activation and RMSNorm for improved training stability and efficiency. <br>

## Input:
**Input Type(s):** Text <br>
**Input Format(s):** String <br>
**Input Parameters:** (1D) <br>
**Other Properties Related to Input:** Supports multilingual input (EN, FR, ES, PT) and Context length up to 32,000 tokens.

## Output:
**Output Type(s):** Text <br>
**Output Format:** String <br>
**Output Parameters:** (1D) <br>
**Other Properties Related to Output:** Generates outputs in supported languages with capabilities across reasoning, code, and instructional tasks.

## Software Integration:
**Runtime Engine(s):** Not specified; supports standard machine learning pipelines such as PyTorch and Hugging Face <br>

**Supported Hardware Microarchitecture Compatibility:** <br>
* NVIDIA Ampere <br>
* NVIDIA Hopper <br>

**[Preferred/Supported] Operating System(s):** Linux

## Model Version(s):
* Falcon3-7B-Instruct v1.0 <br>
* Initial version released in December 2024

# Training, Testing, and Evaluation Datasets:

## Training Dataset:
**Link:** Not publicly available <br>
**Data Collection Method by dataset:** Hybrid (Automated, Human) <br>
**Labeling Method by dataset:** Hybrid (Automated, Human) <br>
**Properties:** 
* Pretrained on 14 teratokens of web, code, STEM, multilingual, and high-quality datasets <br>
* Post-trained on 1.2 million samples of STEM, conversations, code, safety, and function call data <br>

## Testing Dataset:
**Link:** Not publicly available <br>
**Data Collection Method by dataset:** Hybrid (Automated, Human) <br>
**Labeling Method by dataset:** Hybrid (Automated, Human) <br>
**Properties:** NA <br>

## Evaluation Dataset:
**Link:** Not publicly available <br>
**Data Collection Method by dataset:** Hybrid (Automated, Human) <br>
**Labeling Method by dataset:**  Unknown <br>
**Properties:** Benchmark scores for various models, including Falcon3-7B-Instruct, Qwen2.5-7B-Instruct, and Llama-3.1-8B-Instruct <br>

## Inference:
**Engine:** TensorRT-LLM <br>
**Test Hardware:** NVIDIA Ampere

## Ethical Considerations:
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications.  When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).

## Prototype

```python
from openai import OpenAI

client = OpenAI(
base_url = "https://integrate.api.nvidia.com/v1",
api_key = "$NVIDIA_API_KEY"
)

completion = client.chat.completions.create(
model="",
messages=[{"role":"user","content":""}],
temperature=,
top_p=,
max_tokens=,
stream=NaN
)

print(completion.choices[0].message)
```

```python
from langchain_nvidia_ai_endpoints import ChatNVIDIA

client = ChatNVIDIA(
model="",
api_key="$NVIDIA_API_KEY", 
temperature=,
top_p=,
max_tokens=,
)

response = client.invoke([{"role":"user","content":""}])
print(response.content)
```

```javascript
import OpenAI from 'openai';

const openai = new OpenAI({
apiKey: '$NVIDIA_API_KEY',
baseURL: 'https://integrate.api.nvidia.com/v1',
})

async function main() {
const completion = await openai.chat.completions.create({
model: "",
messages: [{"role":"user","content":""}],
temperature: ,
top_p: ,
max_tokens: ,
stream: 
})

process.stdout.write(completion.choices[0]?.message?.content);

}

main();
```

```bash
invoke_url='https://integrate.api.nvidia.com/v1/chat/completions'

authorization_header='Authorization: Bearer '
accept_header='Accept: application/json'
content_type_header='Content-Type: application/json'

data=$'{
"messages": [
{
"role": "user",
"content": ""
}
]
}'

response=$(curl --silent -i -w "\n%{http_code}" --request POST \
--url "$invoke_url" \
--header "$authorization_header" \
--header "$accept_header" \
--header "$content_type_header" \
--data "$data"
)

echo "$response"
```