---
title: "llama2-70b"
publisher: "meta"
type: "endpoint"
updated: "2025-01-17T20:29:36.364Z"
description: "Cutting-edge large language AI model capable of generating text and code in response to prompts."
canonical: "https://build.nvidia.com/meta/llama2-70b"
---

# Model Overview

## Description:

Llama 2 is a large language AI model comprising a collection of models capable of generating text and code in response to prompts.

## Third-Party Community Consideration:
This model is not owned or developed by NVIDIA. This model has been developed and built to a third-party’s requirements for this application and use case; see [Meta's Code Llama Model Card](https://github.com/facebookresearch/codellama/blob/main/MODEL_CARD.md).

## Terms of use

By accessing this model, you are agreeing to the Llama 2 terms and conditions of the [license](https://l.facebook.com/l.php?u=https%3A%2F%2Fai.meta.com%2Fllama%2Flicense&h=AT0DpDKXSE-Yi_pulcu4-5oKNHyiayGrLbQqW7qI_4Ez5y-67Nmyyu38D5NcEmGSifLOtDZAJaQOPMGJg8zCFDJjbYcyJZQPawxWU3-GaVdgStbboYT00yqZvyrxP1DVlbfAog), [acceptable use policy](https://l.facebook.com/l.php?u=https%3A%2F%2Fai.meta.com%2Fllama%2Fuse-policy&h=AT17YXIWa1dxwuzDq96w614_oshaUCWIQYHHHf9BgC0X5UK39DW2tpSP6NzkeadQVs0XBaNlFSo4nydTUcKB5r-Kn4OLs_5IbMEZYAf5hUvyzIwx8-mLom2ic1vw5rSPFUoc9A) and [Meta's privacy policy](https://www.facebook.com/privacy/policy/)

## References(s):

[Llama 2: Open Foundation and Fine-Tuned Chat Models paper](https://scontent-lhr8-2.xx.fbcdn.net/v/t39.2365-6/10000000_662098952474184_2584067087619170692_n.pdf?_nc_cat=105&ccb=1-7&_nc_sid=3c67a6&_nc_ohc=SvLw337Y0sQAX8zTcqT&_nc_ht=scontent-lhr8-2.xx&oh=00_AfC2GBa-ZPdVLSxOnnjuTZHTDu51NZUcR6KubtLe5tEUbw&oe=64D0333F) <br>
[Meta's Llama 2 webpage](https://ai.meta.com/llama/) <br>
[Meta's Llama 2 Model Card webpage](https://github.com/facebookresearch/llama/blob/main/MODEL_CARD.md) <br>

## Model Architecture:
**Architecture Type:** Transformer <br>
**Network Architecture:** Llama 2 <br>
**Model Version:** 0.1 <br>

## Input:
**Input Format:** Text <br>
**Input Parameters:** Temperature, TopP <br>

## Output:
**Output Format:** Text <br>
**Output Parameters:** Max Output Tokens <br>

## Software Integration:
**Supported Hardware Platform(s):** Hopper, Ampere, Turing <br>
**Supported Operating System(s):** Linux <br>

# Inference:
**Engine:** [Triton](https://developer.nvidia.com/triton-inference-server) <br>
**Test Hardware:** Other <br>

## Ethical Considerations:
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications.  When downloaded or used in accordance with our terms of service, developers should work with their supporting model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.  For more detailed information on ethical considerations for this model, please see the Model Card++ Explainability, Bias, Safety & Security, and Privacy Subcards [Insert Link to Model Card++ here].  Please report security vulnerabilities or NVIDIA AI Concerns [here](https://www.nvidia.com/en-us/support/submit-security-vulnerability/).

## Prototype

```python
from openai import OpenAI

client = OpenAI(
base_url = "https://integrate.api.nvidia.com/v1",
api_key = "$NVIDIA_API_KEY"
)

completion = client.chat.completions.create(
model="",
messages=[{"role":"user","content":""}],
temperature=,
top_p=,
max_tokens=,
stream=NaN
)

print(completion.choices[0].message)
```

```python
from langchain_nvidia_ai_endpoints import ChatNVIDIA

client = ChatNVIDIA(
model="",
api_key="$NVIDIA_API_KEY", 
temperature=,
top_p=,
max_tokens=,
)

response = client.invoke([{"role":"user","content":""}])
print(response.content)
```

```javascript
import OpenAI from 'openai';

const openai = new OpenAI({
apiKey: '$NVIDIA_API_KEY',
baseURL: 'https://integrate.api.nvidia.com/v1',
})

async function main() {
const completion = await openai.chat.completions.create({
model: "",
messages: [{"role":"user","content":""}],
temperature: ,
top_p: ,
max_tokens: ,
stream: ,
})

process.stdout.write(completion.choices[0]?.message?.content);

}

main();
```

```bash
curl https://integrate.api.nvidia.com/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $NVIDIA_API_KEY" \
-d '{
"model": "meta/llama2-70b",
"messages": [{"role":"user","content":""}],
"temperature": ,   
"top_p": ,
"max_tokens": ,
"stream":                 
}'
```