yentinglin/llama-3-taiwan-70b-instruct

RUN ANYWHERE

Sovereign AI model finetuned on Traditional Mandarin and English data using the Llama-3 architecture.

Chat
Large Language Models
regional language generation
Code Generation
API Reference

By running the below commands, you accept the NVIDIA AI Enterprise Terms of Use and the NVIDIA Community Models License.

Pull and run yentinglin/llama-3-taiwan-70b-instruct using Docker (this will download the full model and run it in your local environment)

$ docker login nvcr.io
Username: $oauthtoken
Password: <PASTE_API_KEY_HERE>

Pull and run the NVIDIA NIM with the command below. This will download the optimized model for your infrastructure.

export NGC_API_KEY=<PASTE_API_KEY_HERE>
export LOCAL_NIM_CACHE=~/.cache/nim
mkdir -p "$LOCAL_NIM_CACHE"
docker run -it --rm \
    --gpus all \
    --shm-size=16GB \
    -e NGC_API_KEY \
    -v "$LOCAL_NIM_CACHE:/opt/nim/.cache" \
    -u $(id -u) \
    -p 8000:8000 \
    nvcr.io/nim/yentinglin/llama-3-taiwan-70b-instruct:latest

You can now make a local API call using this curl command:

curl -X 'POST' \
'http://0.0.0.0:8000/v1/chat/completions' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
    "model": "yentinglin/llama-3-taiwan-70b-instruct",
    "messages": [{"role":"user", "content":"台灣的夜市文化有什麼特色?"}],
    "max_tokens": 64
}'

For more details on getting started with this NIM, visit the NVIDIA NIM Docs.