Model Overview

Description

SOLAR-10.7B, is an advanced large language model (LLM) with 10.7 billion parameters, that demonstrates superior performance in various natural language processing (NLP) tasks. It's compact, yet remarkably powerful, and demonstrates unparalleled state-of-the-art performance in models with parameters under 30B.

It uses a methodology for scaling LLMs called depth up-scaling (DUS), which encompasses architectural modifications and continued pretraining. In other words, it integrates Mistral 7B weights into the upscaled layers, and finally, continues pre-training for the entire model. It outperforms models with up to 30B parameters, even surpassing the Mixtral 8X7B model.

We at NVIDIA have optimized SOLAR-10.7B using TensorRT-LLM to run optimally on latest NVIDIA GPUs.

Third-Party Community Consideration

This model is not owned or developed by NVIDIA. This model has been developed and built to a third-party’s requirements for this application and use case; see link to the SOLAR-10.7B-Instruct-v1.0 Model Card.

License and Terms of use

GOVERNING TERMS: Your use of this API is governed by the NVIDIA API Trial Service Terms of Use; and the use of this model is governed by the NVIDIA AI Foundation Models Community License and CC BY-NC 4.0 License.

Model Developer: Upstage
Model Release Date: December 13, 2023

Model Architecture

• Architecture Type: Transformer
  
• Network Architecture: Llama

Input

• Input Type: Text
• Input Format: String
• Input Parameters: max_tokens, temperature, top_p, stop, frequency_penalty, presence_penalty, seed

Output

• Output Type: Text
• Output Format: String

Software Integration:

• Supported Hardware Platform(s): NVIDIA Lovelace
  

[Preferred/Supported] Operating System(s):

• Linux
  

Inference

Engine: TensorRT-LLM
Test Hardware: L40S

Usage Instructions

This model has been fine-tuned primarily for single-turn conversation, making it less suitable for multi-turn conversations such as chat.

Ethical Considerations

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse. Please report security vulnerabilities or NVIDIA AI Concerns here.