StarCoder2 Model Overview

Description:

StarCoder2-7b is a state-of-the-art language model with 7 billion parameters, trained on 17 programming languages using The Stack v2 dataset. It employs advanced techniques like Grouped Query Attention and sliding window attention to enhance its performance on coding tasks. The model is optimized to handle a context window of 16,384 tokens and was trained using the Fill-in-the-Middle objective on 3.5+ trillion tokens.

Terms of Use

GOVERNING TERMS: Your use of this model is governed by the BigCode OpenRAIL-M v1 License Agreement.

References(s):

StarCoder2-7b on Hugging Face

Model Architecture:

Architecture Type: Transformer decoder
Network Architecture: Grouped Query Attention, sliding window attention
Model Version: 2.0

Input:

Input Format: Text
Input Parameters: Temperature, Top P, Max Output Tokens

Output:

Output Format: Text
Output Parameters: None

Software Integration:

Supported Hardware Platform(s): NVIDIA L4 GPUs
Supported Operating System(s): Linux

Inference:

Engine: Triton Inference Server
Test Hardware: NVIDIA L4 systems