Kimi-K2-Instruct

Description

Kimi K2 Instruct is a state-of-the-art mixture-of-experts (MoE) language model with 32 billion activated parameters and 1 trillion total parameters. Trained with the Muon optimizer, Kimi K2 achieves exceptional performance across frontier knowledge, reasoning, and coding tasks while being meticulously optimized for agentic capabilities.

This model is ready for commercial/non-commercial use.

Third-Party Community Consideration:

This model is not owned or developed by NVIDIA. This model has been developed and built to a third-party's requirements for this application and use case; see link to Non-NVIDIA Kimi-K2-Instruct Model Card

License and Terms of Use:

GOVERNING TERMS: This trial service is governed by the NVIDIA API Trial Terms of Service. Use of this model is governed by the NVIDIA Community Model License. Additional Information: Modified MIT License.

Deployment Geography:

Global

Use Case:

This model is designed for agentic AI and tool use, including advanced code generation, complex problem-solving, and multilingual applications. It can be used for building autonomous agents that can interact with external systems and APIs, for multi-step reasoning tasks, mathematical problem-solving, and analytical workflows.

Key Features

• Large-Scale Training: Pre-trained a 1T parameter MoE model on 15.5T tokens with zero training instability.
• MuonClip Optimizer: We apply the Muon optimizer to an unprecedented scale, and develop novel optimization techniques to resolve instabilities while scaling up.
• Agentic Intelligence: Specifically designed for tool use, reasoning, and autonomous problem-solving.

Release Date:

Build.NVIDIA.com 07/2025 via link
Huggingface 07/12/2025 via link

Reference(s):

References:

• moonshotai/Kimi-K2-Instruct - Simon Willison's Weblog
• Kimi-k2 : The best Open-Sourced AI model with 1 Trillion params | by Mehul Gupta - Medium
• How to Run Kimi K2 Inside Claude Code: The Ultimate Open-Source AI Coding Combo

Model Architecture:

Architecture Type: Transformer

• Network Architecture: Mixture-of-Experts (MoE)
  
• Total Parameters: 1T
  
• Active Parameters: 32B
  
• Number of Layers (Dense layer included): 61
  
• Attention Hidden Dimension: 7168
  
• MoE Hidden Dimension (per Expert): 2048
  
• Number of Attention Heads: 64
  
• Number of Experts: 384
  
• Selected Experts per Token: 8
  
• Number of Shared Experts: 1
  
• Vocabulary Size: 160K
  
• Context Length: 128K
  
• Attention Mechanism: MLA
  
• Activation Function: SwiGLU
  
• Base Model: Kimi-K2-Base
  

Input:

Input Types: Text
Input Formats: String
Input Parameters: One Dimensional (1D)
Other Input Properties: The model has a context window of up to 128,000 tokens.
Input Context Length (ISL): 128K

Output:

Output Format: String
Output Parameters: One Dimensional (1D)
Other Output Properties: Not applicable.

Our AI models are designed and/or optimized to run on NVIDIA GPU-accelerated systems. By leveraging NVIDIA's hardware (e.g. GPU cores) and software frameworks (e.g., CUDA libraries), the model achieves faster training and inference times compared to CPU-only solutions.

Software Integration:

Runtime Engines:

• NVIDIA NeMo
  
• NVIDIA Riva
  

Supported Hardware:

• NVIDIA Blackwell: B100, B200, GB200
  
• NVIDIA Hopper: H100, H200
  

Operating Systems: Linux

Model Version(s):

Kimi-K2-Instruct v1.0

Training, Testing, and Evaluation Datasets:

Training Dataset

Training Data Collection: Undisclosed
Training Labeling: Undisclosed
Training Properties: Trained on 15.5 trillion tokens.

Testing Dataset

Testing Data Collection: Undisclosed
Testing Labeling: Undisclosed
Testing Properties: Undisclosed

Evaluation Dataset

Evaluation Benchmark Score:

• LiveCodeBench: 53.7% Pass@1
• SWE-bench Verified: 65.8% single-attempt accuracy
• MMLU: 89.5% exact match
• Tau2 retail tasks: 70.6% Avg@4
  

Evaluation Data Collection: Undisclosed
Evaluation Labeling: Undisclosed
Evaluation Properties: LiveCodeBench, SWE-bench, MMLU, Tau2

Evaluation Results

• Bold denotes global SOTA, and underlined denotes open-source SOTA.
• Data points marked with * are taken directly from the model's tech report or blog.
• All metrics, except for SWE-bench Verified (Agentless), are evaluated with an 8k output token length. SWE-bench Verified (Agentless) is limited to a 16k output token length.
• Kimi K2 achieves 65.8% pass@1 on the SWE-bench Verified tests with bash/editor tools (single-attempt patches, no test-time compute). It also achieves a 47.3% pass@1 on the SWE-bench Multilingual tests under the same conditions. Additionally, we report results on SWE-bench Verified tests (71.6%) that leverage parallel test-time compute by sampling multiple sequences and selecting the single best via an internal scoring model.
• To ensure the stability of the evaluation, we employed avg@k on the AIME, HMMT, CNMO, PolyMath-en, GPQA-Diamond, EvalPlus, Tau2.
• Some data points have been omitted due to prohibitively expensive evaluation costs.

Inference

Acceleration Engine: vLLM
Test Hardware: NVIDIA DGX B200

Additional Details

Deployment

You can access Kimi K2's API on https://platform.moonshot.ai , we provide OpenAI/Anthropic-compatible API for you.

The Anthropic-compatible API maps temperature by real_temperature = request_temperature * 0.6 for better compatible with existing applications.

Our model checkpoints are stored in the block-fp8 format, you can find it on Huggingface.

Currently, Kimi-K2 is recommended to run on the following inference engines:

• vLLM
• SGLang
• KTransformers
• TensorRT-LLM

Deployment examples for vLLM and SGLang can be found in the Model Deployment Guide.

Model Usage

Once the local inference service is up, you can interact with it through the chat endpoint:

def simple_chat(client: OpenAI, model_name: str):
    messages = [
        {"role": "system", "content": "You are Kimi, an AI assistant created by Moonshot AI."},
        {"role": "user", "content": [{"type": "text", "text": "Please give a brief self-introduction."}]},
    ]
    response = client.chat.completions.create(
        model=model_name,
        messages=messages,
        stream=False,
        temperature=0.6,
        max_tokens=256
    )
    print(response.choices[0].message.content)

Tool Calling

Kimi-K2-Instruct has strong tool-calling capabilities. To enable them, you need to pass the list of available tools in each request, then the model will autonomously decide when and how to invoke them.

The following example demonstrates calling a weather tool end-to-end:

# Your tool implementation
def get_weather(city: str) -> dict:
    return {"weather": "Sunny"}

# Tool schema definition
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Retrieve current weather information. Call this when the user asks about the weather.",
        "parameters": {
            "type": "object",
            "required": ["city"],
            "properties": {
                "city": {
                    "type": "string",
                    "description": "Name of the city"
                }
            }
        }
    }
}]

# Map tool names to their implementations
tool_map = {
    "get_weather": get_weather
}

def tool_call_with_client(client: OpenAI, model_name: str):
    messages = [
        {"role": "system", "content": "You are Kimi, an AI assistant created by Moonshot AI."},
        {"role": "user", "content": "What's the weather like in Beijing today? Use the tool to check."}
    ]
    finish_reason = None
    while finish_reason is None or finish_reason == "tool_calls":
        completion = client.chat.completions.create(
            model=model_name,
            messages=messages,
            temperature=0.6,
            tools=tools,          # tool list defined above
            tool_choice="auto"
        )
        choice = completion.choices[0]
        finish_reason = choice.finish_reason
        if finish_reason == "tool_calls":
            messages.append(choice.message)
            for tool_call in choice.message.tool_calls:
                tool_call_name = tool_call.function.name
                tool_call_arguments = json.loads(tool_call.function.arguments)
                tool_function = tool_map[tool_call_name]
                tool_result = tool_function(**tool_call_arguments)
                print("tool_result:", tool_result)

                messages.append({
                    "role": "tool",
                    "tool_call_id": tool_call.id,
                    "name": tool_call_name,
                    "content": json.dumps(tool_result)
                })
    print("-" * 100)
    print(choice.message.content)

The tool_call_with_client function implements the pipeline from user query to tool execution. This pipeline requires the inference engine to support Kimi-K2’s native tool-parsing logic. For streaming output and manual tool-parsing, see the Tool Calling Guide.

Ethical Considerations

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

Please report security vulnerabilities or NVIDIA AI Concerns here