Cutting-edge open multimodal model exceling in high-quality reasoning from images.
Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. Gemma 3 models are multimodal, handling text and image input and generating text output, with open weights for both pre-trained variants and instruction-tuned variants. Gemma 3 has a large, 128K context window, multilingual support in over 140 languages, and is available in more sizes than previous versions. Gemma 3 models are well-suited for a variety of text generation and image understanding tasks, including question answering, summarization, and reasoning. Their relatively small size makes it possible to deploy them in environments with limited resources such as laptops, desktops or your own cloud infrastructure, democratizing access to state of the art AI models and helping foster innovation for everyone. This model is ready for commercial use.
This model is not owned or developed by NVIDIA. This model has been developed and built to a third-party’s requirements for this application and use case; see link to Non-NVIDIA Gemma 3 model card.
GOVERNING TERMS: The trial service is governed by the NVIDIA API Trial Terms of Service; and the use of this model is governed by the NVIDIA Community Model License. ADDITIONAL INFORMATION: Gemma Terms of Use.
Global
Models are well-suited for a variety of text generation and image understanding tasks, including question answering, summarization, and reasoning.
At the time of release, this family of models provides high-performance open vision-language model implementations designed from the ground up for responsible AI development compared to similarly sized models.
Using the benchmark evaluation metrics described in this document, these models have shown to provide superior performance to other, comparably-sized open model alternatives.
Model Page: Gemma Authors: Google DeepMind
Architecture Type: Dense decoder-only Transformer model
Input Type(s): Text, Text+Image
Input Format(s):
Input Parameters:
Other Properties Related to Input:
Output Type(s): Text
Output Format: String
Output Parameters: (1D)
Other Properties Related to Output:
Runtime Engine(s): TRT-LLM
Supported Hardware Microarchitecture Compatibility: NVIDIA Ampere, NVIDIA Blackwell, NVIDIA Jetson, NVIDIA Hopper, NVIDIA Lovelace, NVIDIA Pascal, NVIDIA Turing, and NVIDIA Volta architectures
[Preferred/Supported] Operating System(s): Linux
Training was done using JAX and ML Pathways.
JAX allows researchers to take advantage of the latest generation of hardware, including TPUs, for faster and more efficient training of large models. ML Pathways is Google's latest effort to build artificially intelligent systems capable of generalizing across multiple tasks. This is especially suitable for foundation models, including large language models like these ones.
Together, JAX and ML Pathways are used as described in the paper about the Gemini family of models; "the 'single controller' programming model of Jax and Pathways allows a single Python process to orchestrate the entire training run, dramatically simplifying the development workflow."
Data Collection Method by dataset: Hybrid: Human, Automated
Labeling Method by dataset: Hybrid: Human, Automated
These models were trained on a dataset of text data that includes a wide variety of sources. The 27B model was trained with 14 trillion tokens, the 12B model was trained with 12 trillion tokens, 4B model was trained with 4 trillion tokens and 1B with 2 trillion tokens. Here are the key components:
The combination of these diverse data sources is crucial for training a powerful multimodal model that can handle a wide variety of different tasks and data formats.
Here are the key data cleaning and filtering methods applied to the training data:
Data Collection Method by dataset: Hybrid: Human, Automated
Labeling Method by dataset: Hybrid: Human, Automated
Data Collection Method by dataset: Hybrid: Human, Automated
Labeling Method by dataset: Hybrid: Human, Automated
Model evaluation metrics and results are highlighted below.
These models were evaluated against a large collection of different datasets and metrics to cover different aspects of text generation:
Benchmark | Metric | Gemma 3 PT 1B | Gemma 3 PT 4B | Gemma 3 PT 12B | Gemma 3 PT 27B |
---|---|---|---|---|---|
HellaSwag | 10-shot | 62.3 | 77.2 | 84.2 | 85.6 |
BoolQ | 0-shot | 63.2 | 72.3 | 78.8 | 82.4 |
PIQA | 0-shot | 73.8 | 79.6 | 81.8 | 83.3 |
SocialIQA | 0-shot | 48.9 | 51.9 | 53.4 | 54.9 |
TriviaQA | 5-shot | 39.8 | 65.8 | 78.2 | 85.5 |
Natural Questions | 5-shot | 9.48 | 20.0 | 31.4 | 36.1 |
ARC-c | 25-shot | 38.4 | 56.2 | 68.9 | 70.6 |
ARC-e | 0-shot | 73.0 | 82.4 | 88.3 | 89.0 |
WinoGrande | 5-shot | 58.2 | 64.7 | 74.3 | 78.8 |
BIG-Bench Hard | few-shot | 28.4 | 50.9 | 72.6 | 77.7 |
DROP | 1-shot | 42.4 | 60.1 | 72.2 | 77.2 |
Benchmark | Metric | Gemma 3 PT 4B | Gemma 3 PT 12B | Gemma 3 PT 27B |
---|---|---|---|---|
MMLU | 5-shot | 59.6 | 74.5 | 78.6 |
MMLU (Pro COT) | 5-shot | 29.2 | 45.3 | 52.2 |
AGIEval | 3-5-shot | 42.1 | 57.4 | 66.2 |
MATH | 4-shot | 24.2 | 43.3 | 50.0 |
GSM8K | 8-shot | 38.4 | 71.0 | 82.6 |
GPQA | 5-shot | 15.0 | 25.4 | 24.3 |
MBPP | 3-shot | 46.0 | 60.4 | 65.6 |
HumanEval | 0-shot | 36.0 | 45.7 | 48.8 |
Benchmark | Gemma 3 PT 1B | Gemma 3 PT 4B | Gemma 3 PT 12B | Gemma 3 PT 27B |
---|---|---|---|---|
MGSM | 2.04 | 34.7 | 64.3 | 74.3 |
Global-MMLU-Lite | 24.9 | 57.0 | 69.4 | 75.7 |
WMT24++ (ChrF) | 36.7 | 48.4 | 53.9 | 55.7 |
FloRes | 29.5 | 39.2 | 46.0 | 48.8 |
XQuAD (all) | 43.9 | 68.0 | 74.5 | 76.8 |
ECLeKTic | 4.69 | 11.0 | 17.2 | 24.4 |
IndicGenBench | 41.4 | 57.2 | 61.7 | 63.4 |
Benchmark | Gemma 3 PT 4B | Gemma 3 PT 12B | Gemma 3 PT 27B |
---|---|---|---|
COCOcap | 102 | 111 | 116 |
DocVQA (val) | 72.8 | 82.3 | 85.6 |
InfoVQA (val) | 44.1 | 54.8 | 59.4 |
MMMU (pt) | 39.2 | 50.3 | 56.1 |
TextVQA (val) | 58.9 | 66.5 | 68.6 |
RealWorldQA | 45.5 | 52.2 | 53.9 |
ReMI | 27.3 | 38.5 | 44.8 |
AI2D | 63.2 | 75.2 | 79.0 |
ChartQA | 63.6 | 74.7 | 76.3 |
VQAv2 | 63.9 | 71.2 | 72.9 |
BLINK | 38.0 | 35.9 | 39.6 |
OKVQA | 51.0 | 58.7 | 60.2 |
TallyQA | 42.5 | 51.8 | 54.3 |
SpatialSense VQA | 50.9 | 60.0 | 59.4 |
CountBenchQA | 26.1 | 17.8 | 68.0 |
Engine: Transformers
Test Hardware: NVIDIA Hopper
Ethics and safety evaluation approach and results are highlighted below.
The evaluation method included structured evaluations and internal red-teaming testing of relevant content policies. Red-teaming was conducted by a number of different teams, each with different goals and human evaluation metrics. These models were evaluated against a number of different categories relevant to ethics and safety, including:
In addition to development level evaluations, assurance evaluations were conducted using the "arms-length" internal evaluations for responsibility governance decision making. They are conducted separately from the model development team, to inform decision making about release. High level findings are fed back to the model team, but prompt sets are held-out to prevent overfitting and preserve the results' ability to inform decision making. Assurance evaluation results are reported to the Responsibility & Safety Council as part of release review.
For all areas of safety testing, there were major improvements in the categories of child safety, content safety, and representational harms relative to previous Gemma models. All testing was conducted without safety filters to evaluate the model capabilities and behaviors. For both text-to-text and image-to-text, and across all model sizes, the model produced minimal policy violations, and showed significant improvements over previous Gemma models' performance with respect to ungrounded inferences. One limitation of the evaluation was that the models incorporated only English language prompts.
The potential limitations for these models are outlined below.
Open vision-language models (VLMs) models have a wide range of applications across various industries and domains. The following list of potential uses is not comprehensive. The purpose of this list is to provide contextual information about the possible use-cases that the model creators considered as part of model training and development.
NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
Please report security vulnerabilities or NVIDIA AI Concerns here.