⌘KCtrl+K
Distilled version of Qwen 2.5 7B using reasoning data generated by DeepSeek R1 for enhanced performance.
The following are key system requirements and supported features to consider when self-hosting the deepseek-r1-distill-qwen-7b model.
| Precision | Minimum GPU Memory | Recommended GPU Memory |
|---|---|---|
| fp8 | 7 GB | 9 GB |
| bf16 | 15 GB | 17 GB |
Deploying this NIM with less than the recommended amount of GPU memory requires setting the environment variable NIM_RELAX_MEM_CONSTRAINTS=1
| Feature | Supported |
|---|---|
| LoRA Customization | ❌ |
| Fine-tuning Customization | ✅ |
| Tool Calling | ❌ |
| TensorRT-LLM Local Engine Building | ✅ |