---
title: "Fine-tune with Pytorch"
publisher: "nvidia"
type: "playbook"
updated: "2025-10-07T20:33:57.550Z"
description: "Use Pytorch to fine-tune models locally"
canonical: "https://build.nvidia.com/spark/pytorch-fine-tune.md"
---

# Basic idea

This playbook guides you through setting up and using Pytorch for fine-tuning large language models on NVIDIA Spark devices.

# What you'll accomplish

You'll establish a complete fine-tuning environment for large language models (1-70B parameters) on your NVIDIA Spark device. 
By the end, you'll have a working installation that supports parameter-efficient fine-tuning (PEFT) and supervised fine-tuning (SFT).

# What to know before starting

- Previous experience with fine-tuning in Pytorch
- Working with Docker

# Prerequisites
Recipes are specifically for DGX SPARK. Please make sure that OS and drivers are latest.

# Ancillary files

ALl files required for fine-tuning are included in the folder in [the GitHub repository here](https://github.com/NVIDIA/dgx-spark-playbooks/blob/main/nvidia/pytorch-fine-tune).

# Time & risk

* **Time estimate:** 30-45 mins for setup and runing fine-tuning. Fine-tuning run time varies depending on model size 
* **Risks:** Model downloads can be large (several GB), ARM64 package compatibility issues may require troubleshooting.
* **Last Updated:** 01/15/2025
* Add two-Spark distributed finetuning example
* Add detailed instructions to run full SFT, LoRA and qLoRA workflows on Llama3 3B, 8B and 70B models.

## More

- [Instructions](/spark/pytorch-fine-tune/instructions.md)
- [Run on two Sparks](/spark/pytorch-fine-tune/run-two-sparks.md)
- [Troubleshooting](/spark/pytorch-fine-tune/troubleshooting.md)