Reduced memory use and minimal boilerplate for open models
NOTE
These instructions target Linux with a GPU-enabled PyTorch container. Run the install and validation steps inside the container unless noted otherwise.
Confirm the CUDA toolkit and GPU resources on your hardware platform.
nvcc --version
Expected output should show CUDA 13.0.
nvidia-smi
Expected output should show a summary of GPU information.
docker pull nvcr.io/nvidia/pytorch:25.11-py3
docker run --gpus all --ulimit memlock=-1 -it --ulimit stack=67108864 --entrypoint /usr/bin/bash --rm nvcr.io/nvidia/pytorch:25.11-py3
You are now in an interactive shell inside the container for the remaining install and validation steps.
Inside the container:
pip install transformers peft hf_transfer "datasets==4.3.0" "trl==0.26.1"
pip install --no-deps unsloth unsloth_zoo bitsandbytes
Download the test script into the container. The file is also listed under Ancillary files in the Overview tab.
curl -O https://raw.githubusercontent.com/NVIDIA/dgx-spark-playbooks/refs/heads/main/nvidia/playbook-unsloth/assets/test_unsloth.py
This script runs a short fine-tuning job to validate the Unsloth install.
python test_unsloth.py
Expected output in the terminal:
Adapt test_unsloth.py for your own model and dataset:
# Replace the model_name argument in FastModel.from_pretrained with your choice
model_name = "unsloth/Meta-Llama-3.1-8B-bnb-4bit"
# Load your custom dataset (see the dataset load near the top of the script)
dataset = load_dataset("your_dataset_name")
# Adjust training parameters in SFTConfig (for example)
per_device_train_batch_size = 4
max_steps = 1000
For advanced usage, see the Unsloth wiki, including:
When you exit the interactive container (exit or Ctrl-D), the --rm flag removes it. Pulled images remain on the host until you remove them:
docker rmi nvcr.io/nvidia/pytorch:25.11-py3
WARNING
Removing the image deletes the local copy of nvcr.io/nvidia/pytorch:25.11-py3. Re-pull it if you run this playbook again.