Train a Chat Model with NanoChat

Basic idea

nanochat is Andrej Karpathy’s educational ChatGPT-style training stack — popularized as the best ChatGPT that $100 can buy. This playbook walks you through training and chatting with your own model locally: tokenizer training, pretraining, midtraining / supervised fine-tuning (SFT), and inference via a simple web UI or CLI.

What you'll accomplish

• Build a Docker environment with PyTorch and nanochat dependencies on your hardware platform
• Run the full training pipeline (BPE tokenizer → base pretraining → chat fine-tuning → report)
• Chat with your trained checkpoints through a web UI or CLI

What to know before starting

Required:

• Basic Linux command line and shell usage
• Working with Docker containers and GPU passthrough
• Basic understanding of training foundation LLM models

Supported hardware platforms

Use the matrix below to confirm your hardware platform, OS, memory, and whether multi-node applies. This playbook covers single-node training on the platforms listed.

Prerequisites

Hardware requirements

• Supported hardware platform — see Supported hardware platforms matrix above
• Adequate storage for datasets and checkpoints (~50 GB free recommended)

Software requirements

• Docker installed: docker --version
• NVIDIA Container Toolkit configured
• Verify GPU access: nvidia-smi
• Check Docker GPU integration (use the default container tag for your hardware platform from the matrix above):
  Bash

  CopiedCopy

  docker run --rm --gpus all nvcr.io/nvidia/pytorch:<tag> nvidia-smi
  

• Weights & Biases account and API key (required by the launch scripts)
• Hugging Face token (for evaluation / gated datasets)
• Network access to NGC, Hugging Face, and FineWeb / ClimbMix data

Ancillary files

Assets live under assets/ in this playbook:

Upstream reference: nanochat on GitHub.

Time & risk

• Estimated time: Single-node setup is ~30 MIN; a full single-node d24 run is on the order of 12+ hours (mostly unattended compute).
• Risk level: Medium
  • Model training is memory-intensive; changing batch size, depth, or precision can cause OOM
  • Large dataset downloads and checkpoints need substantial disk space
  • Launch scripts exit if WANDB_API_KEY or HF_TOKEN are unset
• Rollback: Stop containers, then remove caches (~/.cache/nanochat or local nanochat_cache/) and the nanochat Docker image (non-destructive to the host OS)
• Last Updated: 07/31/2026
  • Removed DGX Spark support (not ready yet); playbook is Station / single-node only

Credits

• nanochat by Andrej Karpathy
• FineWeb / ClimbMix by Hugging Face (pretraining data)
• SmolTalk by Hugging Face (SFT data)