nanochat is Andrej Karpathy’s educational ChatGPT-style training stack — popularized as the best ChatGPT that $100 can buy. This playbook walks you through training and chatting with your own model locally: tokenizer training, pretraining, midtraining / supervised fine-tuning (SFT), and inference via a simple web UI or CLI.
What you'll accomplish
Build a Docker environment with PyTorch and nanochat dependencies on your hardware platform
Run the full training pipeline (BPE tokenizer → base pretraining → chat fine-tuning → report)
Chat with your trained checkpoints through a web UI or CLI
What to know before starting
Required:
Basic Linux command line and shell usage
Working with Docker containers and GPU passthrough
Basic understanding of training foundation LLM models
Supported hardware platforms
Use the matrix below to confirm your hardware platform, OS, memory, and whether multi-node applies. This playbook covers single-node training on the platforms listed.
Hardware platform
OS
Memory
Multi-node capable hardware
DGX Station
DGX OS (Linux)
~288 GB HBM
—
NOTE
Only platforms listed in the Supported hardware platforms table above are covered by this playbook.
Prerequisites
Hardware requirements
Supported hardware platform — see Supported hardware platforms matrix above
Adequate storage for datasets and checkpoints (~50 GB free recommended)
Software requirements
Docker installed: docker --version
NVIDIA Container Toolkit configured
Verify GPU access: nvidia-smi
Check Docker GPU integration (use the default container tag for your hardware platform from the matrix above):
docker run --rm--gpus all nvcr.io/nvidia/pytorch:<tag> nvidia-smi
Weights & Biases account and API key (required by the launch scripts)
Hugging Face token (for evaluation / gated datasets)
Network access to NGC, Hugging Face, and FineWeb / ClimbMix data
Ancillary files
Assets live under assets/ in this playbook:
File
Purpose
Dockerfile
Single-node image (PyTorch NGC + nanochat dependencies)
Estimated time: Single-node setup is ~30 MIN; a full single-node d24 run is on the order of 12+ hours (mostly unattended compute).
Risk level: Medium
Model training is memory-intensive; changing batch size, depth, or precision can cause OOM
Large dataset downloads and checkpoints need substantial disk space
Launch scripts exit if WANDB_API_KEY or HF_TOKEN are unset
Rollback: Stop containers, then remove caches (~/.cache/nanochat or local nanochat_cache/) and the nanochat Docker image (non-destructive to the host OS)
Last Updated: 07/31/2026
Removed DGX Spark support (not ready yet); playbook is Station / single-node only
