Show HN: Llama 3.1 70B Runs on Single RTX 3090 via NVMe

In an impressive feat of optimization, a developer has demonstrated running the massive Llama 3.1 70 billion parameter language model on a single consumer-grade RTX 3090 GPU. This achievement, which would typically require multiple high-end GPUs or cloud instances, was made possible by a technique that streams model weights directly from fast NVMe solid-state storage, bypassing the CPU's RAM entirely. The method cleverly works around the GPU's limited VRAM (24GB on the RTX 3090) by loading part