← Back to feed

Sub-30s LLM Pretraining on 8×H100? Modded-NanoGPT World Record Attempt (29.90 S)

AI · Technology · 2026-11-22 18:44

A developer achieved a sub-30-second pretraining time for a GPT-2 scale model on 8×H100 GPUs, clocking in at 29.90 seconds. This record-breaking attempt leverages six specific GPU optimizations to challenge existing benchmarks in the Modded-NanoGPT project.

Source: amund.blog


Auto-generated summary for reference only. RSS