← Back to feed

Dust: Pretraining Transformers Without Backpropagation

AI · Technology · 2026-10-05 21:15

Researchers propose Dust, the first zeroth-order optimization method that can compete with backpropagation during Transformer pretraining while reducing computational dependencies.

Source: qlabs.sh


Auto-generated summary for reference only. RSS