Login
From:
Ai2 Blog
(Uncensored)
subscribe
Revisiting critical batch size for large-batch OLMo pretraining
https://allenai.org/blog/critical-batch-size
links
backlinks
Roast topics
Find topics
Find it!
We introduce a more reliable method to measure the critical batch size (CBS), analyze how CBS changes over training, and use this to train OLMo with fewer grad steps.
