Login
From:
Ai2 Blog
(Uncensored)
subscribe
Revisiting critical batch size for large-batch OLMo pretraining
https://allenai.org/blog/critical-batch-size
links
backlinks
We introduce a more reliable method to measure the critical batch size (CBS), analyze how CBS changes over training, and use this to train OLMo with fewer grad steps.
Roast topics
Find topics
Roast it!
Roast topics
Find topics
Find it!
Roast topics
Find topics
Find it!
