Epistemic status: Pretty uncertain, this is a model I have been using to think about neural networks for a while, which does have some support, but is not completely rigorous. I hear a lot of people talk about scaling laws as if they are a property of specific models or...
