Scaling

Scaling laws are the closest thing the field has to a design equation: given a compute budget, they say how large a model to train and how much data to train it on. They are empirical, they have been revised, and they are still the first calculation anyone does.