Scaling Laws: Why Bigger Models Are Better (Usually)

OpenAI's 2020 scaling laws paper showed that model performance improves predictably with scale — more data, more compute, more parameters. This insight drove the race to build ever-larger models.