Category: LLMs & Foundation Models
-
Week 22, 2026 — LLM Training & Scaling Laws
This week brought transformative advances in understanding how large language models scale and train — from a unified theory of scaling failures to practical recipes for MoE hyperparameter transfer and data mixture auditing. Shannon Scaling Law Unifies Training Phenomena Xu Ouyang and colleagues proposed the Shannon Scaling Law, treating LLM training as information transmission over…
-
Why the LLM Scaling Era Is Over — and What Comes Next
The year LLMs stopped getting bigger and started getting smarter. In May 2025, the AI research community was still buzzing about ever-larger models, ever-bigger training runs, and the seemingly inexorable march toward AGI fueled by GPU clusters the size of data centers. By May 2026, the conversation had fundamentally shifted. Not because scaling stopped working…