The year LLMs stopped getting bigger and started getting smarter.
In May 2025, the AI research community was still buzzing about ever-larger models, ever-bigger training runs, and the seemingly inexorable march toward AGI fueled by GPU clusters the size of data centers. By May 2026, the conversation had fundamentally shifted.
Not because scaling stopped working — but because the field realized that how you scale matters more than whether you scale. Over 58 papers surveyed across LLMs and foundation models, the story of this year is one of maturation: the low-hanging fruit of adding more compute is gone, and the new frontier is about precision, efficiency, theoretical grounding, and doing more with less.
Here’s what happened — and why it matters for anyone building with or thinking about AI.
—
The End of Naive Scaling
The year’s most conceptually important paper — the one most likely to be cited five years from now — is “LLMs as Noisy Channels: A Shannon Perspective on Model Capacity and Scaling Laws” (Ouyang et al., May 2026). The authors propose a Shannon Scaling Law: a unified framework that models LLM training as information transmission over a noisy channel.
Using the Shannon-Hartley theorem from information theory, they explain phenomena the old monotonic power laws couldn’t touch:
1. Catastrophic overtraining. The old view: more training = better, until you plateau. The new view: beyond an optimal compute-to-data ratio, the model exhausts the information in its training signal and begins memorizing noise. Performance actively degrades.
2. Quantization degradation. The old view: fewer bits = proportional quality loss. The new view: smaller models have less redundant channel capacity, so quantization noise eats a larger fraction of their effective bandwidth. This explains why quantizing a 7B model hurts far more than quantizing a 70B model — it’s not proportional.
3. Optimal compute-to-data ratios. The Shannon framework predicts specific trade-offs between model size, training tokens, and compute budget — explaining why top-performing labs independently converge on similar training recipes.
This is significant because it moves scaling theory from purely empirical curve-fitting to something grounded in first principles. For practitioners, it means we finally have a theoretical vocabulary for predicting training dynamics. Should you train a 7B model for more tokens or a 13B model for fewer? The Shannon framework gives you a principled way to reason about that trade-off, rather than relying on heuristics and gut instinct.
Complementing this, “Model Collapse as Cultural Evolution” (Guo et al., May 2026) brought iterated learning theory from cultural evolution to bear on synthetic-data degradation. Rather than treating collapse as a statistical inevitability, the authors derived five falsifiable predictions about which linguistic structures degrade first — rare grammatical constructions collapse before common ones — and confirmed them across two models and three languages. The implication: model collapse isn’t a monolithic doom. It’s a predictable, language-specific phenomenon we can learn to manage with targeted mitigation.
—
Inference Efficiency Becomes First-Class
If 2023-2024 was about training bigger models, 2025-2026 was about deploying them smarter. The practical challenge of serving LLMs at scale — managing memory, latency, and cost without sacrificing quality — drove significant innovation across multiple fronts.
KV Cache Compression saw a genuine advance with Meta-Soft (Luo et al., May 2026). Previous static methods like Judge Q used the same compression strategy for every input regardless of information density. Meta-Soft introduced dynamic, composable meta-tokens that adapt their compression per prompt. For long-context applications — document analysis, codebase understanding, multi-turn conversations — this directly addresses the memory blow-up that makes deployment prohibitively expensive at scale. The context-preserving property ensures that compression doesn’t discard information that becomes relevant later in generation.
ModeSwitch-LLM (Sunesh et al., May 2026) took a system-level approach: route each request to the optimal inference mode — FP16 for quality-sensitive tasks, quantization for high-throughput scenarios, speculative decoding for latency-critical applications, or hybrid for balanced workloads — based on cheap workload-level features like input length, desired output length, and latency budget.
The deeper insight here is profound: serving a single model well in production requires orchestrating multiple inference strategies, not optimizing one. The best FP16 implementation in the world still wastes resources if you don’t know when to switch to quantization. A simple factual lookup about the weather doesn’t need the same computational budget as a complex reasoning problem, and ModeSwitch-LLM learns to distinguish these cases from lightweight features.
Perhaps the cleverest paper in this category: Training-Free Looped Transformers (Chen et al., May 2026). The authors retrofitted recurrence onto pretrained models at inference time by looping a contiguous mid-stack block of layers. No fine-tuning, no continued training, no architectural changes — just a lightweight wrapper that feeds the output of a middle block back into itself for additional passes.
The key finding is non-trivial: naive block reapplication — just running the same layers again — degrades performance because layers aren’t designed for iterative use. But with the right choice of loop point and wrapper, the model effectively gets more “thinking time” when the input demands it. This opens the door to deeper reasoning at test time without the cost of training looped architectures end-to-end, and it works on any existing model checkpoint today.
Benchmarking Google Embeddings 2 (Cirillo et al., May 2026) provided a practical reference point for the retrieval side of the LLM stack. GE2 ranked first on every task across BEIR subsets and RAG corpora, but the gap with open-source alternatives (BGE-M3, E5-large) was narrow enough to make them viable for cost-sensitive deployments.
—
The Self-Evolving Agent Revolution
Perhaps the most vibrant sub-area of the year was the emergence of sophisticated skill management systems for LLM agents. This isn’t about making agents that can write code or browse the web — that’s old news. It’s about making agents that can improve themselves systematically, acquiring and refining capabilities at inference time without weight updates.
Three papers form a coherent arc from diagnosis to solution:
“From Raw Experience to Skill Consumption” (Huang et al., May 2026) provided the first comprehensive study spanning the full pipeline — extracting skills from raw experience, storing them in a skill library, retrieving them at inference time, and consuming them effectively. The key finding: domain-level, model-generated skills are finally becoming viable at scale. This marks a turning point for the agent paradigm, suggesting the long-promised vision of agents that improve with experience is within reach.
“Ratchet: A Minimal Hygiene Recipe for Self-Evolving LLM Agents” (Zhang et al., May 2026) delivered a wake-up call. Across multiple benchmarks and agent architectures, LLM-authored skills delivered zero improvement over no-skill baselines. Human-curated skills, meanwhile, delivered +16.2 percentage points.
The bottleneck isn’t skill authoring — LLMs write perfectly good skills. The bottleneck is lifecycle management: writing, retrieving, curating, evaluating, and retiring skills systematically over time. Without a disciplined lifecycle, skill pools accumulate noise, redundancy, and outdated information that actively degrades performance.
Ratchet’s key insight: frozen LLM agents can accumulate reusable knowledge without weight updates — provided they have the right hygiene. This is a design principle for anyone building agentic systems: skill management infrastructure matters as much as skill generation.
“SkillOpt: Executive Strategy for Self-Evolving Agent Skills” (Yang et al., May 2026) went further, arguing that agent skills should be treated as trainable external state — optimized with the same discipline as weight-space optimization. Just as you have learning rate schedules and gradient clipping for neural network training, SkillOpt proposes analogous mechanisms: skill pruning (removing skills that hurt performance), skill merging (combining related skills), and skill scheduling (deciding when to use which skill). This reframes skill acquisition from ad-hoc self-revision to something more like a deep-learning optimizer operating in text space.
But the most provocative paper in this area may be “Compiling Agentic Workflows into LLM Weights” (Dennis et al., May 2026). Rather than managing skills externally in a text-based library, the authors showed that agentic procedures — sequences of steps, tool calls, and decision points — can be compiled directly into LLM weights. Results: near-frontier quality on procedural tasks at roughly 1% of the compute cost of running an external orchestrator alongside a frontier model.
This challenges the entire LangGraph/CrewAI/AutoGen orchestration paradigm by suggesting that for many tasks, the external orchestrator is unnecessary overhead — the procedure belongs in the weights, not in an external controller. If confirmed at scale, this could reshape the architecture of every agentic system.
—
Where Bias Really Comes From
“It’s the humans, not the data” (Bladon & Bent, May 2026) delivered one of the year’s most counterintuitive and practically significant results. Testing seven open-weight LLM pairs (base model vs. chat-tuned version) across 28 country pairs in English, French, and Chinese, they found that geopolitical bias emerges almost entirely in post-training — RLHF, instruction tuning, and safety alignment — not in pre-training data.
The base models showed relatively balanced geopolitical responses. The chat-tuned versions showed systematic Western-aligned favoritism. This shifts the locus of responsibility for bias mitigation from data curation (where most current effort is focused) to alignment methodology. If you want to reduce geopolitical bias in your deployed model, cleaning the pre-training data is the wrong approach — you need to examine your RLHF reward model, your instruction-tuning dataset, and your safety filters.
The language of the prompt amplified the bias: the same model showed different geopolitical leanings depending on whether the conversation was in English, French, or Chinese. This suggests that alignment doesn’t just add bias — it interacts with the model’s language-specific representations in complex ways we don’t yet understand.
“AMEL: Accumulated Message Effects on LLM Judgments” (Temkit, May 2026) revealed a systematic bias in LLM-as-evaluator setups. Across 75,898 API calls to 11 models from 4 providers (OpenAI, Anthropic, Google, Meta), the polarity of prior conversation context — whether previous messages were positive, negative, or neutral — consistently biased subsequent judgments. A model evaluating code quality rated code more harshly if the preceding conversation had been critical, and more generously if it had been praising.
For a field increasingly reliant on LLM-based evaluation, this is a methodological warning that demands careful experimental design. Randomizing evaluation order isn’t enough — you need to control for accumulated context effects. If you’re using LLMs as judges for benchmarks or automated decisions, your results may be systematically biased by the evaluation history.
“Human Decision-Making with Persuasive and Narrative LLM Explanations” (Marusich et al., May 2026) explored the dual-edged nature of LLM explanations. On the positive side, providing LLM-generated explanations alongside predictions improved objective decision accuracy across multiple classification tasks. On the concerning side, the narrative persuasiveness of these explanations shaped human beliefs in ways that sometimes diverged from ground truth — participants found convincing-sounding explanations more persuasive than accurate ones. This tension between helpfulness and influence is the central human-AI interaction challenge of the era, with implications for medical diagnosis support, legal decision-making, and any domain where AI recommendations influence human judgment.
“Inferential Privacy Leakage in Anonymized Conversational AI Logs” (Zaman & Garimella, May 2026) grounded abstract privacy concerns in concrete evidence: 34.5% of ChatGPT user messages from 1,000+ users across four Global South countries contained personal information — names, addresses, phone numbers, financial details — even in conversations ostensibly about non-personal topics. Standard anonymization techniques (removing explicit identifiers, applying differential privacy) were insufficient to prevent re-identification through conversational inference.
“Prompt Overflow: What the Guardrail Inspects Is Not What the Model Infers” (Zhou et al., May 2026) exposed a fundamental architectural mismatch in current safety infrastructure. Guardrail models (classifiers that check inputs for harmful content) typically truncate or segment long prompts to fit within their context windows — but the underlying LLM sees the full, untruncated prompt. This creates an exploit surface entirely orthogonal to semantic prompt injection: an attacker can place harmful content in the part of the prompt the guardrail doesn’t see, but the model does.
—
Beyond English: Multilingual and Cross-Lingual
“Multilingual Knowledge Transfer under Data Constraints via Lexical Interventions” (Sedova et al., May 2026) addressed a pressing practical challenge: how to build LLMs for low-resource languages without requiring large amounts of target-language training data. The authors’ approach — intervening on lexical representations in high-resource languages — achieved effective cross-lingual transfer for scientific reasoning, commonsense inference, and world knowledge tasks. A model trained on English and Chinese could be adapted to Swahili or Nepali with minimal additional data, preserving most of its reasoning capabilities.
“As X, Do Y: How Persona and Task Combine in Instruction-Tuned LLMs” (Xu, May 2026) provided mechanistic understanding of role prompting — something practitioners have been doing intuitively for years. The paper showed that persona effects (“you are an expert physicist”) and task effects (“explain quantum entanglement”) have clean, partially orthogonal additive directions at a specific site in the residual stream: the prompt-to-answer transition in early/mid transformer layers. Knowing where in the network these effects operate tells you how to compose them effectively — and suggests that there are principled limits to how many distinct personas you can layer before they interfere.
—
Architectures: MoE, Optimizers, and Evaluation
“Complete-muE: Optimal Hyperparameter Transfer and Scaling for MoE Models” (Peng et al., May 2026) solved a practical problem that has been a thorn in the side of large-scale training: how to transfer hyperparameters from dense to Mixture-of-Experts architectures, and across different expert counts. The two-bridge system (Bridge-I for dense->MoE transfer, Bridge-II for MoE->larger MoE transfer) fills a gap that existing tools like uP and SDE couldn’t address, because MoE introduces architectural parameters (number of experts, routing sparsity, expert capacity) that have no analogue in dense models.
“Move on Muon: A Hamiltonian probability gradient flow perspective of Muon optimizer” (Mustafi et al., May 2026) provided theoretical grounding for the increasingly popular Muon optimizer. The authors showed that Muon can be understood as a mirror/prox step in the update variable under a regularized orthogonalization map. This kind of theoretical analysis helps the field move optimizer selection from folklore (“Muon seems to work well for this type of model”) to principle (“Muon is appropriate because its update dynamics align with the geometry of the parameter space”).
“Cost-Effective Model Evaluation with Meta-Learning” — MetaEvaluator (Pham et al., May 2026) — addressed the challenge of assessing newly released models on unlabeled data without expensive human annotation. As the model ecosystem explodes (new LLMs released weekly), traditional benchmarking can’t keep up. MetaEvaluator’s meta-learning framework transfers evaluation capabilities across model families, offering a way to estimate model quality on new tasks without running a full evaluation suite.
—
Looking Forward
The LLM research trajectory in May 2025-May 2026 shows a field that is maturing rapidly. The era of “just add more GPUs and more data” is decisively over. The new frontier is about doing more with less — better inference efficiency, smarter skill management, more targeted post-training, and theoretical foundations that actually explain non-monotonic phenomena.
The Shannon Scaling Law provides the theoretical scaffolding for the next generation of scaling research, offering a unified vocabulary for phenomena the old power laws couldn’t touch. The convergence of LLMs with agentic systems suggests that the “foundation model” concept is increasingly serving as a cognitive backbone for systems that go far beyond text.
The open question for the next year: whether the Shannon framework can guide not just understanding of existing models, but the design of fundamentally new ones. If it can, we may be looking at the first truly principled approach to building foundation models since the Transformer itself.
—
This article is part of the Frontier AI Research Digest backfill series, surveying 58 papers from May 2025 – May 2026. Views expressed are the author’s research synthesis, not affiliate endorsements.
Leave a Reply