A focused look at this week’s most significant advances in scaling, efficiency & architectures — 27 papers surveyed from arXiv and leading AI labs.
—
Efficiency remains the quiet revolution in AI research. This week brings advances in attention mechanisms, model compression, speculative decoding, and the mathematics behind training dynamics.
Key Developments
Blackwell Approachability and Gradient Equilibrium are Equivalent — Brian W. Lee, Nika Haghtalab, Michael I. Jordan, Ryan J. Tibshirani
Gradient equilibrium (GEQ) is a recently introduced online optimization framework that generalizes first-order stationarity from offline optimization and abstracts problems like online conformal predi…
Hierarchical Muon: Tiled Newton-Schulz Updates for Efficient Muon Optimization — Ziyuan Tang, Tianshi Xu, Yousef Saad, Yuanzhe Xi
Muon-type optimizers construct update directions for dense neural-network weights by applying a finite Newton-Schulz map to momentum-gradient matrices. For an $H \times W$ matrix, with $r=\min\{H,W\}$…
Don’t Settle at the Mode! Mitigating Diversity Collapse in Pretrained Flow Models via Feature Self-Guidance — Pradhaan S Bhat, Rishubh Parihar, Abhijnya Bhat, R. Venkatesh Babu
State-of-the-art flow models generate stunning images from text or image prompts. However, they suffer from diversity collapse when generating multiple samples under the same conditioning. Existing me…
Training & Scaling
MinGram: A Minimalist Unigram Tokenizer with High Compression and Competitive Morphological Alignment
Sander Land
The Unigram tokenizer uses an elegant representation which makes it straightforward to edit vocabularies, but its training is comparatively heavy and complex. We introduce MinGram (Minimalist Unigram), which keeps the token-list representation but…
CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMs
Shigeng Wang, Chao Li, Yangyuxuan Kang, Jiawei Fan, Anbang Yao
In this paper, we present CAT-Q, Cost-efficient and Accurate Ternary Quantization, for compressing and accelerating LLMs. Unlike existing state-of-the-art ternary quantization methods that rely on data-intensive and costly quantization-aware…
ViQ: Text-Aligned Visual Quantized Representations at Any Resolution
Xumin Yu, Zuyan Liu, Zhenyu Yang, Yuhao Dong, Shengsheng Qian, Jiwen Lu, Han Hu, Yongming Rao
A unified representation for text and vision is a natural pursuit, as it enables simpler multimodal modeling and more efficient training. However, representing images as discrete signals in the same way as text inevitably introduces severe…
Agent Systems
Minimax PAC Bounds for Learning in Exogenous Contextual MDPs
Corentin Pla, Hugo Richard, Marc Abeille, Vianney Perchet
We study PAC learning in tabular discounted Markov decision processes with exogenous i.i.d. contexts, with discount factor $γ$, finite state space $\mathcal X$, action space $\mathcal A$, and context space $\mathcal Z$. At each time step, a context…
Additional Research
Self-Modulating Quantum Fast-Weight Programmers for Efficient Adaptive Sequential Learning
Samuel Yen-Chi Chen, Yifeng Peng, Kuo-Chung Peng, Jiun-Cheng Jiang, Chun-Hua Lin, Junghoon Justin Pa
Recent advances in quantum machine learning have motivated efficient models for sequential data processing. In this paper, we propose Self-Modulating Quantum Fast Weight Programmers, or Self-Modulating QFWP, which extends Quantum Fast Weight…
Fast algorithms for learning a Gaussian under halfspace truncation with optimal sample complexity
Haitong Liu, Deepak Narayanan Sridharan, David Steurer, Manuel Wiedmer
We study the fundamental problem of learning a high-dimensional Gaussian truncated to an unknown halfspace. Lee, Mehrotra and Zampetakis (FOCS’24) recently obtained the first polynomial time algorithm for this problem, but their resulting sample…
Stochastic Gradient Optimization with Model-Assisted Sampling
Jonne Pohjankukka, Jukka Heikkonen
This work addresses the problem of variance in stochastic gradient estimation for machine learning optimization. Deep learning relies on mini-batch methods such as stochastic gradient descent, which approximate full gradients but introduce noise,…
Kolmogorov Arnold networks (KAN) for aerodynamic prediction: a comparison with MLPs and GNNs
Miguel Jaraiz, Fermin Gutierrez, Pablo Yeste, Miguel Sánchez-Domínguez, Eusebio Valero, Gonzalo Rubi
Kolmogorov Arnold networks (KAN) have recently been introduced as a (deep) neural network architecture whose trainable parameters adapt the activation functions, instead of the coefficients of the affine transformations at the core of traditional…
Compositionality and the lexicon in evolutionary semantics
Fausto Carcassi
Formal semantics has shown that sentence meanings arise by recursively composing lexical meanings, yet much of the literature on semantic universals models either lexicons with fixed signal structures or holistic composition without interpretable…
Jailbreaking for the Average Jane: Choosing Optimal Jailbreaks via Bandit Algorithms for Automatically Enhanced Queries
Prarabdh Shukla, Ritik, Suhas Rao, Arpit Agarwal, Arjun Bhagoji
With a profusion of jailbreaks for LLMs now widely known, a growing concern is that non-expert malicious actors (“the average Jane”) could elicit actionable responses to malicious requests. In this work, we examine whether this concern is…
HyperDFlash: MHC-Aligned Block Speculative Decoding with Gated Residual Reduction
Luxi Lin, Shuang Peng, Rui Ma, Junhao Hua, Shuwei Fan, Zhengda Qin, Qiang Wang, Hongjian Sun, Fangmi
We present HyperDFlash, a block-parallel speculative decoding framework tailored to the novel multi-hyper-connection (MHC) architecture proposed by DeepSeek-V4. Despite the strong initial-token drafting performance of the native Multi-Token…
TMP: Tree-structured Mixed-policy Pruning for Large-scale Image Generation and Editing
Peizhen Zhang, Yang Li, Xunsong Li, Songtao Liu, Zewen Liu, Qiangqiang Hu, Guotong Guo, Jupeng Ding,
Modern image generation model rapidly grows their sizes to meet high-fidelity image synthesis. However, they gradually become unaffordable for their enormous parameter consumption and computation budget that lead to massive resources requirement…
Explainable Outlier Detection for Interval-valued Data
Catarina P. Loureiro, M. Rosário Oliveira, Paula Brito, Lina Oliveira
Explainability is increasingly recognized as a key aspect of outlier detection. However, for complex data structures such as interval-valued data, it remains largely unexplored. Building on an outlier detection framework based on the Interval…
MiniMax-01: Scaling Foundation Models with Lightning Attention
MiniMax, Aonian Li, Bangwei Gong, Bo Yang, Boji Shan, Chang Liu, Cheng Zhu, Chunhao Zhang, Congchao
We introduce MiniMax-01 series, including MiniMax-Text-01 and MiniMax-VL-01, which are comparable to top-tier models while offering superior capabilities in processing longer contexts. The core lies in lightning attention and its efficient scaling….
HL-index: Fast Reachability Query in Hypergraphs
Peiting Xie, Xiangjun Zai, Yanping Wu, Xiaoyang Wang, Wenjie Zhang, Lu Qin
Reachability in hypergraphs is essential for modeling complex groupwise interactions in real-world applications such as co-authorship, social network, and biological analysis, where relationships go beyond pairwise interactions. In this paper, we…
Symplectic Neural Networks for learning Generalized Hamiltonians
Harsh Choudhary, Vyacheslav Kungurtsev, Chandan Gupta, Melvin Leok, Georgios Korpas
Hamiltonian Neural Networks (HNNs) integrate physical priors into neural models by learning a system’s Hamiltonian, improving generalization and sample efficiency. Identifying the system Hamiltonian from noisy observations of state variables is a…
Not All Actions Are Equal: Rethinking Conditioning for Dexterous World Model
Zizhao Yuan, Zhengtu Liang, Taowen Wang, Qiwei Liang, Yichi Wang, Yunheng Wang, Yuetong Fang, Lusong
Recent advances in action-conditioned world models show promising progress in modeling complex interactions and forecasting future states under diverse action sequences. While these models are often driven by stronger visual representations and…
Focusing on What Matters: Saliency-Harnessing Accurate Routing for Diffusion MoE
Haoyou Deng, Keyu Yan, Chaojie Mao, Xiang Wang, Yu Liu, Changxin Gao, Nong Sang
Mixture-of-Experts (MoE) architectures have emerged as a powerful paradigm for scaling diffusion models in visual generation. Recent advancements have focused on adaptively allocating computational resources across diverse tokens to improve…
Neural Texture Compression using Hypernetworks
Belcour Laurent
Recent work on neural texture compression has demonstrated that it is possible to learn small, per-material texture representations (composed of latent textures and a small Multi-Layer Perceptron decoder) that can be decoded in real-time during…
BOWConnect: Parallel Bayesian Optimization over Windows with Learned Local Cost Maps for Sample-Efficient Kinodynamic Motion Planning
Sourav Raxit, Abdullah Al Redwan Newaz, Jose Fuentes, Leonardo Bobadilla
This paper presents BOWConnect, a bidirectional parallel kinodynamic motion planner that addresses three fundamental limitations of existing sampling-based methods: sample inefficiency in high-dimensional state spaces, unreliable cost heuristics…
UAV-MapFusion: RTK-Aligned Uncertainty-Aware Coarse-to-Fine Multi-Session UAV Mapping
Feng Pan, Chunran Zheng, Bing Xue, Yukang Cui, Jiayu Wen, Zhiyu Chen, Wei Wang
Large-scale point cloud maps are essential for robotics and spatial intelligence tasks. UAVs provide an efficient means for large-scale map acquisition; however, due to limited flight endurance and onboard storage, mapping a large-scale scene…
Scalable Operator Learning via Nyström Approximation With Denoising Applications
Naveen Gupta, Vaibhav Silmana, S. Sivananthan
In this paper, we study Nyström subsampling for vector-valued regression in vector-valued reproducing kernel Hilbert spaces. Standard kernel methods often suffer from prohibitive computational costs due to the construction and inversion of large…
What Does a Pathological Speech Assessment Model Know about Acoustic Features? A Case Study on Oral and Oropharyngeal Cancer Patients
Tuan Nguyen, Corinne Fredouille, Alain Ghio, Muriel Lalain, Virginie Woisard
This work investigates the interpretability of a Wav2Vec 2.0based speech intelligibility assessment model for oral and oropharyngeal cancer patients through canonical correlation analysis. By measuring the correlation between the model embeddings…
OFDM Waveform Optimization for Bistatic Integrated Sensing and Communications
Ruolin Du, Zhiqiang Wei, Zai Yang, Ya-Feng Liu, Bingpeng Zhou, Derrick Wing Kwan Ng
This paper investigates the design of orthogonal frequency-division multiplexing (OFDM) waveforms for bistatic integrated sensing and communication (ISAC) systems. In the considered framework, an ISAC transmitter jointly optimizes subcarrier…
—
Looking Ahead
The pace of AI research shows no signs of slowing. Stay tuned for next week’s digest covering the latest breakthroughs.
This digest is part of the Frontier AI Research Brief series, covering the most significant AI research each week.
Leave a Reply