A focused look at this week’s most significant advances in scaling, efficiency & architectures \u2014 27 papers surveyed from arXiv and leading AI labs.<\/p>\n
—<\/p>\n
Efficiency remains the quiet revolution in AI research. This week brings advances in attention mechanisms, model compression, speculative decoding, and the mathematics behind training dynamics.<\/p>\n
Blackwell Approachability and Gradient Equilibrium are Equivalent<\/strong> \u2014 Brian W. Lee, Nika Haghtalab, Michael I. Jordan, Ryan J. Tibshirani<\/em><\/p>\n Gradient equilibrium (GEQ) is a recently introduced online optimization framework that generalizes first-order stationarity from offline optimization and abstracts problems like online conformal predi…<\/p>\n arXiv<\/a><\/p>\n Hierarchical Muon: Tiled Newton-Schulz Updates for Efficient Muon Optimization<\/strong> \u2014 Ziyuan Tang, Tianshi Xu, Yousef Saad, Yuanzhe Xi<\/em><\/p>\n Muon-type optimizers construct update directions for dense neural-network weights by applying a finite Newton-Schulz map to momentum-gradient matrices. For an $H \\times W$ matrix, with $r=\\min\\{H,W\\}$…<\/p>\n arXiv<\/a><\/p>\n Don’t Settle at the Mode! Mitigating Diversity Collapse in Pretrained Flow Models via Feature Self-Guidance<\/strong> \u2014 Pradhaan S Bhat, Rishubh Parihar, Abhijnya Bhat, R. Venkatesh Babu<\/em><\/p>\n State-of-the-art flow models generate stunning images from text or image prompts. However, they suffer from diversity collapse when generating multiple samples under the same conditioning. Existing me…<\/p>\n arXiv<\/a><\/p>\n MinGram: A Minimalist Unigram Tokenizer with High Compression and Competitive Morphological Alignment<\/strong><\/p>\n Sander Land<\/em><\/p>\n The Unigram tokenizer uses an elegant representation which makes it straightforward to edit vocabularies, but its training is comparatively heavy and complex. We introduce MinGram (Minimalist Unigram), which keeps the token-list representation but…<\/p>\n arXiv<\/a><\/p>\n CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMs<\/strong><\/p>\n Shigeng Wang, Chao Li, Yangyuxuan Kang, Jiawei Fan, Anbang Yao<\/em><\/p>\n In this paper, we present CAT-Q, Cost-efficient and Accurate Ternary Quantization, for compressing and accelerating LLMs. Unlike existing state-of-the-art ternary quantization methods that rely on data-intensive and costly quantization-aware…<\/p>\n arXiv<\/a><\/p>\n ViQ: Text-Aligned Visual Quantized Representations at Any Resolution<\/strong><\/p>\n Xumin Yu, Zuyan Liu, Zhenyu Yang, Yuhao Dong, Shengsheng Qian, Jiwen Lu, Han Hu, Yongming Rao<\/em><\/p>\n A unified representation for text and vision is a natural pursuit, as it enables simpler multimodal modeling and more efficient training. However, representing images as discrete signals in the same way as text inevitably introduces severe…<\/p>\n arXiv<\/a><\/p>\n Minimax PAC Bounds for Learning in Exogenous Contextual MDPs<\/strong><\/p>\n Corentin Pla, Hugo Richard, Marc Abeille, Vianney Perchet<\/em><\/p>\n We study PAC learning in tabular discounted Markov decision processes with exogenous i.i.d. contexts, with discount factor $\u03b3$, finite state space $\\mathcal X$, action space $\\mathcal A$, and context space $\\mathcal Z$. At each time step, a context…<\/p>\n arXiv<\/a><\/p>\n Self-Modulating Quantum Fast-Weight Programmers for Efficient Adaptive Sequential Learning<\/strong><\/p>\n Samuel Yen-Chi Chen, Yifeng Peng, Kuo-Chung Peng, Jiun-Cheng Jiang, Chun-Hua Lin, Junghoon Justin Pa<\/em><\/p>\n Recent advances in quantum machine learning have motivated efficient models for sequential data processing. In this paper, we propose Self-Modulating Quantum Fast Weight Programmers, or Self-Modulating QFWP, which extends Quantum Fast Weight…<\/p>\n arXiv<\/a><\/p>\n Fast algorithms for learning a Gaussian under halfspace truncation with optimal sample complexity<\/strong><\/p>\n Haitong Liu, Deepak Narayanan Sridharan, David Steurer, Manuel Wiedmer<\/em><\/p>\n We study the fundamental problem of learning a high-dimensional Gaussian truncated to an unknown halfspace. Lee, Mehrotra and Zampetakis (FOCS’24) recently obtained the first polynomial time algorithm for this problem, but their resulting sample…<\/p>\n arXiv<\/a><\/p>\n Stochastic Gradient Optimization with Model-Assisted Sampling<\/strong><\/p>\n Jonne Pohjankukka, Jukka Heikkonen<\/em><\/p>\n This work addresses the problem of variance in stochastic gradient estimation for machine learning optimization. Deep learning relies on mini-batch methods such as stochastic gradient descent, which approximate full gradients but introduce noise,…<\/p>\n arXiv<\/a><\/p>\n Kolmogorov Arnold networks (KAN) for aerodynamic prediction: a comparison with MLPs and GNNs<\/strong><\/p>\n Miguel Jaraiz, Fermin Gutierrez, Pablo Yeste, Miguel S\u00e1nchez-Dom\u00ednguez, Eusebio Valero, Gonzalo Rubi<\/em><\/p>\n Kolmogorov Arnold networks (KAN) have recently been introduced as a (deep) neural network architecture whose trainable parameters adapt the activation functions, instead of the coefficients of the affine transformations at the core of traditional…<\/p>\n arXiv<\/a><\/p>\n Compositionality and the lexicon in evolutionary semantics<\/strong><\/p>\n Fausto Carcassi<\/em><\/p>\n Formal semantics has shown that sentence meanings arise by recursively composing lexical meanings, yet much of the literature on semantic universals models either lexicons with fixed signal structures or holistic composition without interpretable…<\/p>\n arXiv<\/a><\/p>\n Jailbreaking for the Average Jane: Choosing Optimal Jailbreaks via Bandit Algorithms for Automatically Enhanced Queries<\/strong><\/p>\n Prarabdh Shukla, Ritik, Suhas Rao, Arpit Agarwal, Arjun Bhagoji<\/em><\/p>\n With a profusion of jailbreaks for LLMs now widely known, a growing concern is that non-expert malicious actors (“the average Jane”) could elicit actionable responses to malicious requests. In this work, we examine whether this concern is…<\/p>\n arXiv<\/a><\/p>\n HyperDFlash: MHC-Aligned Block Speculative Decoding with Gated Residual Reduction<\/strong><\/p>\n Luxi Lin, Shuang Peng, Rui Ma, Junhao Hua, Shuwei Fan, Zhengda Qin, Qiang Wang, Hongjian Sun, Fangmi<\/em><\/p>\n We present HyperDFlash, a block-parallel speculative decoding framework tailored to the novel multi-hyper-connection (MHC) architecture proposed by DeepSeek-V4. Despite the strong initial-token drafting performance of the native Multi-Token…<\/p>\n arXiv<\/a><\/p>\n TMP: Tree-structured Mixed-policy Pruning for Large-scale Image Generation and Editing<\/strong><\/p>\n Peizhen Zhang, Yang Li, Xunsong Li, Songtao Liu, Zewen Liu, Qiangqiang Hu, Guotong Guo, Jupeng Ding,<\/em><\/p>\n Modern image generation model rapidly grows their sizes to meet high-fidelity image synthesis. However, they gradually become unaffordable for their enormous parameter consumption and computation budget that lead to massive resources requirement…<\/p>\n arXiv<\/a><\/p>\n Explainable Outlier Detection for Interval-valued Data<\/strong><\/p>\n Catarina P. Loureiro, M. Ros\u00e1rio Oliveira, Paula Brito, Lina Oliveira<\/em><\/p>\n Explainability is increasingly recognized as a key aspect of outlier detection. However, for complex data structures such as interval-valued data, it remains largely unexplored. Building on an outlier detection framework based on the Interval…<\/p>\n arXiv<\/a><\/p>\n MiniMax-01: Scaling Foundation Models with Lightning Attention<\/strong><\/p>\n MiniMax, Aonian Li, Bangwei Gong, Bo Yang, Boji Shan, Chang Liu, Cheng Zhu, Chunhao Zhang, Congchao<\/em><\/p>\n We introduce MiniMax-01 series, including MiniMax-Text-01 and MiniMax-VL-01, which are comparable to top-tier models while offering superior capabilities in processing longer contexts. The core lies in lightning attention and its efficient scaling….<\/p>\n arXiv<\/a><\/p>\n HL-index: Fast Reachability Query in Hypergraphs<\/strong><\/p>\n Peiting Xie, Xiangjun Zai, Yanping Wu, Xiaoyang Wang, Wenjie Zhang, Lu Qin<\/em><\/p>\n Reachability in hypergraphs is essential for modeling complex groupwise interactions in real-world applications such as co-authorship, social network, and biological analysis, where relationships go beyond pairwise interactions. In this paper, we…<\/p>\n arXiv<\/a><\/p>\n Symplectic Neural Networks for learning Generalized Hamiltonians<\/strong><\/p>\n Harsh Choudhary, Vyacheslav Kungurtsev, Chandan Gupta, Melvin Leok, Georgios Korpas<\/em><\/p>\n Hamiltonian Neural Networks (HNNs) integrate physical priors into neural models by learning a system’s Hamiltonian, improving generalization and sample efficiency. Identifying the system Hamiltonian from noisy observations of state variables is a…<\/p>\n arXiv<\/a><\/p>\n Not All Actions Are Equal: Rethinking Conditioning for Dexterous World Model<\/strong><\/p>\n Zizhao Yuan, Zhengtu Liang, Taowen Wang, Qiwei Liang, Yichi Wang, Yunheng Wang, Yuetong Fang, Lusong<\/em><\/p>\n Recent advances in action-conditioned world models show promising progress in modeling complex interactions and forecasting future states under diverse action sequences. While these models are often driven by stronger visual representations and…<\/p>\n arXiv<\/a><\/p>\n Focusing on What Matters: Saliency-Harnessing Accurate Routing for Diffusion MoE<\/strong><\/p>\n Haoyou Deng, Keyu Yan, Chaojie Mao, Xiang Wang, Yu Liu, Changxin Gao, Nong Sang<\/em><\/p>\n Mixture-of-Experts (MoE) architectures have emerged as a powerful paradigm for scaling diffusion models in visual generation. Recent advancements have focused on adaptively allocating computational resources across diverse tokens to improve…<\/p>\n arXiv<\/a><\/p>\n Neural Texture Compression using Hypernetworks<\/strong><\/p>\n Belcour Laurent<\/em><\/p>\n Recent work on neural texture compression has demonstrated that it is possible to learn small, per-material texture representations (composed of latent textures and a small Multi-Layer Perceptron decoder) that can be decoded in real-time during…<\/p>\n arXiv<\/a><\/p>\n BOWConnect: Parallel Bayesian Optimization over Windows with Learned Local Cost Maps for Sample-Efficient Kinodynamic Motion Planning<\/strong><\/p>\n Sourav Raxit, Abdullah Al Redwan Newaz, Jose Fuentes, Leonardo Bobadilla<\/em><\/p>\n This paper presents BOWConnect, a bidirectional parallel kinodynamic motion planner that addresses three fundamental limitations of existing sampling-based methods: sample inefficiency in high-dimensional state spaces, unreliable cost heuristics…<\/p>\n arXiv<\/a><\/p>\n UAV-MapFusion: RTK-Aligned Uncertainty-Aware Coarse-to-Fine Multi-Session UAV Mapping<\/strong><\/p>\n Feng Pan, Chunran Zheng, Bing Xue, Yukang Cui, Jiayu Wen, Zhiyu Chen, Wei Wang<\/em><\/p>\n Large-scale point cloud maps are essential for robotics and spatial intelligence tasks. UAVs provide an efficient means for large-scale map acquisition; however, due to limited flight endurance and onboard storage, mapping a large-scale scene…<\/p>\n arXiv<\/a><\/p>\n Scalable Operator Learning via Nystr\u00f6m Approximation With Denoising Applications<\/strong><\/p>\n Naveen Gupta, Vaibhav Silmana, S. Sivananthan<\/em><\/p>\n In this paper, we study Nystr\u00f6m subsampling for vector-valued regression in vector-valued reproducing kernel Hilbert spaces. Standard kernel methods often suffer from prohibitive computational costs due to the construction and inversion of large…<\/p>\n arXiv<\/a><\/p>\n What Does a Pathological Speech Assessment Model Know about Acoustic Features? A Case Study on Oral and Oropharyngeal Cancer Patients<\/strong><\/p>\n Tuan Nguyen, Corinne Fredouille, Alain Ghio, Muriel Lalain, Virginie Woisard<\/em><\/p>\n This work investigates the interpretability of a Wav2Vec 2.0based speech intelligibility assessment model for oral and oropharyngeal cancer patients through canonical correlation analysis. By measuring the correlation between the model embeddings…<\/p>\nTraining & Scaling<\/h2>\n
Agent Systems<\/h2>\n
Additional Research<\/h2>\n