A focused look at this week’s most significant advances in rl & post-training — 37 papers surveyed from arXiv and leading AI labs.
—
Reinforcement learning continues to drive the most impressive post-training gains. This week covers advances in RL algorithms, reward modeling, and the surprising effectiveness of RL without ground-truth solutions.
Key Developments
Scaling Multi-Reference Image Generation with Dynamic Reward Optimization — Wenwang Huang, Yusen Fu, Junjie Wang, Mengfei Huang, Yulin Li, Gan Liu, Jing Cai, Yancheng He, Zhuotao Tian
While personalized image generation has achieved remarkable progress, multi-reference image generation (MRIG) remains a challenging task. Most existing benchmarks fail to adequately evaluate complex M…
Beyond Global Divergences: A Local-Mass Perspective on Bayesian Inference — Hanli Xu, Fengxiang He, Sarat Moka
Global objectives, such as KL divergence and ELBO, are widely used in Bayesian inference for measuring distributional discrepancy. This paper studies their local-mass behaviour that is not directly ca…
Reinforcement Learning without Ground-Truth Solutions can Improve LLMs — Yingyu Lin, Qiyue Gao, Nikki Lijing Kuang, Xunpeng Huang, Kun Zhou, Tongtong Liang, Zhewei Yao, Yi-An Ma, Yuxiong He
Reinforcement learning with verifiable rewards (RLVR) for training LLMs typically rely on ground-truth answers to assign rewards, limiting their applicability to tasks where the ground-truth solution…
Training & Scaling
Paved with True Intents: Intent-Aware Training Improves LLM Safety Classification Across Training Regimes
Jeremias Ferrao, Niclas Müller-Hof, Iustin Sîrbu, Traian Rebedea, Yftah Ziser
We argue that safety classifiers should model user intent as an explicit signal between the prompt and the final label. To study this, we introduce AIMS, a human-annotated dataset of 1,724 difficult safety prompts, each paired with an intent…
State Representation Matters in Deep Reinforcement Learning: Application to Energy Trading
Jesper Klicks, Sander Vržina, Vincent François-Lavet
Energy trading decisions depend not only on current market prices, but also on expected future market conditions, and operational constraints. This makes the state representation given to a reinforcement learning agent an important design choice….
The Geometry of Updates: Fisher Alignment at Vocabulary Scale
John Sweeney
Training-free source selection for LLM families with shared vocabularies arises in scientific string domains such as SMILES, protein, and genomic sequences, where candidate corpora share a tokenizer but differ in prediction targets. This creates an…
RolloutPipe: Overlapping Pipelined Rollout and Training in Disaggregated On-Policy LLM Reinforcement Learning
Rongjian Chen, Jianmin Hu, Kejiang Ye, Minxian Xu
Large language model (LLM) post-training for reasoning increasingly relies on reinforcement learning with verifiable rewards (RLVR), where models learn from ground-truth feedback on mathematical, logical, and scientific tasks. To enable flexible…
Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards
Ritesh Thawkar, Shravan Venkatraman, Omkar Thawakar, Abdelrahman Shaker, Fahad Khan, Hisham Cholakka
Most unified large multimodal models (LMMs) that support both visual understanding and image generation still rely on curated post-training supervision, such as human annotations, preference labels, or external reward models. We ask whether a…
PortraitGen: Exemplar-Driven GRPO with Dual-Reward Guidance for Photorealistic Portrait Generation
Xiaomin Li, Qian Liang, Yinan Li, Ying Zhang, Chen Li, Jing Lyu, Huchuan Lu, Xu Jia
Reinforcement Learning like Group Relative Policy Optimization (GRPO) has significantly advanced text-to-image post-training. However, current methods often favor superficial aesthetics, such as over-saturated colors, leaving critical flaws like AI…
Reasoning & Inference
Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
Zhenhua Xu, Dongsheng Chen, Jian Li, Yitong Lin, Zhebo Wang, Jiafu Wu, Yizhang Jin, Chengjie Wang, M
Building general-purpose role-playing agents that faithfully portray any character from a natural-language profile remains challenging. The dominant paradigm — supervised fine-tuning — encourages behavioral mimicry without deep, human-like…
Additional Research
Hallucination in World Models is Predictable and Preventable
Nicklas Hansen, Xiaolong Wang
Modern generative world models render increasingly realistic action-controllable futures, yet they frequently hallucinate: rollouts remain visually fluent while drifting from the ground-truth dynamics. We hypothesize that hallucination concentrates…
Designing Reward Signals for Portable Query Generation: A Case Study in Industrial Semantic Job Search
Ping Liu, Qianqi Shen, Jianqiang Shen, Wenqiong Liu, Rajat Arora, Yunxiang Ren, Chunnan Yao, Dan Xu,
Job-search platforms rely on low-bandwidth query interfaces that often fail to capture the high-dimensional complexity of candidate profiles. We present an end-to-end RLAIF (Reinforcement Learning from AI Feedback) framework to generate…
Towards Explainable Adjudicative Variance: Quantifying Judicial Discretion via Gated Multi-Task Learning
Stanisław Sójka, Felix Steffek, Matthias Grabmair
Legal outcome prediction must disentangle objective case facts from adjudicative context. Merit-based rulings rely on factual evidence while technical disposals may hinge on judicial discretion. We propose a Judge-Aware Gated Multi-Task Learning…
AIGP: An LLM-Based Framework for Long-Term Value Alignment in E-Commerce Pricing
Chennan Ma, Yanning Zhang, Siqi Hong, Xiuchong Wang, Fei Xiao, Keping Yang
Traditional dynamic pricing models in large-scale e-commerce suffer from limited interpretability, poor utilization of unstructured information, and misalignment with long-term business objectives such as cumulative Gross Merchandise Value (GMV),…
Evaluation Pitfalls and Challenges in Multimedia Event Extraction
Philipp Seeberger, Steffen Freisinger, Tobias Bocklet, Korbinian Riedhammer
Multimedia event extraction aims to jointly identify events and their arguments across multiple modalities, such as text and images, to support more comprehensive event understanding. While recent work reports steady and substantial progress, the…
Geometric Gradient Rectification for Safe Open-Set Semi-Supervised Learning
Jiahe Chen, Qian Shao, Qiyuan Chen, Jiaying He, Jintai Chen, Jian Wu, Hongxia Xu
Open-set semi-supervised learning aims to leverage unlabeled data that may contain out-of-distribution outliers while maintaining performance on in-distribution classes. Existing methods mainly follow two paradigms: filtering suspicious samples or…
PlanRL: A Trajectory Planning Architecture for Reinforcement Learning-based Driving Experts
Joonhee Lim, Yongjae Lee, Jangho Shin, Dongsuk Kum
Reinforcement learning (RL) has become a prominent framework for developing driving experts in autonomous vehicles. However, most existing RL-based experts are designed to output direct control commands (e.g., throttle, steering), which suffer from…
Asymptotically Optimal Learning for Parametric Prophet Inequalities
Jung-hun Kim, Anna Grebennikova, Vianney Perchet
We study learning in prophet inequalities with i.i.d. rewards drawn from an exponential-type parametric family with an unknown parameter $θ$, a class that includes exponential, Pareto, and bounded-support power-family distributions. We first…
The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence
MiniMax, :, Aili Chen, Aonian Li, Baichuan Zhou, Bangwei Gong, Binyang Jiang, Boji Dan, Changqing
We introduce the MiniMax-M2 series, a family of Mixture-of-Experts language models built around the principle that mini activations can unleash maximum real-world intelligence. The flagship M2 contains 229.9B total parameters with only 9.8B…
Understanding Domain-Aware Distribution Alignment in Budgeted Entity Matching
Nicholas Pulsone, Gregory Goren, Roee Shraga
Entity Matching (EM) is a core operation in the data integration pipeline, where records from different sources are compared to determine whether they refer to the same real-world entity. Recent work has incorporated domain information and…
AI Healthcare Chatbots as Information Infrastructure: A Large-Scale Study of User-Reported Breakdowns
Muhammad Hassan, Ramazan Yener, Ece Gumusel, Masooda Bashir
AI healthcare chatbots are increasingly used to support health information seeking and self-management, yet their performance and impact on users remains to be studied. This study examines over 15,000 user reviews from 59 AI healthcare chatbot apps…
Simulation-based inference for rapid Bayesian parameter estimation in epidemiological models: a comparison with MCMC
Alina Bazarova, Johann Fredrik Jadebeck, Henrik Zunker, Carolina J. Klett-Tammen, Torben Heinsohn, W
Mechanistic epidemiological models are widely used to support infectious disease forecasting and public-health decision making. Bayesian calibration of such models is commonly performed using Markov chain Monte Carlo (MCMC), which can become…
Automating Potential-based Reward Shaping with Vision Language Model Guidance
Henrik Müller, Daniel Kudenko
Sparse rewards are inherently challenging for reinforcement learning agents as they lack intermediate feedback to guide exploration and to correctly attribute the sparse success rewards to relevant parts of the trajectory. Naive reward shaping can…
Heavy-Ball Q-Learning with Residual Weighting Correction
Donghwan Lee
This paper proposes a corrected heavy-ball Q-learning method for reinforcement learning (RL) and establishes its convergence. It also identifies conditions under which the method is theoretically guaranteed to converge faster than standard…
Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs
Sinie van der Ben, Raphaël Baur, Yannick Metz, Mennatallah El-Assady
Recent work identified emotion vectors in Claude Sonnet 4.5, which are internal representations that encode emotion concepts, causally influence behavior, and exhibit geometry mirroring human psychological structure. We test the generality of these…
Beyond Surface Forms: A Comprehensive, Mechanism-Oriented Taxonomy of Indirect Linguistic Encoding for LLM-Based Coded Language Detection
Hamid Reza Firoozfar, Mohammadsadegh Abolhasani, Reza Mousavi, Paul Jen-Hwa Hu
To avoid moderation and surveillance on social media, some users routinely invent indirect linguistic expressions (ILE) that camouflage sensitive meanings. Such expressions surface as algospeak, euphemisms, and adversarial obfuscation, depending on…
Term-Centric Hierarchy Induction from Heterogeneous Corpora
Elena Senger, Yuri Campbell, Jan-Peter Bergmann, Rob van der Goot, Barbara Plank
Organizing knowledge from diverse text sources into interpretable hierarchies is crucial for tasks such as policy analysis, innovation monitoring, and exploratory domain mapping. Existing taxonomy induction methods typically rely on document-level…
OPID: On-Policy Skill Distillation for Agentic Reinforcement Learning
Shuo Yang, Jinyang Wu, Zhengxi Lu, Yuhao Shen, Fan Zhang, Lang Feng, Shuai Zhang, Haoran Luo, Zheng
Outcome-based reinforcement learning provides a stable optimization backbone for language agents, but its sparse trajectory-level rewards provide little guidance on which intermediate decisions should be reinforced or suppressed. On-policy…
SAM2Matting: Generalized Image and Video Matting
Ruiqi Shen, Guangquan Jie, Chang Liu, Henghui Ding
Despite impressive advances in image matting, video matting remains challenging due to the inherent gap between high-level tracking, which requires frame-wise understanding, and low-level matting, which focuses on extremely fine-grained details….
TraMP-LLaMA: Generative Interpretability with Decoupled Instruction Tuning for Facial Expression Quality Assessment
Shuchao Duan, Alan Whone, Hossein Rahmani, Jun Liu, Majid Mirmehdi
Existing facial expression quality assessment (FEQA) methods typically produce only a severity score, without explicitly communicating the observable facial motion evidence that supports the prediction. This limits interpretability and makes it…
Bridging Performance and Generalization in Reinforcement Learning for Agile Flight
Jonathan Green, Jiaxu Xing, Nico Messikommer, Angel Romero, Davide Scaramuzza
Autonomous drone racing is a fundamentally challenging regime for autonomous aerial robots, requiring time-optimal control while operating under persistent actuation saturation. While reinforcement learning (RL) has achieved human-level performance…
RobOralScan: Learning Active Intraoral Scanning for Robotic Dental Reconstruction
Jinhyung Lee, Haeun Yun, Siwon Kim, Gihyun Baek, Sungho Moon, Sehyun Hwang, Sunghoon Im
Intraoral scanning is widely used for digital optical impressions in prosthodontic, implant, and orthodontic treatment, but full-arch and long-span scanning remain labor-intensive tasks with limited automation. In the confined oral cavity,…
Data-Driven Duration Management — Term Structure Forecasting Using Machine Learning
Tobias Lausser, Joao Eduardo Vuolo, Rudi Zagst
This paper compares different methods for forecasting the term structure of U.S. and European zero-coupon government bonds using both traditional econometric and Machine Learning (ML) approaches. We compare classical models (e.g., Dynamic…
Statistical and Structural Approaches to Algorithmic Fairness
Antonio Ferrara
Modern machine learning systems have outgrown their origins as isolated predictive constructs, evolving into complex socio-technical architectures that actively mediate human opportunity. As algorithms increasingly determine access to economic and…
Evolutionary Optimization Reveals Structural Constraints on Reservoir Architecture for Spatiotemporal Chaos
Nima Dehghani
Biological systems maintain function in fluctuating environments by transforming past stimulation into internal dynamical states that support future-oriented responses. Reservoir computing provides a computational analogue, but standard…
Analysis of Charged-Particle/Photon Observables in Hadronic Multiparticle Production
MiniMax Collaboration
In order to analyze data on joint charged-particle/photon distributions from an experimental search (T-864, MiniMax) for disoriented chiral condensate (DCC) at the Fermilab Tevatron collider, we have identified robust observables, ratios of…
ReinforceGen: Hybrid Skill Policies with Automated Data Generation and Reinforcement Learning
**
—
Looking Ahead
The pace of AI research shows no signs of slowing. Stay tuned for next week’s digest covering the latest breakthroughs.
This digest is part of the Frontier AI Research Brief series, covering the most significant AI research each week.
Leave a Reply