A focused look at this week’s most significant advances in rl & post-training \u2014 37 papers surveyed from arXiv and leading AI labs.<\/p>\n
—<\/p>\n
Reinforcement learning continues to drive the most impressive post-training gains. This week covers advances in RL algorithms, reward modeling, and the surprising effectiveness of RL without ground-truth solutions.<\/p>\n
Scaling Multi-Reference Image Generation with Dynamic Reward Optimization<\/strong> \u2014 Wenwang Huang, Yusen Fu, Junjie Wang, Mengfei Huang, Yulin Li, Gan Liu, Jing Cai, Yancheng He, Zhuotao Tian<\/em><\/p>\n While personalized image generation has achieved remarkable progress, multi-reference image generation (MRIG) remains a challenging task. Most existing benchmarks fail to adequately evaluate complex M…<\/p>\n arXiv<\/a><\/p>\n Beyond Global Divergences: A Local-Mass Perspective on Bayesian Inference<\/strong> \u2014 Hanli Xu, Fengxiang He, Sarat Moka<\/em><\/p>\n Global objectives, such as KL divergence and ELBO, are widely used in Bayesian inference for measuring distributional discrepancy. This paper studies their local-mass behaviour that is not directly ca…<\/p>\n arXiv<\/a><\/p>\n Reinforcement Learning without Ground-Truth Solutions can Improve LLMs<\/strong> \u2014 Yingyu Lin, Qiyue Gao, Nikki Lijing Kuang, Xunpeng Huang, Kun Zhou, Tongtong Liang, Zhewei Yao, Yi-An Ma, Yuxiong He<\/em><\/p>\n Reinforcement learning with verifiable rewards (RLVR) for training LLMs typically rely on ground-truth answers to assign rewards, limiting their applicability to tasks where the ground-truth solution…<\/p>\n arXiv<\/a><\/p>\n Paved with True Intents: Intent-Aware Training Improves LLM Safety Classification Across Training Regimes<\/strong><\/p>\n Jeremias Ferrao, Niclas M\u00fcller-Hof, Iustin S\u00eerbu, Traian Rebedea, Yftah Ziser<\/em><\/p>\n We argue that safety classifiers should model user intent as an explicit signal between the prompt and the final label. To study this, we introduce AIMS, a human-annotated dataset of 1,724 difficult safety prompts, each paired with an intent…<\/p>\n arXiv<\/a><\/p>\n State Representation Matters in Deep Reinforcement Learning: Application to Energy Trading<\/strong><\/p>\n Jesper Klicks, Sander Vr\u017eina, Vincent Fran\u00e7ois-Lavet<\/em><\/p>\n Energy trading decisions depend not only on current market prices, but also on expected future market conditions, and operational constraints. This makes the state representation given to a reinforcement learning agent an important design choice….<\/p>\n arXiv<\/a><\/p>\n The Geometry of Updates: Fisher Alignment at Vocabulary Scale<\/strong><\/p>\n John Sweeney<\/em><\/p>\n Training-free source selection for LLM families with shared vocabularies arises in scientific string domains such as SMILES, protein, and genomic sequences, where candidate corpora share a tokenizer but differ in prediction targets. This creates an…<\/p>\n arXiv<\/a><\/p>\n RolloutPipe: Overlapping Pipelined Rollout and Training in Disaggregated On-Policy LLM Reinforcement Learning<\/strong><\/p>\n Rongjian Chen, Jianmin Hu, Kejiang Ye, Minxian Xu<\/em><\/p>\n Large language model (LLM) post-training for reasoning increasingly relies on reinforcement learning with verifiable rewards (RLVR), where models learn from ground-truth feedback on mathematical, logical, and scientific tasks. To enable flexible…<\/p>\n arXiv<\/a><\/p>\n Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards<\/strong><\/p>\n Ritesh Thawkar, Shravan Venkatraman, Omkar Thawakar, Abdelrahman Shaker, Fahad Khan, Hisham Cholakka<\/em><\/p>\n Most unified large multimodal models (LMMs) that support both visual understanding and image generation still rely on curated post-training supervision, such as human annotations, preference labels, or external reward models. We ask whether a…<\/p>\n arXiv<\/a><\/p>\n PortraitGen: Exemplar-Driven GRPO with Dual-Reward Guidance for Photorealistic Portrait Generation<\/strong><\/p>\n Xiaomin Li, Qian Liang, Yinan Li, Ying Zhang, Chen Li, Jing Lyu, Huchuan Lu, Xu Jia<\/em><\/p>\n Reinforcement Learning like Group Relative Policy Optimization (GRPO) has significantly advanced text-to-image post-training. However, current methods often favor superficial aesthetics, such as over-saturated colors, leaving critical flaws like AI…<\/p>\n arXiv<\/a><\/p>\n Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization<\/strong><\/p>\n Zhenhua Xu, Dongsheng Chen, Jian Li, Yitong Lin, Zhebo Wang, Jiafu Wu, Yizhang Jin, Chengjie Wang, M<\/em><\/p>\n Building general-purpose role-playing agents that faithfully portray any character from a natural-language profile remains challenging. The dominant paradigm — supervised fine-tuning — encourages behavioral mimicry without deep, human-like…<\/p>\n arXiv<\/a><\/p>\n Hallucination in World Models is Predictable and Preventable<\/strong><\/p>\n Nicklas Hansen, Xiaolong Wang<\/em><\/p>\n Modern generative world models render increasingly realistic action-controllable futures, yet they frequently hallucinate: rollouts remain visually fluent while drifting from the ground-truth dynamics. We hypothesize that hallucination concentrates…<\/p>\n arXiv<\/a><\/p>\n Designing Reward Signals for Portable Query Generation: A Case Study in Industrial Semantic Job Search<\/strong><\/p>\n Ping Liu, Qianqi Shen, Jianqiang Shen, Wenqiong Liu, Rajat Arora, Yunxiang Ren, Chunnan Yao, Dan Xu,<\/em><\/p>\n Job-search platforms rely on low-bandwidth query interfaces that often fail to capture the high-dimensional complexity of candidate profiles. We present an end-to-end RLAIF (Reinforcement Learning from AI Feedback) framework to generate…<\/p>\n arXiv<\/a><\/p>\n Towards Explainable Adjudicative Variance: Quantifying Judicial Discretion via Gated Multi-Task Learning<\/strong><\/p>\n Stanis\u0142aw S\u00f3jka, Felix Steffek, Matthias Grabmair<\/em><\/p>\n Legal outcome prediction must disentangle objective case facts from adjudicative context. Merit-based rulings rely on factual evidence while technical disposals may hinge on judicial discretion. We propose a Judge-Aware Gated Multi-Task Learning…<\/p>\n arXiv<\/a><\/p>\n AIGP: An LLM-Based Framework for Long-Term Value Alignment in E-Commerce Pricing<\/strong><\/p>\n Chennan Ma, Yanning Zhang, Siqi Hong, Xiuchong Wang, Fei Xiao, Keping Yang<\/em><\/p>\n Traditional dynamic pricing models in large-scale e-commerce suffer from limited interpretability, poor utilization of unstructured information, and misalignment with long-term business objectives such as cumulative Gross Merchandise Value (GMV),…<\/p>\n arXiv<\/a><\/p>\n Evaluation Pitfalls and Challenges in Multimedia Event Extraction<\/strong><\/p>\n Philipp Seeberger, Steffen Freisinger, Tobias Bocklet, Korbinian Riedhammer<\/em><\/p>\n Multimedia event extraction aims to jointly identify events and their arguments across multiple modalities, such as text and images, to support more comprehensive event understanding. While recent work reports steady and substantial progress, the…<\/p>\n arXiv<\/a><\/p>\n Geometric Gradient Rectification for Safe Open-Set Semi-Supervised Learning<\/strong><\/p>\n Jiahe Chen, Qian Shao, Qiyuan Chen, Jiaying He, Jintai Chen, Jian Wu, Hongxia Xu<\/em><\/p>\n Open-set semi-supervised learning aims to leverage unlabeled data that may contain out-of-distribution outliers while maintaining performance on in-distribution classes. Existing methods mainly follow two paradigms: filtering suspicious samples or…<\/p>\n arXiv<\/a><\/p>\n PlanRL: A Trajectory Planning Architecture for Reinforcement Learning-based Driving Experts<\/strong><\/p>\n Joonhee Lim, Yongjae Lee, Jangho Shin, Dongsuk Kum<\/em><\/p>\n Reinforcement learning (RL) has become a prominent framework for developing driving experts in autonomous vehicles. However, most existing RL-based experts are designed to output direct control commands (e.g., throttle, steering), which suffer from…<\/p>\n arXiv<\/a><\/p>\n Asymptotically Optimal Learning for Parametric Prophet Inequalities<\/strong><\/p>\n Jung-hun Kim, Anna Grebennikova, Vianney Perchet<\/em><\/p>\n We study learning in prophet inequalities with i.i.d. rewards drawn from an exponential-type parametric family with an unknown parameter $\u03b8$, a class that includes exponential, Pareto, and bounded-support power-family distributions. We first…<\/p>\n arXiv<\/a><\/p>\n The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence<\/strong><\/p>\n MiniMax, :, Aili Chen, Aonian Li, Baichuan Zhou, Bangwei Gong, Binyang Jiang, Boji Dan, Changqing <\/em><\/p>\n We introduce the MiniMax-M2 series, a family of Mixture-of-Experts language models built around the principle that mini activations can unleash maximum real-world intelligence. The flagship M2 contains 229.9B total parameters with only 9.8B…<\/p>\n arXiv<\/a><\/p>\n Understanding Domain-Aware Distribution Alignment in Budgeted Entity Matching<\/strong><\/p>\n Nicholas Pulsone, Gregory Goren, Roee Shraga<\/em><\/p>\n Entity Matching (EM) is a core operation in the data integration pipeline, where records from different sources are compared to determine whether they refer to the same real-world entity. Recent work has incorporated domain information and…<\/p>\n arXiv<\/a><\/p>\n AI Healthcare Chatbots as Information Infrastructure: A Large-Scale Study of User-Reported Breakdowns<\/strong><\/p>\n Muhammad Hassan, Ramazan Yener, Ece Gumusel, Masooda Bashir<\/em><\/p>\n AI healthcare chatbots are increasingly used to support health information seeking and self-management, yet their performance and impact on users remains to be studied. This study examines over 15,000 user reviews from 59 AI healthcare chatbot apps…<\/p>\n arXiv<\/a><\/p>\n Simulation-based inference for rapid Bayesian parameter estimation in epidemiological models: a comparison with MCMC<\/strong><\/p>\n Alina Bazarova, Johann Fredrik Jadebeck, Henrik Zunker, Carolina J. Klett-Tammen, Torben Heinsohn, W<\/em><\/p>\n Mechanistic epidemiological models are widely used to support infectious disease forecasting and public-health decision making. Bayesian calibration of such models is commonly performed using Markov chain Monte Carlo (MCMC), which can become…<\/p>\n arXiv<\/a><\/p>\n Automating Potential-based Reward Shaping with Vision Language Model Guidance<\/strong><\/p>\n Henrik M\u00fcller, Daniel Kudenko<\/em><\/p>\n Sparse rewards are inherently challenging for reinforcement learning agents as they lack intermediate feedback to guide exploration and to correctly attribute the sparse success rewards to relevant parts of the trajectory. Naive reward shaping can…<\/p>\n arXiv<\/a><\/p>\n Heavy-Ball Q-Learning with Residual Weighting Correction<\/strong><\/p>\n Donghwan Lee<\/em><\/p>\n This paper proposes a corrected heavy-ball Q-learning method for reinforcement learning (RL) and establishes its convergence. It also identifies conditions under which the method is theoretically guaranteed to converge faster than standard…<\/p>\n arXiv<\/a><\/p>\n Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs<\/strong><\/p>\n Sinie van der Ben, Rapha\u00ebl Baur, Yannick Metz, Mennatallah El-Assady<\/em><\/p>\n Recent work identified emotion vectors in Claude Sonnet 4.5, which are internal representations that encode emotion concepts, causally influence behavior, and exhibit geometry mirroring human psychological structure. We test the generality of these…<\/p>\n arXiv<\/a><\/p>\n Beyond Surface Forms: A Comprehensive, Mechanism-Oriented Taxonomy of Indirect Linguistic Encoding for LLM-Based Coded Language Detection<\/strong><\/p>\n Hamid Reza Firoozfar, Mohammadsadegh Abolhasani, Reza Mousavi, Paul Jen-Hwa Hu<\/em><\/p>\n To avoid moderation and surveillance on social media, some users routinely invent indirect linguistic expressions (ILE) that camouflage sensitive meanings. Such expressions surface as algospeak, euphemisms, and adversarial obfuscation, depending on…<\/p>\n arXiv<\/a><\/p>\n Term-Centric Hierarchy Induction from Heterogeneous Corpora<\/strong><\/p>\n Elena Senger, Yuri Campbell, Jan-Peter Bergmann, Rob van der Goot, Barbara Plank<\/em><\/p>\n Organizing knowledge from diverse text sources into interpretable hierarchies is crucial for tasks such as policy analysis, innovation monitoring, and exploratory domain mapping. Existing taxonomy induction methods typically rely on document-level…<\/p>\n arXiv<\/a><\/p>\n OPID: On-Policy Skill Distillation for Agentic Reinforcement Learning<\/strong><\/p>\n Shuo Yang, Jinyang Wu, Zhengxi Lu, Yuhao Shen, Fan Zhang, Lang Feng, Shuai Zhang, Haoran Luo, Zheng <\/em><\/p>\n Outcome-based reinforcement learning provides a stable optimization backbone for language agents, but its sparse trajectory-level rewards provide little guidance on which intermediate decisions should be reinforced or suppressed. On-policy…<\/p>\n arXiv<\/a><\/p>\n SAM2Matting: Generalized Image and Video Matting<\/strong><\/p>\n Ruiqi Shen, Guangquan Jie, Chang Liu, Henghui Ding<\/em><\/p>\n Despite impressive advances in image matting, video matting remains challenging due to the inherent gap between high-level tracking, which requires frame-wise understanding, and low-level matting, which focuses on extremely fine-grained details….<\/p>\n arXiv<\/a><\/p>\n TraMP-LLaMA: Generative Interpretability with Decoupled Instruction Tuning for Facial Expression Quality Assessment<\/strong><\/p>\n Shuchao Duan, Alan Whone, Hossein Rahmani, Jun Liu, Majid Mirmehdi<\/em><\/p>\n Existing facial expression quality assessment (FEQA) methods typically produce only a severity score, without explicitly communicating the observable facial motion evidence that supports the prediction. This limits interpretability and makes it…<\/p>\n arXiv<\/a><\/p>\n Bridging Performance and Generalization in Reinforcement Learning for Agile Flight<\/strong><\/p>\n Jonathan Green, Jiaxu Xing, Nico Messikommer, Angel Romero, Davide Scaramuzza<\/em><\/p>\n Autonomous drone racing is a fundamentally challenging regime for autonomous aerial robots, requiring time-optimal control while operating under persistent actuation saturation. While reinforcement learning (RL) has achieved human-level performance…<\/p>\n arXiv<\/a><\/p>\n RobOralScan: Learning Active Intraoral Scanning for Robotic Dental Reconstruction<\/strong><\/p>\n Jinhyung Lee, Haeun Yun, Siwon Kim, Gihyun Baek, Sungho Moon, Sehyun Hwang, Sunghoon Im<\/em><\/p>\n Intraoral scanning is widely used for digital optical impressions in prosthodontic, implant, and orthodontic treatment, but full-arch and long-span scanning remain labor-intensive tasks with limited automation. In the confined oral cavity,…<\/p>\n arXiv<\/a><\/p>\n Data-Driven Duration Management — Term Structure Forecasting Using Machine Learning<\/strong><\/p>\n Tobias Lausser, Joao Eduardo Vuolo, Rudi Zagst<\/em><\/p>\n This paper compares different methods for forecasting the term structure of U.S. and European zero-coupon government bonds using both traditional econometric and Machine Learning (ML) approaches. We compare classical models (e.g., Dynamic…<\/p>\n arXiv<\/a><\/p>\n Statistical and Structural Approaches to Algorithmic Fairness<\/strong><\/p>\n Antonio Ferrara<\/em><\/p>\n Modern machine learning systems have outgrown their origins as isolated predictive constructs, evolving into complex socio-technical architectures that actively mediate human opportunity. As algorithms increasingly determine access to economic and…<\/p>\n arXiv<\/a><\/p>\n Evolutionary Optimization Reveals Structural Constraints on Reservoir Architecture for Spatiotemporal Chaos<\/strong><\/p>\n Nima Dehghani<\/em><\/p>\n Biological systems maintain function in fluctuating environments by transforming past stimulation into internal dynamical states that support future-oriented responses. Reservoir computing provides a computational analogue, but standard…<\/p>\n arXiv<\/a><\/p>\n Analysis of Charged-Particle\/Photon Observables in Hadronic Multiparticle Production<\/strong><\/p>\n MiniMax Collaboration<\/em><\/p>\n In order to analyze data on joint charged-particle\/photon distributions from an experimental search (T-864, MiniMax) for disoriented chiral condensate (DCC) at the Fermilab Tevatron collider, we have identified robust observables, ratios of…<\/p>\nTraining & Scaling<\/h2>\n
Reasoning & Inference<\/h2>\n
Additional Research<\/h2>\n