A focused look at this week’s most significant advances in video & image generation — 33 papers surveyed from arXiv and leading AI labs.
—
Visual generation is racing forward on multiple fronts: diffusion models get faster and more controllable, new architectures emerge, and video generation inches toward production quality.
Key Developments
PhysRAG: Enhancing Physics-Awareness in Video Generation via Retrieval-Augmented Generation — Kexu Cheng, Zicheng Liu, Mingju Gao, Chunhe Song, Hao Tang
Developing physically aware video generation models remains a significant challenge due to the difficulty in capturing diverse physical phenomena, such as thermal dynamics, mechanics, and optics. In t…
Error-Conditioned Neural Solvers — Haina Jiang, Liam Wang, Peng-Chen Chen, Min Seop Kwak, Seungryong Kim, Brian Bell, Jeong Joon Park
Neural surrogate models offer fast approximate mappings from PDE parameters to solutions, but they typically treat solving as a purely statistical task: once trained, they struggle to correct their ow…
DanceOPD: On-Policy Generative Field Distillation — Wei Zhou, Xiongwei Zhu, Zelin Xu, Bo Dong, Lixue Gong, Yongyuan Liang, Meng Chu, Leigang Qu, Lingdong Kong, Wei Liu, Tat
Modern image generation demands a single model that unifies diverse capabilities, including text-to-image (T2I), local editing, and global editing. However, these capabilities are rarely naturally ali…
Training & Scaling
LISA: Likelihood Score Alignment for Visual-condition Controllable Generation
Yanghao Wang, Hongxu Chen, Jiazhen Liu, Zhenqi He, Rui Liu, Zhen Wang, Long Chen
The prevalent dual-branch paradigm, i.e., training a side network to encode visual conditions and fusing its intermediate-layer features to a frozen pretrained main network, has shown remarkable success in visual-condition controllable generation….
Sculpting NeRF Geometry: Human-Preference Fine-Tuning of a 3D-Aware Face GAN
Archer Moore, Mingming Gong, Liam Hodgkinson
Reinforcement learning from human feedback (RLHF) for 3D generation is now established across a number of works, but most existing pipelines optimise explicit surface representations, often by converting radiance fields into meshes and training…
FedReLa: Imbalanced Federated Learning via Re-Labeling
Guangzheng Hu, Patricia Menéndez, Feng Liu, Mingming Gong, Guanghui Wang, Liuhua Peng
Federated learning has emerged as the foremost approach for decentralized model training with privacy preservation. The global class imbalance and cross-client data heterogeneity naturally coexist, and the mismatch between local and global…
Efficiency
SatSplatDiff: Geometry-preserving generative refinement for high-fidelity satellite Gaussian Splatting
Jiyong Kim, Shuang Song, Ronjgun Qin
Gaussian Splatting has been recently explored for satellite 3D reconstruction, demonstrating flexibility and efficiency in representing radiometrically diverse satellite scenes. However, the limited top viewpoint of satellite imagery results in…
Additional Research
PanoImager: Geometry-Guided Novel View Synthesis and Reconstruction from Sparse Panoramic Views
Zhisong Xu, Takeshi Oishi
Panoramic sensing offers wide field-of-view coverage, yet 3D reconstruction from sparse panoramas remains challenging under rotation-dominant, weak-parallax motion. In such regimes, SfM/SLAM initialization is often ill-conditioned and unreliable….
Deviance-style normalization for jointly overdispersed counts
Akshay Balsubramani
We introduce a Dirichlet–multinomial (DM) deviance residualization for sparse, jointly overdispersed count matrices, the regime that dominates sequencing-based biochemical assays. The DM null treats each sample’s count vector as a fixed-total…
EO-WM: A Physically Informed World Model for Probabilistic Earth Observation Forecasting
Junwei Luo, Shuai Yuan, Zhenya Yang, Yansheng Li, Zhe Liu, Hengshuang Zhao
Earth Observation (EO) forecasting aims to predict future Earth surface dynamics from satellite observations under changing meteorological conditions. In this paper, we view this task as a partially observed, weather-driven world modeling problem,…
From Celebrities to Anyone: Characterizing AI Nudification Content, Technology, and Community Dynamics on 4chan
Chi Cui, Yixin Wu, Yang Zhang
AI nudification uses generative models to create synthetic non-consensual sexually explicit imagery (SNEACI) of real individuals. Prior work has examined dedicated nudification platforms and model repositories, finding that most targets are female…
Safe Autoregressive Image Generation with Iterative Self-Improving Codebooks
Yunqi Xue, Zhijiang Li, Philip Torr, Jindong Gu
Unlike diffusion-based models that operate in continuous latent spaces, autoregressive unified multimodal models produce images by sequentially predicting discretized visual tokens. These tokens are derived from a codebook that maps embeddings to…
How to evaluate clustering with ground truth?
Pasi Fränti
External indexes can be used for cluster evaluation when ground truth is available. We review the most common external validity indexes focusing on set-matching-based measures. We recommend centroid index (CI), because it is an intuitive…
Adaptive Utility driven Resource Orchestration for Resilient AI (AURORA-AI)
Rahul Umesh Mhapsekar, Ilias Cherkaoui, Lizy Abraham, Indrakshi Dey
Modern AI systems are increasingly deployed under non-stationary computational, demographic, and operational conditions in which static resource allocation strategies degrade both predictive performance and human-centric properties such as fairness…
Event-Aware Instructed Assistant for Referring Video Segmentation
Jinyu Liu, Henghui Ding, Shuting He, Yu-Gang Jiang
Existing referring video segmentation methods often treat a video as a single event consisting of multiple images, overlooking the fact that a video typically contains multiple distinct events. Under such a mechanism, the model needs to directly…
Recovering Governing Equations from Solution Data: Identifiability Bounds for Linear and Nonlinear ODEs
Yang Pan, Helmut Bölcskei
Learning governing equations from observed solution data is a fundamental challenge in scientific machine learning…
fTNN: a tensor neural network for fractional PDEs
Qingkui Ma, Hehu Xie, Xiaobo Yin
We develop the fTNN, a deterministic tensor neural network subspace method for problems involving the fractional Laplacian on bounded domains, taking the fractional Poisson equation and time-dependent fractional advection-diffusion equation as…
Uncertainty quantification via conformal prediction in data assimilation
Catherine George, Alireza Javanmardi, Tijana Janjić, Eyke Hüllermeier
Quantifying the evolution of uncertainty is critical to both probabilistic forecasting and data assimilation in numerical weather prediction. In this study, we investigate the applicability of conformal prediction (CP), a recent machine learning…
From Vajrayana Tara to Bengali Baul: A Computational Study of Lexical Transmission Across Buddhist, Shakta, and Vaishnava Traditions in Bengal
Joy Bose
We present a computational corpus study of vocabulary relationships across eight tradition layers of Bengali and Sanskrit devotional literature spanning the 8th to 19th centuries, encompassing Buddhist Vajrayana, Shakta Tantra, Vaishnava, and Baul…
ConvMemory v3: A Validity Context Layer for Conversational Memory via Target-Conditioned Relation Verification
Taiheng Pan
Conversational memory retrieval optimizes relevance, yet a retrieved memory can be relevant and simultaneously outdated: a later turn updates, corrects, or supersedes it. ConvMemory v3 adds a validity context layer that detects and surfaces this…
PhysiFormer: Learning to Simulate Mechanics in World Space
Yiming Chen, Yushi Lan, Andrea Vedaldi
We present PhysiFormer, a diffusion transformer for physically-plausible 3D object motion. Unlike video world models that operate in view-dependent pixel space, PhysiFormer represents objects as 3D meshes expressed in world coordinates. Given the…
RayPE: Ray-Space Positional Encoding for 3D-Aware Video Generation
Minghao Yin, Jiahao Lu, Wenbo Hu, Wang Zhao, Shan Ying, Kai Han
Modern video diffusion transformers position their tokens through RoPE on the (u,v,t) axes — a description of the camera’s sampling grid that says nothing about the 3D structure of the scene. We observe that the geometric relation between two…
Pseudo-Text-Conditioned 3D Grounding DINO for Organ Localization in Abdominal CT
Siqi Chen, Han Gong, Keyi Hou, Jingxuan Yang, Sheethal Bhat, Andreas Maier
Reliable organ localization in abdominal CT can provide spatial priors for downstream trauma analysis. We propose CT-3GDINO, a lightweight 3D detector that adapts a Grounding-DINO-style query-based architecture to fixed organ localization using…
Tractography-Driven Synthetic Data Generation for Fiber Bundle Segmentation in Tracer Histology
Kyriaki-Margarita Bintsi, Sparsh Makharia, Yaël Balbastre, Joselyn Romero Avila, Julia F. Lehman, Su
Diffusion MRI (dMRI) tractography enables non-invasive reconstruction of white-matter pathways, but its accuracy is fundamentally limited by indirect, low-resolution measurements of axonal organization. Tracer injection studies in non-human…
Continual Robot Policy Learning via Variational Neural Dynamics
Jiaxu Xing, Zhiyuan Zhu, Yunfan Ren, Ismail Geles, Yifan Zhai, Rudolf Reiter, Davide Scaramuzza
Robots deployed in the real world rarely operate under a single fixed dynamics model: wind changes, payloads vary, batteries drain, contacts shift, and hardware wears. Yet most learning-based controllers are trained once and deployed as if learning…
Humanoid-DART: Humanoid Loco-Manipulation using Diffusion-guided Augmentation through Relabeling and Tracking
Pranav Debbad, Kanish Thiagarajan, Victor Dhédin, Shafeef Omar, Majid Khadiv
Imitating human demonstrations has emerged as a dominant paradigm for learning humanoid loco-manipulation policies. However, scaling these approaches remains challenging due to the high cost of collecting diverse demonstrations and the need for…
All you need is log
Akshay Balsubramani
Comparing two probability distributions is a basic building block of statistics and machine learning, and the right family is well understood: the Rényi divergences of order $α\in[0,\infty]$ are the unique family monotone under data processing and…
When Does Synthetic Data Augmentation Improve Score-Based Imbalanced Classification?
Zhengchi Ma, Pengfei Lyu, Anru R. Zhang
Synthetic data augmentation is widely used to mitigate class imbalance, but its theoretical effects on score-based classification remain poorly understood. This paper develops a framework for characterizing when synthetic minority augmentation can…
Positivity of a Hadamard Product
Roger A. Horn, Shengxuan Luo, Hongwei Xu, Zai Yang
A notable difference between the ordinary and Hadamard products is that the Hadamard product of two singular positive semidefinite matrices can be nonsingular, and one of the factors can even be indefinite. We present an eigenvalue lower bound for…
Architecture as physical prior: cooperative neural network for nuclear masses
Peiwen Zai, Wei Cheng, Feng-Shou Zhang
Machine learning approaches to nuclear mass prediction have achieved remarkable accuracy, but typically rely on existing theoretical baselines or hand-crafted physics features. Here we demonstrate that these prerequisites can be supplanted by…
UniDriveDreamer: A Single-Stage Multimodal World Model for Autonomous Driving
Guosheng Zhao, Yaozeng Wang, Xiaofeng Wang, Zheng Zhu, Tingdong Yu, Guan Huang, Yongchen Zai, Ji Jia
World models have demonstrated significant promise for data synthesis in autonomous driving. However, existing methods predominantly concentrate on single-modality generation, typically focusing on either multi-camera video or LiDAR sequence…
GalaxyDiT: Efficient Video Generation with Guidance Alignment and Adaptive Proxy in Diffusion Transformers
**
Editing Physiological Signals in Videos Using Latent Representations
**
—
Looking Ahead
The pace of AI research shows no signs of slowing. Stay tuned for next week’s digest covering the latest breakthroughs.
This digest is part of the Frontier AI Research Brief series, covering the most significant AI research each week.
Leave a Reply