Proposes DistIL, a distributional imitation learning algorithm with monotonic improvement guarantees, leveraging rich feedback for complex reasoning tasks.

Rishabh Agrawal, Jacob Fein-Ashley, Paria Rashidinejad

Flash-WAM uses modality-aware step distillation to reduce inference from 50 steps to 1, achieving 23× speedup while maintaining high task success rates.

Arman Akbari, Ci Zhang, Arash Akbari et al.

Curvature-aware dynamic precision control using L-BFGS curvature info balances PINN accuracy and efficiency.

Yingjie Shao, Ioannis N. Athanasiadis, George van Voorn et al.

LoopMoE combines sparse routing with iterative computation, improving performance across 9 benchmarks.

Wenkai Chen, Tianshu Li, Wenyong Huang et al.

Synthetic benchmarks control data properties to evaluate concept bottleneck models' robustness and interpretability.

Julian Skirzynski, Harry Cheon, Shreyas Kadekodi et al.

dMX employs differentiable gradient-based optimization to assign mixed floating-point precisions, balancing accuracy and efficiency in large language models.

Giuseppe Franco, Ian Colbert, Pablo Monteagudo-Lago et al.

Skill-RM unifies heterogeneous evaluation criteria via agent skills, enabling dynamic resource orchestration, outperforming traditional judges with a 3-6% improvement on RewardBench2.

Tao Chen, Gangwei Jiang, Pengyu Cheng et al.

Introducing the 'Sleep' paradigm with Knowledge Seeding and Dreaming mechanisms enables LLMs to self-modify and consolidate memories for continual learning.

Ali Behrouz, Farnoosh Hashemi, Vahab Mirrokni

Proposes ROSA, a reward distribution-based framework for inducing diverse behaviors without performance loss, leveraging set functions and unbiased gradient estimators.

Anthony GX-Chen, Ankit Anand, Gheorghe Comanici et al.

Combining linear probes and generation probabilities, this work reveals the world model recovery capabilities of fine-tuned LLMs in classical planning.

Patrick Emami, Nan Qiang, Peter Graf

This paper introduces an equilibrium propagation (EP)-based training method for deep predictive coding networks (PCNs), achieving 13.23% Top-5 error on ImageNet with a 10-layer VGG model, close to the 12.2% baseline of backpropagation.

Tugdual Kerjan, Rasmus Høier, Benjamin Scellier

Flicker-DDPM accelerates sampling by 3.33× using 1/f colored noise while improving generation quality.

KeXiang Mao, FanCheng Li

LiftQuant enables continuous bit-width control via high-dimensional projection, compressing a 70B model to 2.4 bits with superior performance.

Liulu He, XuanAng Liu, Juntao Liu et al.

Qift proposes fixed no-zero W2 level sets based on Gaussian-like weight distribution, significantly improving LLaMA W2A4 inference performance.

Chi-Wei Huang, Chia-Chi Tsai

Proposes a local perturbation theory revealing low-dimensional conflict subspaces in multi-domain RL, with short refreshes improving performance by geometric contraction.

Lei Yang, Siyu Ding, Deyi Xiong

TrOPD employs trust-region strategies and multiple KL estimators to stabilize on-policy distillation, outperforming SOTA with +6.18 performance points.

Xingrun Xing, Haoqing Wang, Boyan Gao et al.

LeAP learns permutation-based gates and removes 3,600+ dimensions from a 12,000+ dimensional industrial model with zero business-metric degradation.

Yihong Huang, Chen Chu, Fei Chen et al.

Proposed a decision-focused on-policy learning method for contextual linear optimization with partial feedback, showing lower cumulative regret than baselines.

Wyame Benslimane, Tinghan Ye, Pascal Van Hentenryck et al.

The paper argues LLMs should learn personalized rather than aggregated preferences, citing social choice theory and experimental data.

Cristina Garbacea

GNMR controls stability in low-precision language model training by comparing gradient norms with historical means.

Boao Kong, Weichen Jia, Engao Zhang et al.