LeanDojo introduces ReProver, a retrieval-augmented language model trained on 98,734 theorems, achieving 51.2% proof success with only one week GPU training.

Kaiyu Yang, Aidan M. Swope, Alex Gu et al.

GKD (Generalized Knowledge Distillation) leverages self-generated samples and teacher feedback, significantly boosting T5 model performance on summarization, translation, and reasoning tasks.

Rishabh Agarwal, Nino Vieillard, Yongchao Zhou et al.

iPDP extends PDP for dynamic modeling, adapting to concept drift.

Maximilian Muschalik, Fabian Fumagalli, Rohit Jagtani et al.

VoxMol employs a score-based generative framework using voxel grids, enabling fast and realistic 3D molecule synthesis for drug discovery.

Pedro O. Pinheiro, Joshua Rackers, Joseph Kleinhenz et al.

MiROCL method excels on industrial and synthetic data, improving performance by over 30%.

Haozhe Wang, Chao Du, Panyan Fang et al.

Prodigy estimates D to optimize learning rates adaptively, outperforming D-Adaptation and nearing hand-tuned Adam in experiments.

Konstantin Mishchenko, Aaron Defazio

AdaBatAL method improves active learning efficiency with adaptive batch sizes, significantly enhancing Bayesian optimization performance.

Masaki Adachi, Satoshi Hayakawa, Martin Jørgensen et al.

Large-scale human perception study reveals poor correlation of existing metrics with perceived realism; introduces DINOv2-ViT-L/14 for improved evaluation of diffusion models.

George Stein, Jesse C. Cresswell, Rasa Hosseinzadeh et al.

This paper proves transformers can implement broad machine learning algorithms and adaptively select algorithms across tasks, with near-optimal predictive performance.

Yu Bai, Fan Chen, Huan Wang et al.

Inference-time intervention (ITI) shifts activations in select attention heads to improve LLaMA's truthfulness from 32.5% to 65.1%, using minimal data and computational cost.

Kenneth Li, Oam Patel, Fernanda Viégas et al.

Proposes nonparametric iterative machine teaching (NIMT) via functional optimization; introduces random and greedy algorithms with theoretical ITD bounds.

Chen Zhang, Xiaofeng Cao, Weiyang Liu et al.

Proposed GODE framework employs bi-level contrastive learning to fuse molecular graphs with knowledge graph substructures, boosting molecular property prediction by 12.7% ROC-AUC and 34.4% RMSE/MAE.

Pengcheng Jiang, Cao Xiao, Tianfan Fu et al.

This paper introduces the Galerkin method for spectral algorithms, outperforming graph Laplacian approaches, especially in high-dimensional and large-scale settings, with efficient structured kernel implementation.

Vivien Cabannes, Francis Bach

A geometric regularization approach to handle noise in deep constrained clustering, significantly improving accuracy.

Tri Nguyen, Shahana Ibrahim, Xiao Fu

Proposes Multi-Task Diffusion Model (MTDiff) with Transformer and prompt learning for multi-task offline RL, outperforming state-of-the-art on Meta-World and Maze2D.

Haoran He, Chenjia Bai, Kang Xu et al.

Proposes GRIT, a message-passing-free graph Transformer leveraging inductive biases, achieving state-of-the-art results on multiple benchmarks.

Liheng Ma, Chen Lin, Derek Lim et al.

Proposed FactFormer uses axial factorized kernel integral in Transformer for scalable high-dimensional PDE surrogate modeling.

Zijie Li, Dule Shu, Amir Barati Farimani

MeZO, a memory-efficient zeroth-order optimizer, enables fine-tuning of 30B models with 12× less memory, matching performance of backpropagation.

Sadhika Malladi, Tianyu Gao, Eshaan Nichani et al.

LATM framework uses GPT-4 to generate reusable Python tools, reducing inference costs significantly.

Tianle Cai, Xuezhi Wang, Tengyu Ma et al.

Proposes DPOK, a reinforcement learning framework with KL regularization for online fine-tuning of diffusion models, improving text-image alignment and image quality.

Ying Fan, Olivia Watkins, Yuqing Du et al.