This paper proves that invalid action masking corresponds to valid policy gradients, scaling efficiently in large action spaces, outperforming penalty methods.

Shengyi Huang, Santiago Ontañón

Proposes NPAS and SSNs for automatic parameter-efficient neural network training under arbitrary budgets.

Bryan A. Plummer, Nikoli Dryden, Julius Frost et al.

This paper introduces semi-supervised learning with Big Self-Supervised Models (SimCLRv2), achieving 73.9% top-1 accuracy on ImageNet with only 1% labels, outperforming previous methods by 10x.

Ting Chen, Simon Kornblith, Kevin Swersky et al.

Proposes PipeDream-2BW, a memory-efficient pipeline training system for large-scale DNNs, achieving up to 20× speedup via double-buffered weight updates and automated model partitioning.

Deepak Narayanan, Amar Phanishayee, Kaiyu Shi et al.

AWAC combines offline datasets with online fine-tuning, boosting robotic skill learning efficiency.

Ashvin Nair, Abhishek Gupta, Murtaza Dalal et al.

Proposed Graph Substructure Networks (GSN) enhance GNN expressivity by counting subgraph isomorphisms, surpassing WL limitations.

Giorgos Bouritsas, Fabrizio Frasca, Stefanos Zafeiriou et al.

Proposed improved training techniques for score-based generative models, achieving high-resolution image generation comparable to GANs.

Yang Song, Stefano Ermon

Introduces Maximal Coding Rate Reduction (MCR²) principle to maximize the difference between total and class-specific coding rates, enhancing discriminability and diversity.

Yaodong Yu, Kwan Ho Ryan Chan, Chong You et al.

Introduces adaptive GPR-GNN with learnable weights, improving node classification on both homophilic and heterophilic graphs.

Eli Chien, Jianhao Peng, Pan Li et al.

BYOL employs dual networks with a momentum target to learn image representations without negative samples, achieving 74.3% top-1 accuracy on ImageNet.

Jean-Bastien Grill, Florian Strub, Florent Altché et al.

Proposes PELEG, an explicit algorithm achieving instance-optimal sample complexity in linear bandits via no-regret strategies.

Mohammadi Zaki, Avi Mohan, Aditya Gopalan

FrugalML framework optimizes API usage to achieve higher accuracy and significant cost savings within budget constraints.

Lingjiao Chen, Matei Zaharia, James Zou

Ansor uses hierarchical search combined with evolutionary algorithms to generate high-performance tensor programs, achieving up to 3.8× speedup.

Lianmin Zheng, Chengfan Jia, Minmin Sun et al.

Linformer reduces self-attention complexity from O(n²) to O(n) using low-rank approximation, enabling efficient long-sequence modeling.

Sinong Wang, Belinda Z. Li, Madian Khabsa et al.

Performer with FAVOR achieves linear complexity in long protein sequence modeling, enabling efficient large-scale biological analysis.

Krzysztof Choromanski, Valerii Likhosherstov, David Dohan et al.

Analyzes the impact of mild over-parameterization on the optimization landscape of shallow ReLU networks, revealing local non-convexity but global favorable properties, with theoretical guarantees.

Itay Safran, Gilad Yehudai, Ohad Shamir

This paper advances Orthogonal Monte Carlo (OMC) theory by applying negative dependence, deriving exponential error bounds, and introduces Near-Orthogonal Monte Carlo (NOMC) for superior high-dimensional sampling.

Han Lin, Haoxian Chen, Tianyi Zhang et al.

MOPO optimizes offline policies using dynamics uncertainty penalties, excelling on D4RL benchmarks.

Tianhe Yu, Garrett Thomas, Lantao Yu et al.

This study analyzes implementation details of PPO and TRPO, revealing that code optimizations significantly boost performance.

Logan Engstrom, Andrew Ilyas, Shibani Santurkar et al.

Optimizing alignment and uniformity in contrastive loss enhances representation learning performance.

Tongzhou Wang, Phillip Isola