MEST framework employs Elastic Mutation and Soft Memory Bound to enable accurate, fast sparse training on edge devices, boosting Top-1 accuracy on ImageNet by over 2% at 90% sparsity.

Geng Yuan, Xiaolong Ma, Wei Niu et al.

Introduces Linear State-Space Layers (LSSL), combining RNN, convolution, and continuous-time models, excelling at long dependencies.

Albert Gu, Isys Johnson, Karan Goel et al.

Multitask prompted training (T0) achieves state-of-the-art zero-shot performance, outperforming models up to 16× larger on diverse NLP tasks.

Victor Sanh, Albert Webson, Colin Raffel et al.

Proposes LSPE architecture with learnable structural and positional encodings, boosting molecular dataset performance up to 64.14%.

Vijay Prakash Dwivedi, Anh Tuan Luu, Thomas Laurent et al.

Proposed Crystal Diffusion Variational Autoencoder (CDVAE) leverages diffusion processes and SE(3) equivariant GNNs to generate stable periodic materials, outperforming prior methods.

Tian Xie, Xiang Fu, Octavian-Eugen Ganea et al.

Proposes successor representation (SR)-based framework for automatic option discovery and composition, enhancing exploration and planning in reinforcement learning.

Marlos C. Machado, Andre Barreto, Doina Precup et al.

Carefully tuned recurrent off-policy RL matches or exceeds specialized POMDP algorithms across 18/21 environments, demonstrating strong baseline performance.

Tianwei Ni, Benjamin Eysenbach, Ruslan Salakhutdinov

Proposes SE(3) equivariant GNNs with geometric and physical info, achieving state-of-the-art in molecular and physics tasks.

Johannes Brandstetter, Rob Hesselink, Elise van der Pol et al.

Proposes 8-bit optimizers with block-wise dynamic quantization, maintaining 32-bit performance while reducing memory by 75%.

Tim Dettmers, Mike Lewis, Sam Shleifer et al.

Proposes Anomaly Transformer using association discrepancy for unsupervised time series anomaly detection, achieving SOTA results.

Jiehui Xu, Haixu Wu, Jianmin Wang et al.

XR-Transformer employs multi-resolution recursive fine-tuning, reducing training time from 23 days to 29 hours on Amazon-3M, with Prec@1 reaching 54%.

Jiong Zhang, Wei-cheng Chang, Hsiang-fu Yu et al.

Multiwavelet neural operator compresses kernels for high-accuracy PDE solutions, achieving 2-10x error reduction.

Gaurav Gupta, Xiongye Xiao, Paul Bogdan

Proposes verification error as a key metric for machine unlearning; analyzes SGD to design low-error training objectives.

Anvith Thudi, Gabriel Deza, Varun Chandrasekaran et al.

MetaDrive employs procedural generation and real data import to create diverse driving scenarios, enhancing RL generalization in autonomous driving.

Quanyi Li, Zhenghao Peng, Lan Feng et al.

Proposes neural network-based model reduction for MPM, enabling continuous deformation mapping and real-time large-scale simulation.

Peter Yichen Chen, Maurizio M. Chiaramonte, Eitan Grinspun et al.

Revealed that point adjustment (PA) overestimates TAD performance; proposed new baseline and PA%K evaluation protocol.

Siwon Kim, Kukjin Choi, Hyun-Soo Choi et al.

Introduces block movement pruning, combining it with movement pruning to accelerate Transformer models by 2.4x and reduce size by 74% while maintaining performance.

François Lagunas, Ella Charlaix, Victor Sanh et al.

EMA: ensemble-based membership inference for robust data removal auditing, outperforming KS-distance methods.

Yangsibo Huang, Xiaoxiao Li, Kai Li

Integrates formal verification with adversarial training to enhance DNN robustness, achieving 20% accuracy improvement against attacks on CIFAR-10.

Wenjie Ruan, Xinping Yi, Xiaowei Huang

Proposes neural operators combining integral kernels and nonlinear activations for PDE operator approximation, achieving universal approximation and discretization invariance.

Nikola Kovachki, Zongyi Li, Burigede Liu et al.