Using random label/noise experiments, reveals deep networks' capacity to memorize, challenging classical generalization theories.

Chiyuan Zhang, Samy Bengio, Moritz Hardt et al.

Proposes Equation Learner (EQL), a neural network that learns analytical expressions and excels in extrapolation beyond training data.

Georg Martius, Christoph H. Lampert

Deep ReLU networks more efficiently approximate smooth functions in Sobolev spaces than shallow networks.

Dmitry Yarotsky

Combining deep video prediction with MPC enables robots to manipulate unseen objects without calibration or physical models.

Chelsea Finn, Sergey Levine

SeqGAN combines GAN with policy gradient reinforcement learning to generate high-quality discrete sequences, overcoming gradient issues.

Lantao Yu, Weinan Zhang, Jun Wang et al.

PGA and SGLD poison MovieLens factorization recommenders; at β=0.6, SGLD reaches detection-test p-values above 0.7.

Bo Li, Yining Wang, Aarti Singh et al.

Using the Polyak-Lojasiewicz (PL) inequality, this paper proves linear convergence of gradient and proximal gradient methods for non-strong convex problems.

Hamed Karimi, Julie Nutini, Mark Schmidt

Combines VAE-GAN embedding with action-conditioned RNNs for realistic highway video prediction, maintaining scene coherence over 100 frames.

Eder Santana, George Hotz

Proposes data augmentation, temporal shift adaptation, distillation, and embedding prediction to improve RNN session recommendation, achieving 12.8% and 14.8% gains.

Yong Kiam Tan, Xinxing Xu, Yong Liu

Proposes Wide & Deep model combining linear and neural networks, boosting Google Play app recommendation performance with +3.9% online acquisition.

Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen et al.

Matching Networks enable one-shot learning, boosting ImageNet accuracy to 93.2%, Omniglot to 93.8%.

Oriol Vinyals, Charles Blundell, Timothy Lillicrap et al.

Proposes advanced techniques like feature matching and virtual batch normalization to improve GAN stability and image quality, achieving state-of-the-art semi-supervised results.

Tim Salimans, Ian Goodfellow, Wojciech Zaremba et al.

A-LTM employs knowledge distillation to retain old tasks during sequential learning, addressing catastrophic forgetting with a dual-system architecture.

Tommaso Furlanello, Jiaping Zhao, Andrew M. Saxe et al.

OpenAI Gym standardizes RL environments via a common API and versioning, starting with five benchmark families.

Greg Brockman, Vicki Cheung, Ludwig Pettersson et al.

Proposes an unsupervised video prediction model based on pixel motion, generalizing to unseen objects, trained on 59,000 robot interactions with 1.5 million frames.

Chelsea Finn, Ian Goodfellow, Sergey Levine

Extends Rademacher contraction inequality to vector-valued functions; introduces a vector contraction bound replacing Gaussian with symmetric sub-Gaussian variables, applicable to multi-class learning, K-means, and meta-learning.

Andreas Maurer

Introduces a benchmark suite of 31 continuous control tasks, systematically evaluating deep RL algorithms, revealing strengths and limitations across diverse scenarios.

Yan Duan, Xi Chen, Rein Houthooft et al.

Proposes an O(√n) memory training algorithm for deep networks, with only one extra forward pass, enabling training of 1000-layer ResNets on limited hardware.

Tianqi Chen, Bing Xu, Chiyuan Zhang et al.

CMA-ES is an adaptive covariance matrix evolution strategy that efficiently solves high-dimensional, non-linear, non-convex optimization problems, outperforming traditional methods.

Nikolaus Hansen

Weight normalization reparameterizes weights as w = g * (v / ||v||), accelerating training and improving convergence.

Tim Salimans, Diederik P. Kingma