Diffusion-Convolutional Neural Networks (DCNN) leverage graph diffusion matrices for node classification, achieving 86-90% accuracy on benchmark datasets.

James Atwood, Don Towsley

BinaryConnect trains DNNs with binary weights during propagation, reducing multiplications to simple additions, achieving near state-of-the-art accuracy on MNIST, CIFAR-10, SVHN.

Matthieu Courbariaux, Yoshua Bengio, Jean-Pierre David

Deep spatial autoencoders extract environment feature points for visuomotor learning, combined with trajectory optimization for robotic control.

Chelsea Finn, Xin Yu Tan, Yan Duan et al.

Deep deterministic policy gradient (DDPG) enables stable, end-to-end reinforcement learning in high-dimensional continuous action spaces using deep neural networks.

Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel et al.

Proposes a Wasserstein distance-based loss for multi-label learning, utilizing entropic regularization for efficient approximation, enhancing semantic smoothness.

Charlie Frogner, Chiyuan Zhang, Hossein Mobahi et al.

Proposes spectral networks with graph estimation for non-Euclidean data, matching or surpassing Dropout networks with fewer parameters.

Mikael Henaff, Joan Bruna, Yann LeCun

Permutation search is effective, but PP-Index/MI-file are not always faster than direct search.

Bilegsaikhan Naidan, Leonid Boytsov, Eric Nyberg

Proposes end-to-end training of deep visuomotor policies using Guided Policy Search with a 92,000-parameter CNN for direct image-to-torque mapping.

Sergey Levine, Chelsea Finn, Trevor Darrell et al.

K-FAC approximates Fisher matrix via Kronecker decomposition, enabling faster natural gradient optimization in neural networks.

James Martens, Roger Grosse

LINE efficiently embeds large-scale networks by optimizing first- and second-order proximities with edge sampling, handling millions of nodes and billions of edges.

Jian Tang, Meng Qu, Mingzhe Wang et al.

Exp3.G classifies feedback graphs: strongly observable gives ~√(αT), weakly observable ~δ^(1/3)T^(2/3), and unobservable Θ(T).

Noga Alon, Nicolò Cesa-Bianchi, Ofer Dekel et al.

Proposes a meta-algorithm (SAOL) transforming low-regret algorithms into strongly adaptive ones, ensuring near-optimal performance on every interval with \( O(\log T) \) overhead.

Amit Daniely, Alon Gonen, Shai Shalev-Shwartz

TRPO (Trust Region Policy Optimization) guarantees monotonic policy improvement using KL constraints, excelling in large neural network policy training for robotics and Atari games.

John Schulman, Sergey Levine, Philipp Moritz et al.

Proposes a multi-layer LSTM encoder-decoder framework for unsupervised video representation learning, improving action recognition especially with limited labeled data.

Nitish Srivastava, Elman Mansimov, Ruslan Salakhutdinov

MADE uses masked autoencoders to enforce autoregressive constraints, boosting distribution estimation efficiency.

Mathieu Germain, Karol Gregor, Iain Murray et al.

Proposes Batch Normalization (BN) to standardize layer inputs, accelerating training and reducing internal covariate shift, achieving 4.9% top-5 error on ImageNet.

Sergey Ioffe, Christian Szegedy

Introduced a visual attention-based image captioning model achieving state-of-the-art performance on MS COCO.

Kelvin Xu, Jimmy Ba, Ryan Kiros et al.

Deep Fried ConvNets replace fully connected layers with adaptive Fastfood transforms, reducing parameters by over 90% without accuracy loss.

Zichao Yang, Marcin Moczulski, Misha Denil et al.

Adam combines first and second moment estimates for adaptive learning rates, accelerating large-scale stochastic optimization.

Diederik P. Kingma, Jimmy Ba

Elastic Averaging SGD (EASGD) leverages elastic force to enable efficient distributed deep learning, reducing communication overhead while enhancing exploration, leading to faster convergence and better accuracy.

Sixin Zhang, Anna Choromanska, Yann LeCun