math.OC 1905.10874

RSN: Randomized Subspace Newton

RSN solves Newton systems in random subspaces and usually outpaces GD, AGD, and full Newton across six datasets.

Robert M. Gower, Dmitry Kovalev, Felix Lieder et al.

2019-05-27 20
cs.CL 1905.10650

Are Sixteen Heads Really Better than One?

This study reveals that many attention heads in multi-head attention are redundant; introduces gradient-based greedy pruning to improve efficiency.

Paul Michel, Omer Levy, Graham Neubig

2019-05-26 1481 citations 38
cs.CL 1905.07830

HellaSwag: Can a Machine Really Finish Your Sentence?

HellaSwag dataset leverages adversarial filtering to challenge state-of-the-art models, exposing their limitations in commonsense reasoning with only ~48% accuracy versus 95% humans.

Rowan Zellers, Ari Holtzman, Yonatan Bisk et al.

2019-05-20 25