cs.CL 2004.09095

The State and Fate of Linguistic Diversity and Inclusion in the NLP World

This paper analyzes global linguistic resource distribution, proposes a six-category classification, highlights disparities affecting multilingual NLP, and urges community focus on low-resource languages.

Pratik Joshi, Sebastin Santy, Amar Budhiraja et al.

2020-04-20 1454 citations 54
cs.CV 2004.07780

Shortcut Learning in Deep Neural Networks

Shortcut learning in deep learning affects model generalization in complex scenarios.

Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis et al.

2020-04-17 39
cs.LG 2004.06977

On Learning Rates and Schrödinger Operators

Using spectral analysis of Schrödinger operators, the paper reveals that SGD's linear convergence rate depends exponentially on the learning rate, especially in nonconvex optimization.

Bin Shi, Weijie J. Su, Michael I. Jordan

2020-04-15 37
cs.CL 2004.05150

Longformer: The Long-Document Transformer

Introduces Longformer with linear attention combining local windows and global tokens, enabling efficient long document modeling surpassing RoBERTa.

Iz Beltagy, Matthew E. Peters, Arman Cohan

2020-04-11 37