cs.LG 2007.14062

Big Bird: Transformers for Longer Sequences

BigBird introduces sparse attention with global, local, and random tokens, reducing complexity from quadratic to linear, enabling long sequence modeling.

Manzil Zaheer, Guru Guruganesh, Avinava Dubey et al.

2020-07-28 55
cs.CV 2007.08501

Accelerating 3D Deep Learning with PyTorch3D

PyTorch3D introduces modular, differentiable rendering and operators, achieving up to 10× speedup in 3D deep learning tasks on ShapeNet.

Nikhila Ravi, Jeremy Reizenstein, David Novotny et al.

2020-07-17 48
cs.CV 2007.08336

Event Enhanced High-Quality Image Recovery

Proposes eSL-Net, a sparse learning framework, achieving 7-12dB PSNR improvement for high-quality event camera image reconstruction.

Bishan Wang, Jingwei He, Lei Yu et al.

2020-07-16 152 citations 46
cs.CV 2007.05667

To Filter Prune, or to Layer Prune, That Is The Question

Proposes LayerPrune framework, using layer-level pruning based on importance metrics to surpass filter pruning in latency reduction while maintaining comparable accuracy.

Sara Elkerdawy, Mostafa Elhoushi, Abhineet Singh et al.

2020-07-11 47 citations 35