cs.CV 2106.14156

Post-Training Quantization for Vision Transformer

Proposed a post-training quantization algorithm for vision transformers, achieving 81.29% top-1 accuracy on ImageNet with DeiT-B model.

Zhenhua Liu, Yunhe Wang, Kai Han et al.

2021-06-27 14
cs.LG 2106.13948

Core Challenges in Embodied Vision-Language Planning

Unified EVLP taxonomy, analyzing algorithms, datasets, and challenges; emphasizing model generalization and real-world deployment.

Jonathan Francis, Nariaki Kitamura, Felix Labelle et al.

2021-06-26 65
cs.CV 2106.13230

Video Swin Transformer

Video Swin Transformer uses local spatiotemporal attention, achieving 84.9% top-1 accuracy on Kinetics-400 with 28.2M parameters, outperforming global attention models.

Ze Liu, Jia Ning, Yue Cao et al.

2021-06-25 49
cs.LG 2106.12575

Weisfeiler and Lehman Go Cellular: CW Networks

Proposes CW networks with cell complexes, surpassing WL test, for enhanced graph expressivity, especially in molecular graphs.

Cristian Bodnar, Fabrizio Frasca, Nina Otter et al.

2021-06-24 44
stat.ML 2106.12034

Pure Exploration in Kernel and Neural Bandits

Adaptive embedding in kernel and neural bandits achieves sample complexity depending only on effective dimension.

Yinglun Zhu, Dongruo Zhou, Ruoxi Jiang et al.

2021-06-23 42
cs.CV 2106.10823

3D Object Detection for Autonomous Driving: A Survey

This survey reviews 3D object detection methods for autonomous driving, emphasizing multi-modal fusion, datasets, and recent advances, with PV-RCNN achieving 82.86% mAP on KITTI.

Rui Qian, Xin Lai, Xirong Li

2021-06-21 49
cs.CL 2106.09685

LoRA: Low-Rank Adaptation of Large Language Models

LoRA introduces low-rank matrices to freeze pre-trained weights, reducing trainable parameters by 10,000x, with performance comparable or better than full fine-tuning.

Edward J. Hu, Yelong Shen, Phillip Wallis et al.

2021-06-18 22282 citations 42
cs.LG 2106.08441

Online Learning with Uncertain Feedback Graphs

Proposes algorithms Exp3-IP and Exp3-GR for online learning with uncertain feedback graphs, achieving sublinear regret bounds under mild conditions.

Pouya M Ghari, Yanning Shen

2021-06-16 4 citations 40
cs.AI 2106.07139

Pre-Trained Models: Past, Present and Future

This paper reviews the evolution of pre-trained models (PTMs), focusing on architectures like BERT and GPT, highlighting their transformative impact on AI.

Xu Han, Zhengyan Zhang, Ning Ding et al.

2021-06-14 52