Mining commonsense knowledge using pretrained models without updating weights, outperforming supervised methods.

Joshua Feldman, Joe Davison, Alexander M. Rush

Using DNN-generated response patterns (RP) to fit IRT models, revealing task difficulty with high correlation to human data.

John P. Lalor, Hao Wu, Hong Yu

LXMERT employs a multi-encoder Transformer architecture with multi-task pretraining, achieving state-of-the-art results on VQA, GQA, and NLVR2 datasets.

Hao Tan, Mohit Bansal

Constructed PubLayNet with 360k images via XML-PDF matching, enabling accurate document layout recognition and transfer learning.

Xu Zhong, Jianbin Tang, Antonio Jimeno Yepes

Proposes KBRD, an end-to-end knowledge-enhanced recommendation dialogue system, improving recommendation accuracy by 0.7 and dialogue diversity using R-GCN and Transformer.

Qibin Chen, Junyang Lin, Yichang Zhang et al.

This paper proves attention weights in Transformers are non-identifiable for sequences longer than head dimension, introduces effective attention, and shows token identity is mainly encoded in embedding angles.

Gino Brunner, Yang Liu, Damián Pascual et al.

Proposed ReDR network uses dynamic reasoning and reinforcement learning to improve conversational question generation, outperforming baselines on CoQA.

Boyuan Pan, Hao Li, Ziyu Yao et al.

This study leverages pre-trained BERT, GPT-2, RoBERTa checkpoints in a Transformer seq2seq model, achieving state-of-the-art results in translation and summarization.

Sascha Rothe, Shashi Narayan, Aliaksei Severyn

TREQS model uses deep learning to generate SQL queries, enhancing question answering efficiency on electronic medical records.

Ping Wang, Tian Shi, Chandan K. Reddy

ELI5 introduces a large-scale long-form QA dataset with multi-source web data, trained with multi-task Seq2Seq models outperforming traditional baselines.

Angela Fan, Yacine Jernite, Ethan Perez et al.

BAM employs multi-task distillation with teacher annealing, achieving >1.5% improvement on GLUE benchmarks over standard methods.

Kevin Clark, Minh-Thang Luong, Urvashi Khandelwal et al.

Introduces Product Keys-based structured memory layer, enabling billion-parameter capacity with efficient exact search, outperforming deeper transformers in language modeling.

Guillaume Lample, Alexandre Sablayrolles, Marc'Aurelio Ranzato et al.

Proposes a collaborative referring task under continuous, partially observable context, with a large-scale dataset of 6,760 dialogues to evaluate neural models.

Takuma Udagawa, Akiko Aizawa

Saliency-based interpretation method reveals word alignments in NMT, outperforming attention weights with up to 20% AER reduction, applicable offline and online.

Shuoyang Ding, Hainan Xu, Philipp Koehn

Neural latent variable model for open domain event extraction, outperforming state-of-the-art on large-scale news datasets.

Xiao Liu, Heyan Huang, Yue Zhang

This study introduces a deep learning-based personalized persuasive dialogue system, achieving 59.6% F1 on a 300-sample annotated dataset, advancing social good applications.

Xuewei Wang, Weiyan Shi, Richard Kim et al.

This study analyzes BERT's attention heads, revealing their alignment with syntactic relations, and demonstrates dependency parsing with 77% accuracy using attention maps.

Kevin Clark, Urvashi Khandelwal, Omer Levy et al.

BIGPATENT dataset offers 1.3 million patent abstracts, advancing high-compression abstractive summarization.

Eva Sharma, Chen Li, Lu Wang

Proposes substantiated perspective discovery task; constructs PERSPECTRUM dataset with 1000 claims, 10,000 perspectives, 8,000 evidence paragraphs; baseline models outperform simple methods but lag behind humans.

Sihao Chen, Daniel Khashabi, Wenpeng Yin et al.

Proposes email subject line generation using a two-stage extractive-abstractive model, trained on AESLC, outperforming baselines with ROUGE-1 score of 25.41.

Rui Zhang, Joel Tetreault