Transforming recommendation into a goal-oriented dialogue game, using self-supervised Bot-Play training, significantly improves interactive recommendation performance.

Dongyeop Kang, Anusha Balakrishnan, Pararth Shah et al.

Introduces control tasks and selectivity metric to evaluate probe interpretability; finds second-layer ELMo representations are more linguistically meaningful.

John Hewitt, Percy Liang

MoverScore combines contextual embeddings and Earth Mover's Distance for high semantic correlation evaluation.

Wei Zhao, Maxime Peyrard, Fei Liu et al.

Introduces TabFact dataset and combines Table-BERT with LPA models for table-based fact verification, achieving ~65% accuracy, below human performance.

Wenhu Chen, Hongmin Wang, Jianshu Chen et al.

This study systematically evaluates pretrained language models (like BERT) for factual knowledge storage without fine-tuning, showing strong performance in relation extraction and open-domain QA.

Fabio Petroni, Tim Rocktäschel, Patrick Lewis et al.

Using geometric analysis, this study reveals that BERT, ELMo, and GPT-2 produce highly anisotropic, layer-dependent, context-sensitive word embeddings with less than 5% variance explained by static vectors.

Kawin Ethayarajh

Mining commonsense knowledge using pretrained models without updating weights, outperforming supervised methods.

Joshua Feldman, Joe Davison, Alexander M. Rush

Using DNN-generated response patterns (RP) to fit IRT models, revealing task difficulty with high correlation to human data.

John P. Lalor, Hao Wu, Hong Yu

LXMERT employs a multi-encoder Transformer architecture with multi-task pretraining, achieving state-of-the-art results on VQA, GQA, and NLVR2 datasets.

Hao Tan, Mohit Bansal

Constructed PubLayNet with 360k images via XML-PDF matching, enabling accurate document layout recognition and transfer learning.

Xu Zhong, Jianbin Tang, Antonio Jimeno Yepes

Proposes KBRD, an end-to-end knowledge-enhanced recommendation dialogue system, improving recommendation accuracy by 0.7 and dialogue diversity using R-GCN and Transformer.

Qibin Chen, Junyang Lin, Yichang Zhang et al.

This paper proves attention weights in Transformers are non-identifiable for sequences longer than head dimension, introduces effective attention, and shows token identity is mainly encoded in embedding angles.

Gino Brunner, Yang Liu, Damián Pascual et al.

Proposed ReDR network uses dynamic reasoning and reinforcement learning to improve conversational question generation, outperforming baselines on CoQA.

Boyuan Pan, Hao Li, Ziyu Yao et al.

This study leverages pre-trained BERT, GPT-2, RoBERTa checkpoints in a Transformer seq2seq model, achieving state-of-the-art results in translation and summarization.

Sascha Rothe, Shashi Narayan, Aliaksei Severyn

TREQS model uses deep learning to generate SQL queries, enhancing question answering efficiency on electronic medical records.

Ping Wang, Tian Shi, Chandan K. Reddy

ELI5 introduces a large-scale long-form QA dataset with multi-source web data, trained with multi-task Seq2Seq models outperforming traditional baselines.

Angela Fan, Yacine Jernite, Ethan Perez et al.

BAM employs multi-task distillation with teacher annealing, achieving >1.5% improvement on GLUE benchmarks over standard methods.

Kevin Clark, Minh-Thang Luong, Urvashi Khandelwal et al.

Introduces Product Keys-based structured memory layer, enabling billion-parameter capacity with efficient exact search, outperforming deeper transformers in language modeling.

Guillaume Lample, Alexandre Sablayrolles, Marc'Aurelio Ranzato et al.

Proposes a collaborative referring task under continuous, partially observable context, with a large-scale dataset of 6,760 dialogues to evaluate neural models.

Takuma Udagawa, Akiko Aizawa

Saliency-based interpretation method reveals word alignments in NMT, outperforming attention weights with up to 20% AER reduction, applicable offline and online.

Shuoyang Ding, Hainan Xu, Philipp Koehn