Released the first publicly available Clinical BERT model, improving performance on clinical NLP tasks.

Emily Alsentzer, John R. Murphy, Willie Boag et al.

Proposed HUSE framework unifies human and statistical evaluation for NLG quality and diversity.

Tatsunori B. Hashimoto, Hugh Zhang, Percy Liang

fairseq is a PyTorch-based sequence modeling toolkit supporting large-scale distributed training and multi-task applications.

Myle Ott, Sergey Edunov, Alexei Baevski et al.

SciBERT leverages scientific text pretraining to enhance scientific NLP tasks, surpassing BERT.

Iz Beltagy, Kyle Lo, Arman Cohan

Proposed competence-based curriculum learning reduces NMT training time by up to 70%, boosting BLEU scores by 2.2 points.

Emmanouil Antonios Platanios, Otilia Stretcu, Graham Neubig et al.

Study shows debiasing methods like HARD-DEBIASED and GN-GLOVE fail to fully remove gender bias in word embeddings.

Hila Gonen, Yoav Goldberg

FAST framework with backtracking achieves 17% relative gain and 6% absolute improvement on SPL in VLN tasks.

Liyiming Ke, Xiujun Li, Yonatan Bisk et al.

Supports 102 languages with Transformer-based multilingual NMT, outperforming previous models in low-resource and high-resource settings.

Roee Aharoni, Melvin Johnson, Orhan Firat

Reinforcement learning-based curriculum automatically learns sample presentation order, boosting BLEU +3.4 in single training run.

Gaurav Kumar, George Foster, Colin Cherry et al.

This study demonstrates that attention weights poorly correlate with feature importance and cannot reliably explain model predictions.

Sarthak Jain, Byron C. Wallace

Introduces a BERT-based two-stage text summarization model, achieving a ROUGE average of 33.33 on CNN/Daily Mail.

Haoyu Zhang, Jianjun Xu, Ji Wang

Proposed bi-directional dual encoder with additive margin softmax; achieved P@1 >86% on UN corpus retrieval tasks.

Yinfei Yang, Gustavo Hernandez Abrego, Steve Yuan et al.

BERT is framed as a Markov Random Field language model, generating diverse but slightly lower-quality sentences.

Alex Wang, Kyunghyun Cho

Proposed MT-DNN combines BERT pretraining with multi-task learning, achieving 82.7% on GLUE, outperforming SOTA by 2.2%.

Xiaodong Liu, Pengcheng He, Weizhu Chen et al.

Introduces an interpretable attention extension with SGD optimization, achieving near Giza++ accuracy in neural machine translation word alignment.

Thomas Zenkel, Joern Wuebker, John DeNero

Proposes a question-entailment-based QA system combining IR and deep learning, achieving 29.8% improvement on medical datasets.

Asma Ben Abacha, Dina Demner-Fushman

This study evaluates end-to-end NLG systems, highlighting seq2seq models' strengths and semantic control issues, based on 62 systems from 17 institutions.

Ondřej Dušek, Jekaterina Novikova, Verena Rieser

Proposes unsupervised and supervised cross-lingual pretraining methods (XLM), achieving state-of-the-art results on XNLI, unsupervised BLEU 34.3, supervised BLEU 38.5, surpassing prior work.

Guillaume Lample, Alexis Conneau

BERT excels in capturing English syntactic phenomena, notably in subject-verb agreement tasks.

Yoav Goldberg

Introduces a single BiLSTM encoder for 93 languages, enabling zero-shot cross-lingual transfer with strong results on multiple benchmarks.

Mikel Artetxe, Holger Schwenk