Proposed a salience-aware learning method for table-based fact verification, achieving SOTA on TabFact.

Fei Wang, Kexuan Sun, Jay Pujara et al.

This paper introduces instruction tuning on a 137B parameter model, significantly improving zero-shot performance across 60 NLP tasks, outperforming GPT-3 on many benchmarks.

Jason Wei, Maarten Bosma, Vincent Y. Zhao et al.

Introduces MULTI-EURLEX, a multilingual, multi-label legal dataset, and explores zero-shot cross-lingual transfer using models like XLM-ROBERTA and MT5 with adaptation strategies.

Ilias Chalkidis, Manos Fergadiotis, Ion Androutsopoulos

Introduces FinQA, a financial QA dataset with expert-annotated multi-step numerical reasoning, highlighting model performance gaps.

Zhiyu Chen, Wenhu Chen, Charese Smiley et al.

ALiBi introduces linear distance biases in attention scores, enabling models trained on short sequences to extrapolate to longer inputs efficiently.

Ofir Press, Noah A. Smith, Mike Lewis

This paper explores sentence embeddings from T5, proposing three extraction methods, outperforming Sentence-BERT and SimCSE, with scaling improving performance.

Jianmo Ni, Gustavo Hernández Ábrego, Noah Constant et al.

A systematic survey of prompting methods in NLP, detailing models, prompt design, answer engineering, and experimental validation with specific datasets.

Pengfei Liu, Weizhe Yuan, Jinlan Fu et al.

TAPEX learns a neural SQL executor from synthetic SQL-output pairs and sets SOTA on WikiSQL, WTQ, SQA, and TabFact.

Qian Liu, Bei Chen, Jiaqi Guo et al.

Enhancing dialogue faithfulness with controllable features using GPT-2 and T5, improving objectivity and informativeness.

Hannah Rashkin, David Reitter, Gaurav Singh Tomar et al.

This survey comprehensively reviews recent advances in dialogue summarization, including datasets, models, and future directions, emphasizing multi-domain and multi-modal techniques.

Xiachong Feng, Xiaocheng Feng, Bing Qin

Proposes Adapt-and-Distill, combining vocabulary expansion and knowledge distillation to develop small, efficient domain-specific models outperforming BERT BASE.

Yunzhi Yao, Shaohan Huang, Wenhui Wang et al.

KaggleDBQA leverages real-world databases and documentation, boosting Text-to-SQL accuracy by over 13.2% in practical settings.

Chia-Hsuan Lee, Oleksandr Polozov, Matthew Richardson

PALMS uses small curated datasets to steer large language models toward societal values, reducing bias and toxicity effectively.

Irene Solaiman, Christy Dennison

LoRA introduces low-rank matrices to freeze pre-trained weights, reducing trainable parameters by 10,000x, with performance comparable or better than full fine-tuning.

Edward J. Hu, Yelong Shen, Phillip Wallis et al.

Sequence-level training for NAT using reinforcement algorithms and BoN loss, achieving 3-4 BLEU points improvement on WMT tasks.

Chenze Shao, Yang Feng, Jinchao Zhang et al.

EMDR2 introduces end-to-end training of multi-document retriever and reader using EM algorithm, improving accuracy by 2-3 points on benchmarks.

Devendra Singh Sachan, Siva Reddy, William Hamilton et al.

This paper evaluates adapter-based tuning for pretrained language models, showing superior performance in low-resource and cross-lingual tasks, with better regularization and robustness.

Ruidan He, Linlin Liu, Hai Ye et al.

This study evaluates the relationship between perplexity and human-like reading behavior across Japanese and English, revealing language-specific differences.

Tatsuki Kuribayashi, Yohei Oseki, Takumi Ito et al.

SEM-TAB-FACTS introduces a new dataset and tasks for fact verification in scientific tables, featuring 180K+ statements and 16M+ evidence annotations.

Nancy X. R. Wang, Diwakar Mahajan, Marina Danilevsky et al.

ByT5, a Transformer-based byte-level pre-trained model, matches token-based performance, improves robustness, with five model sizes and innovative training strategies.

Linting Xue, Aditya Barua, Noah Constant et al.