This survey reviews three categories of NLG evaluation metrics: human, automatic, and machine-learned, analyzing their strengths, weaknesses, and future directions.

Asli Celikyilmaz, Elizabeth Clark, Jianfeng Gao

Proposed neural models (Transformers and multilingual RNNs) with data augmentation achieved over 90% accuracy across 45 typologically diverse languages.

Ekaterina Vylomova, Jennifer White, Elizabeth Salesky et al.

wav2vec 2.0 achieves speech recognition via self-supervised learning, reaching 1.8/3.3 WER.

Alexei Baevski, Henry Zhou, Abdelrahman Mohamed et al.

CRISS iteratively mines pseudo-parallel sentences using pretrained sentence embeddings, boosting unsupervised translation BLEU by 2.4 on average.

Chau Tran, Yuqing Tang, Xian Li et al.

Pretrained FinBERT on 4.9B tokens, outperforming generic BERT in financial sentiment tasks with up to 15.6% accuracy gain.

Yi Yang, Mark Christopher Siy UY, Allen Huang

ETHOS dataset offers hate speech detection based on YouTube and Reddit comments, using binary and multi-label classification.

Ioannis Mollas, Zoe Chrysopoulou, Stamatis Karlos et al.

Scaling up to 175B parameters, GPT-3 achieves remarkable few-shot and zero-shot NLP performance, surpassing many fine-tuned models.

Tom B. Brown, Benjamin Mann, Nick Ryder et al.

Movement pruning leverages first-order importance scores during fine-tuning, achieving high sparsity with minimal accuracy loss, outperforming magnitude pruning.

Victor Sanh, Thomas Wolf, Alexander M. Rush

Introduced INFOTABS dataset for semi-structured table inference; models underperform on complex reasoning, highlighting challenges.

Vivek Gupta, Maitrey Mehta, Pegah Nokhiz et al.

This paper uses structural probes to analyze multilingual BERT, revealing shared syntactic subspaces across languages, indicating learned universal grammatical relations.

Ethan A. Chi, John Hewitt, Christopher D. Manning

This study uses controlled experiments to reveal that neural models encode linguistic features independently of task relevance, highlighting the role of pretrained embeddings.

Abhilasha Ravichander, Yonatan Belinkov, Eduard Hovy

Introduces KLEJ, a comprehensive Polish NLP benchmark with 9 tasks, and HerBERT, a dedicated Transformer model, achieving top performance.

Piotr Rybak, Robert Mroczkowski, Janusz Tracz et al.

Introduced CLAMS, a multilingual syntactic evaluation suite, revealing that monolingual LSTM models excel in simple dependencies but struggle with complex structures; multilingual BERT performs well in English but poorly in other languages.

Aaron Mueller, Garrett Nicolai, Panayiota Petrou-Zeniou et al.

Improved natural language generation via loss truncation enhances the factual accuracy and reliability of generated text.

Daniel Kang, Tatsunori Hashimoto

Introduced ToTTo dataset with 120K samples, combining manual revision and highlighted cells for high-precision table-to-text generation.

Ankur P. Parikh, Xuezhi Wang, Sebastian Gehrmann et al.

AxCell employs table semantic segmentation and ULMFiT to automatically extract results from ML papers, achieving significant accuracy improvements.

Marcin Kardas, Piotr Czapla, Pontus Stenetorp et al.

LogicalFactChecker integrates graph neural networks and logical operations to achieve state-of-the-art table fact verification, surpassing previous methods.

Wanjun Zhong, Duyu Tang, Zhangyin Feng et al.

UD v2 advances multilingual dependency parsing by expanding treebanks to over 90 languages, introducing multiword expressions and enhanced dependency relations, achieving over 85% accuracy in CoNLL 2018.

Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter et al.

Models are optimized via multi-objective framework; nucleus sampling outperforms others; likelihood trap identified and addressed.

Hugh Zhang, Daniel Duckworth, Daphne Ippolito et al.

Proposes logical NLG from open-domain tables, using Transformer and pre-trained models, achieving significant gains in logical fidelity.

Wenhu Chen, Jianshu Chen, Yu Su et al.