This paper analyzes global linguistic resource distribution, proposes a six-category classification, highlights disparities affecting multilingual NLP, and urges community focus on low-resource languages.

Pratik Joshi, Sebastin Santy, Amar Budhiraja et al.

INLP (Iterative Nullspace Projection) removes bias from neural representations by iterative training of linear classifiers and nullspace projection, significantly reducing gender bias in word embeddings.

Shauli Ravfogel, Yanai Elazar, Hila Gonen et al.

HybridQA integrates heterogeneous data with a multi-hop reasoning framework using SQLNet and BERT, achieving over 52% EM on complex questions.

Wenhu Chen, Hanwen Zha, Zhiyu Chen et al.

Pretrained Transformers like BERT show improved out-of-distribution robustness across seven NLP datasets.

Dan Hendrycks, Xiaoyuan Liu, Eric Wallace et al.

Introduces Longformer with linear attention combining local windows and global tokens, enabling efficient long document modeling surpassing RoBERTa.

Iz Beltagy, Matthew E. Peters, Arman Cohan

BLEURT, based on BERT with synthetic pretraining, significantly improves text generation evaluation accuracy and robustness.

Thibault Sellam, Dipanjan Das, Ankur P. Parikh

QAGS uses question answering to evaluate factual consistency of summaries, outperforming ROUGE with a correlation of 54.52 on CNN/DM.

Alex Wang, Kyunghyun Cho, Mike Lewis

DynaBERT combines adaptive width and depth, trained via knowledge distillation, achieving performance comparable to BERT-base with improved efficiency.

Lu Hou, Zhiqi Huang, Lifeng Shang et al.

Proposes a framework to evaluate diversity metrics via correlation with controlled parameters, highlighting human superiority in content diversity detection.

Guy Tevet, Jonathan Berant

Using entmax for training and sampling, reducing mismatch, improving diversity and coherence in text generation.

Pedro Henrique Martins, Zita Marinho, André F. T. Martins

Proposes SPARQA, a skeleton-based semantic parser, achieving 21.53 F1 on GraphQuestions and 31.57 P@1 on ComplexWebQuestions, outperforming SOTA.

Yawei Sun, Lingling Zhang, Gong Cheng et al.

SAC accelerates self-attention via sparse adaptive connections, significantly reducing memory usage.

Xiaoya Li, Yuxian Meng, Mingxin Zhou et al.

This study evaluates BERT and RoBERTa calibration, showing temperature scaling and label smoothing significantly improve in- and out-of-domain calibration.

Shrey Desai, Greg Durrett

PathVQA employs semi-automated pipeline to create a dataset with 4,998 images and 32,799 QA pairs, advancing medical VQA research.

Xuehai He, Yichen Zhang, Luntian Mou et al.

AraBERT, a Transformer-based Arabic language model, achieves state-of-the-art results on multiple NLP tasks, surpassing multilingual BERT significantly.

Wissam Antoun, Fady Baly, Hazem Hajj

This paper systematically reviews BERT's internal mechanisms, analyzing its encoding of syntax, semantics, and world knowledge, and discusses model improvements and compression.

Anna Rogers, Olga Kovaleva, Anna Rumshisky

CodeBERT, a Transformer-based bimodal pre-trained model, uses MLM and RTD objectives to improve natural language-code understanding, achieving SOTA on code search and documentation.

Zhangyin Feng, Daya Guo, Duyu Tang et al.

ConvLab-2 integrates state-of-the-art models, supports multi-dataset evaluation, and provides diagnostic tools for task-oriented dialogue systems.

Qi Zhu, Zheng Zhang, Yan Fang et al.

REALM integrates a trainable neural retriever with language models, boosting open-domain QA accuracy by 4-16%, with improved interpretability and modularity.

Kelvin Guu, Kenton Lee, Zora Tung et al.

This survey comprehensively reviews knowledge graph representation, acquisition, and applications, emphasizing transformer, GNN, and temporal models.

Shaoxiong Ji, Shirui Pan, Erik Cambria et al.