Proposes pQRNN, a parameter-efficient multilingual model achieving 95.9% of mBERT's performance with 350x fewer parameters.

Prabhu Kaliamoorthi, Aditya Siddhant, Edward Li et al.

This paper introduces a fine-tuning framework for multilingual language models with unsupervised objectives, achieving state-of-the-art word alignment results, surpassing traditional methods.

Zi-Yi Dou, Graham Neubig

EMMA learns text-to-entity grounding in MESSENGER, achieving over 40% higher zero-shot win rates, while the hardest stage remains at 10%.

Austin W. Hanjie, Victor Zhong, Karthik Narasimhan

This study introduces DelBERT, a derivational segmentation-based model, which significantly improves BERT's semantic understanding of complex words, especially in low-frequency and out-of-vocabulary cases.

Valentin Hofmann, Janet B. Pierrehumbert, Hinrich Schütze

BanglaBERT, an ELECTRA-based low-resource Bangla language model trained on 27.5GB data, achieves SOTA on four NLP tasks and introduces the BLUB benchmark.

Abhik Bhattacharjee, Tahmid Hasan, Wasi Uddin Ahmad et al.

Prefix-Tuning introduces trainable continuous prefixes in frozen models, using only 0.1% parameters, achieving comparable performance in generation tasks.

Xiang Lisa Li, Percy Liang

EarlyBERT leverages early structured lottery tickets to cut training time by 35-45%, maintaining performance.

Xiaohan Chen, Yu Cheng, Shuohang Wang et al.

Systematic comparison of multilingual vs. monolingual models reveals dedicated tokenizers significantly boost performance, especially for resource-scarce languages.

Phillip Rust, Jonas Pfeiffer, Ivan Vulić et al.

HateCheck provides 29 functional tests to reveal key weaknesses in hate speech detection models.

Paul Röttger, Bertram Vidgen, Dong Nguyen et al.

A joint reranking and verification model achieves strong performance on TabFact dataset.

Michael Schlichtkrull, Vladimir Karpukhin, Barlas Oğuz et al.

Quantized Transformer for unsupervised extractive opinion summarization, significantly improving summary quality.

Stefanos Angelidis, Reinald Kim Amplayo, Yoshihiko Suhara et al.

Introduces ZEST dataset to evaluate zero-shot learning from task descriptions; T5 achieves only 12%, highlighting challenges.

Orion Weller, Nicholas Lourie, Matt Gardner et al.

WikiAsp is a multi-domain aspect-based summarization dataset using Wikipedia articles for aspect annotation.

Hiroaki Hayashi, Prashant Budania, Peng Wang et al.

Using RoBERTa-based ClimateQA for financial report climate info extraction, achieving 82% F1 score.

Alexandra Luccioni, Emily Baylor, Nicolas Duchene

Proposes 2WikiMultiHopQA, combining structured and unstructured data, with explicit reasoning paths, enhancing interpretability and reasoning evaluation.

Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara et al.

Introduces Social-Chemistry dataset and Neural Norm Transformer, achieving 78% accuracy in social norm reasoning over 292k rules.

Maxwell Forbes, Jena D. Hwang, Vered Shwartz et al.

MixKD integrates mixup data augmentation into knowledge distillation, significantly enhancing model generalization and efficiency.

Kevin J Liang, Weituo Hao, Dinghan Shen et al.

HyperGAT leverages hypergraph attention to improve text classification, achieving over 86% accuracy on benchmark datasets.

Kaize Ding, Jianling Wang, Jundong Li et al.

This paper introduces a dual approach combining curated wordlist filtering and semi-supervised Transformer models to boost low-resource language Web text identification accuracy from 5.5% to 71.2%.

Isaac Caswell, Theresa Breiner, Daan van Esch et al.

Proposes TRACIE dataset and SYMTIME model, leveraging distant supervision for implicit event temporal reasoning, achieving 78.9% accuracy.

Ben Zhou, Kyle Richardson, Qiang Ning et al.