The study finds that relation decoding in Transformer models can be approximated by linear transformations.

Evan Hernandez, Arnab Sen Sharma, Tal Haklay et al.

Study on catastrophic forgetting in LLMs during continual fine-tuning; BLOOMZ shows less forgetting than mT0.

Yun Luo, Zhen Yang, Fandong Meng et al.

RAVEN combines retrieval-augmented masked and prefix language modeling to enhance in-context learning.

Jie Huang, Wei Ping, Peng Xu et al.

Proposes AutoMQM, leveraging large language models like PaLM-2 for fine-grained MT error detection and classification, significantly enhancing interpretability.

Patrick Fernandes, Daniel Deutsch, Mara Finkelstein et al.

InstructGLM uses natural language prompts for graph structure description, surpassing GNNs in node classification with instruction fine-tuning.

Ruosong Ye, Caiqi Zhang, Runhui Wang et al.

Proposes GTE, a general text embedding model trained via multi-stage contrastive learning, with only 110M parameters, surpassing large models in multiple benchmarks.

Zehan Li, Xin Zhang, Yanzhao Zhang et al.

This paper introduces Dynalang, a multimodal world model that predicts future text and images using diverse language cues, outperforming traditional language-conditioned policies.

Jessy Lin, Yuqing Du, Olivia Watkins et al.

Evaluates instruction-following models for QA correctness and faithfulness, proposing new metrics.

Vaibhav Adlakha, Parishad BehnamGhader, Xing Han Lu et al.

Introduces RRTF framework boosting PanGu-Coder2 to 62.20% pass@1, outperforming all open-source models.

Bo Shen, Jiaxin Zhang, Taihong Chen et al.

Proposes Skill-It framework using skill dependency graphs to optimize data sampling, improving language model training efficiency.

Mayee F. Chen, Nicholas Roberts, Kush Bhatia et al.

Proposes LLM-Rec, using four prompting strategies to enrich item descriptions, significantly improving recommendation accuracy.

Hanjia Lyu, Song Jiang, Hanqing Zeng et al.

Proposes RippleEdits benchmark to evaluate ripple effects in knowledge editing, revealing current methods' inability to maintain consistency.

Roi Cohen, Eden Biran, Ori Yoran et al.

Jina Embeddings excel in MTEB using contrastive training on T5 architecture.

Michael Günther, Louis Milliken, Jonathan Geuter et al.

FLASK introduces a fine-grained, skill-based evaluation framework for LLMs, improving interpretability and reliability.

Seonghyeon Ye, Doyoung Kim, Sungdong Kim et al.

This study constructs a comprehensive taxonomy of NLP research, analyzes over 80,000 papers, and identifies key trends, highlighting language models and responsible AI as future priorities.

Tim Schopf, Karim Arabi, Florian Matthes

DecompEval reformulates NLG evaluation as an instruction-based QA task using instruction-tuned PLMs, employing sentence decomposition for interpretability and generalization.

Pei Ke, Fei Huang, Fei Mi et al.

This paper provides a comprehensive overview of recent advances in Large Language Models (LLMs), covering architectural innovations and multimodal applications.

Humza Naveed, Asad Ullah Khan, Shi Qiu et al.

Predictive Pipelined Decoding (PPD) reduces LLM decoding latency by parallelizing token prediction while maintaining output consistency.

Seongjun Yang, Gibbeum Lee, Jaewoong Cho et al.

Introduces BeaverTails dataset with disentangled helpfulness and harmlessness annotations, using PPO-Lagrangian for safety RLHF, improving AI safety alignment with 33k QA and 36k preference pairs.

Jiaming Ji, Mickel Liu, Juntao Dai et al.

Proposes a semiring-weighted Earley parser with complexity reduced to O(N³|G|), enabling efficient large-scale grammar parsing.

Andreas Opedal, Ran Zmigrod, Tim Vieira et al.