This study evaluates LLMs (GPT-4, Gemini, Claude) in NLP peer review, revealing high defect rates and limited professional judgment, with scores averaging 7.45 vs. human 6.41.

Jiangshu Du, Yibo Wang, Wenting Zhao et al.

PE-Rank leverages passage embeddings for efficient listwise reranking, reducing latency by 4.5x while maintaining performance.

Qi Liu, Bo Wang, Nan Wang et al.

SEC-QA framework generates financial multi-document QA datasets, enhancing model evaluation accuracy.

Viet Dac Lai, Michael Krumdick, Charles Lovering et al.

Introduces a lexically grounded subword segmentation method, improving morphological plausibility.

Jindřich Libovický, Jindřich Helcl

ChatGLM-4, based on Transformer, trained on 10 trillion tokens, rivals GPT-4, supports multi-tool autonomous decision-making.

Team GLM, :, Aohan Zeng et al.

Satyrn combines structured data analysis with LLMs, achieving 86% factual accuracy in report generation.

Marko Sterbentz, Cameron Barrie, Shubham Shahi et al.

Identifies leading whitespace in subword vocabularies causes probability inconsistencies; proposes trailing whitespace decoding to fix, affecting cognitive and NLP tasks.

Byung-Doh Oh, William Schuler

Proposes BlockPruner, a training-free block-level pruning method for Transformer MHA and MLP, evaluated via perplexity, outperforming state-of-the-art baselines.

Longguang Zhong, Fanqi Wan, Ruijun Chen et al.

Proposes dynamic oracle-guided autoregressive training, improving NER and summarization, with exact and approximate algorithms for specific metrics.

Jianing Yang, Harshine Visvanathan, Yilin Wang et al.

OLMES standardizes large language model evaluation, covering prompt formatting, example selection, probability normalization, ensuring reproducibility and fairness.

Yuling Gu, Oyvind Tafjord, Bailey Kuehl et al.

cPAPERS dataset integrates OpenReview comments with arXiv documents for multimodal Q&A in scientific papers.

Anirudh Sundar, Jin Xu, William Gay et al.

UICoder fine-tunes LLMs for UI code generation using automated feedback, achieving near-proprietary model performance with synthetic data and iterative filtering.

Jason Wu, Eldon Schoop, Alan Leung et al.

This paper introduces Masked Diffusion Language Models (MDLM), achieving near state-of-the-art perplexity by combining efficient training, Rao-Blackwellized objectives, and semi-autoregressive sampling.

Subham Sekhar Sahoo, Marianne Arriola, Yair Schiff et al.

TextGrad employs natural language feedback as 'gradients' to optimize complex AI systems, improving performance across tasks like QA, coding, and drug design.

Mert Yuksekgonul, Federico Bianchi, Joseph Boen et al.

CREAM employs middle-focused positional encoding via index interpolation, enabling efficient extension of LLM context from 4K to 256K tokens with minimal fine-tuning.

Tong Wu, Yanpeng Zhao, Zilong Zheng

MaskLID is a training-free multi-language detection method using iterative masking to improve code-switching identification.

Amir Hossein Kargaran, François Yvon, Hinrich Schütze

MedREQAL constructs a systematic review-based biomedical QA dataset to evaluate six large language models' medical knowledge recall, revealing current limitations.

Juraj Vladika, Phillip Schneider, Florian Matthes

NATURAL PLAN benchmark evaluates LLMs on real-world natural language planning tasks; models achieve below 50% success, highlighting significant gaps.

Huaixiu Steven Zheng, Swaroop Mishra, Hugh Zhang et al.

Buffer of Thoughts (BoT) enhances LLM reasoning by storing and retrieving high-level thought templates, boosting accuracy by 20%+ on complex tasks.

Ling Yang, Zhaochen Yu, Tianjun Zhang et al.

Proposes viewing LLM hallucinations as confabulation with high narrativity, empirically showing increased coherence and potential cognitive value.

Peiqi Sui, Eamon Duede, Sophie Wu et al.