Proposes Speculative Sampling, enabling 2-2.5x speedup in decoding 70B-parameter Chinchilla model by generating multiple tokens per call while maintaining distribution fidelity.

Charlie Chen, Sebastian Borgeaud, Geoffrey Irving et al.

GSM-IC shows LLMs are easily distracted; SC and ignore-the-noise instructions partly recover accuracy.

Freda Shi, Xinyun Chen, Kanishka Misra et al.

In-Context RALM boosts language model performance by prepending documents, significantly reducing perplexity.

Ori Ram, Yoav Levine, Itay Dalmedigos et al.

Leveraging large language models (LLMs) as versatile decomposers through evidence and question decomposition significantly enhances table-based reasoning, surpassing human performance on TabFact.

Yunhu Ye, Binyuan Hui, Min Yang et al.

REPLUG enhances black-box large models like GPT-3 by simple retrieval-based input augmentation, boosting performance by 6.3% without parameter fine-tuning.

Weijia Shi, Sewon Min, Michihiro Yasunaga et al.

Proposes sample-efficient pretraining using child-inspired datasets (10M-100M words) with Transformer models and curriculum learning, improving low-resource NLP.

Alex Warstadt, Leshem Choshen, Aaron Mueller et al.

DetectGPT uses probability curvature to achieve zero-shot detection, boosting AUROC from 0.81 to 0.95 on GPT-NeoX fake news.

Eric Mitchell, Yoonho Lee, Alexander Khazatsky et al.

LUMEN combines pre-computed and on-the-fly encoding, improving QA performance with reduced compute, especially at larger scales.

Michiel de Jong, Yury Zemlyanskiy, Nicholas FitzGerald et al.

Grounds language and cognition in neural mechanisms; evaluates LLMs' formal vs. functional competence, highlighting strengths and gaps.

Kyle Mahowald, Anna A. Ivanova, Idan A. Blank et al.

SlideVQA introduces a multi-image document VQA dataset and a Seq2Seq model with multi-task learning, boosting numerical reasoning by 12%.

Ryota Tanaka, Kyosuke Nishida, Kosuke Nishida et al.

Self-Instruct uses model self-generated instructions to improve instruction tuning, boosting GPT-3 performance by 33%.

Yizhong Wang, Yeganeh Kordi, Swaroop Mishra et al.

This paper reveals that Transformer attention embodies a dual form of gradient descent, interpreting in-context learning (ICL) as implicit fine-tuning, validated through empirical analysis and a momentum attention mechanism.

Damai Dai, Yutao Sun, Li Dong et al.

Survey on reasoning in LLMs, highlighting Chain-of-Thought prompting, achieving 74.4% on GSM8K.

Jie Huang, Kevin Chen-Chuan Chang

Proposes Fine-tune-CoT, using large models to generate reasoning samples, greatly enhancing small model reasoning performance.

Namgyu Ho, Laura Schmid, Se-Young Yun

HALIE framework evaluates human-AI interaction, revealing that superior non-interactive metrics do not guarantee better user experience.

Mina Lee, Megha Srivastava, Amelia Hardy et al.

INSTRUCTOR is a single text embedding model using instruction finetuning, achieving 3.4% improvement across 70 tasks without further training.

Hongjin Su, Weijia Shi, Jungo Kasai et al.

Proposes an unsupervised optimal transport-based hallucination detector for NMT, outperforming previous model-based and external methods.

Nuno M. Guerreiro, Pierre Colombo, Pablo Piantanida et al.

Proposed PROMPTIST, a reinforcement learning framework, improves text-to-image prompts, boosting reward by over 30% on Stable Diffusion.

Yaru Hao, Zewen Chi, Li Dong et al.

Using internal source contribution (ALTI+) and cross-lingual embeddings (LaBSE) improves hallucination detection accuracy by 2x, with enhanced reranking strategies.

David Dale, Elena Voita, Loïc Barrault et al.

This study evaluates zero-shot Chain-of-Thought prompting's bias and toxicity, revealing larger models exhibit increased social biases.

Omar Shaikh, Hongxin Zhang, William Held et al.