Scaling a 1.5M multi-turn instruction dataset UltraChat and fine-tuning LLaMA-13B yields UltraLLaMA, outperforming Vicuna with a 9.02/10 score.

Ning Ding, Yulin Chen, Bokai Xu et al.

Proposes synthetic feedback-based alignment, training ALMoST (65.2% accuracy) without extensive human annotations or proprietary models.

Sungdong Kim, Sanghwan Bae, Jamin Shin et al.

LogicLLM enhances logical reasoning in LLMs via self-supervised training, excelling on ReClor and LogiQA-v2 benchmarks.

Fangkai Jiao, Zhiyang Teng, Bosheng Ding et al.

Proposes GDP-Zero, an open-loop Monte Carlo Tree Search method leveraging prompting large language models for goal-oriented dialogue planning without training.

Xiao Yu, Maximillian Chen, Zhou Yu

Introduces GQA method to convert multi-head models to multi-query models using 5% compute, achieving MQA-like speed and MHA-like quality.

Joshua Ainslie, James Lee-Thorp, Michiel de Jong et al.

Introduces MULTILINGUALHOLISTICBIAS, a dataset extending HOLISTICBIAS to 50 languages, revealing gender translation biases with an average 8 spBLEU gap favoring masculine references.

Marta R. Costa-jussà, Pierre Andrews, Eric Smith et al.

Proposes iEvaLM, an LLM-based interactive evaluation framework, boosting CRS performance and explainability with simulated multi-turn user interactions.

Xiaolei Wang, Xinyu Tang, Wayne Xin Zhao et al.

This study evaluates ChatGPT and GPT-4 as automatic evaluators for abstractive summarization, revealing significant stability and reliability limitations across dimensions and systems.

Chenhui Shen, Liying Cheng, Xuan-Phi Nguyen et al.

Proposes ALTI-Logit, a residual and attention-based explanation method outperforming gradients and perturbations, with strong results on linguistic and translation tasks.

Javier Ferrando, Gerard I. Gállego, Ioannis Tsiamas et al.

Adaptive-Consistency method reduces sampling budget while maintaining high accuracy, with an average accuracy drop of less than 0.1%.

Pranjal Aggarwal, Aman Madaan, Yiming Yang et al.

CRITIC framework integrates external tools for self-correction, boosting QA, math reasoning, and toxicity reduction by 7-16% improvements.

Zhibin Gou, Zhihong Shao, Yeyun Gong et al.

LLM-Pruner: a structural pruning method that compresses large models with only 50K data in 3 hours, maintaining multi-task performance.

Xinyin Ma, Gongfan Fang, Xinchao Wang

Using large language models (LLMs) with in-context learning (ICL) enables zero-shot temporal knowledge graph forecasting, matching SOTA performance without fine-tuning.

Dong-Ho Lee, Kian Ahrabian, Woojeong Jin et al.

DoReMi uses small models with Minimax Optimization to reweight data domains, speeding up large model pretraining.

Sang Michael Xie, Hieu Pham, Xuanyi Dong et al.

Reframes autoregressive decoding as a nonlinear system solved via Jacobi and Gauss-Seidel methods, achieving up to 38% speedup without model modification.

Andrea Santilli, Silvio Severino, Emilian Postolache et al.

SLiC-HF leverages sequence likelihood calibration with human feedback to enhance performance on the TL;DR task.

Yao Zhao, Rishabh Joshi, Tianqi Liu et al.

PaLM 2 combines UL2-style objectives, compute-optimal scaling, and multilingual data to improve reasoning and translation with smaller, faster models.

Rohan Anil, Andrew M. Dai, Orhan Firat et al.

SpecInfer uses tree-based speculative inference and verification to accelerate large language model serving, achieving 1.5-2.8× speedup without quality loss.

Xupeng Miao, Gabriele Oliaro, Zhihao Zhang et al.

C-Eval assesses Chinese models' multi-disciplinary reasoning; GPT-4 scores over 60%.

Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu et al.

Political Compass probing plus controlled continued pretraining traces partisan-corpus effects into hate-speech and misinformation classifiers.

Shangbin Feng, Chan Young Park, Yuhan Liu et al.