This survey reviews LLM-based multi-agent systems, highlighting architectures like DyLAN, ProAgent, and their performance in software, social, and strategic tasks.

Taicheng Guo, Xiuying Chen, Yaqi Wang et al.

ChatQA surpasses GPT-4 in conversational QA and RAG using a two-stage instruction tuning and dense retriever.

Zihan Liu, Wei Ping, Rajarshi Roy et al.

Self-Rewarding language model using iterative DPO training, surpassing Claude 2, Gemini Pro, and GPT-4 0613 on AlpacaEval 2.0 with Llama 2 70B.

Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho et al.

Introduces R-Judge benchmark, evaluates LLMs' safety risk detection with 74.45% top score, revealing significant room for improvement.

Tongxin Yuan, Zhiwei He, Lingzhong Dong et al.

Introduced IQC method to create MMIQC dataset; Qwen-72B-MMIQC achieves 45.0% accuracy on MATH benchmark.

Haoxiong Liu, Yifan Zhang, Yifan Luo et al.

Contrastive Preference Optimization (CPO) boosts 7B/13B LLM translation, surpassing SFT, matching GPT-4 performance.

Haoran Xu, Amr Sharaf, Yunmo Chen et al.

Proposes Speculative Decoding to accelerate LLM inference by 5× using draft-verify strategies, maintaining quality.

Heming Xia, Zhe Yang, Qingxiu Dong et al.

Leveraging large language models (GPT-3.5, GPT-4, DepGPT) combined with deep learning and transformers for accurate Bengali social media depression detection.

Ahmadul Karim Chowdhury, Md. Saidur Rahman Sujon, Md. Shirajus Salekin Shafi et al.

Proposes BGM, a bridge model trained via supervised and reinforcement learning, to optimize the connection between retrievers and LLMs, boosting QA and personalized generation by 10-15%.

Zixuan Ke, Weize Kong, Cheng Li et al.

EASYTOOL leverages content extraction and standardization to generate concise tool instructions, greatly improving LLM tool utilization efficiency.

Siyu Yuan, Kaitao Song, Jiangjie Chen et al.

Survey NLP performance on dialect datasets, propose improvement methods.

Aditya Joshi, Raj Dabre, Diptesh Kanojia et al.

Introduces DRFR metric and InFoBench benchmark, significantly improving LLM instruction-following evaluation reliability.

Yiwei Qin, Kaiqiang Song, Yebowen Hu et al.

Activation Beacon introduces progressive activation compression, enabling efficient long-text processing with up to 8x compression, doubling inference speed.

Peitian Zhang, Zheng Liu, Shitao Xiao et al.

GRAM achieves 73.68% ANLS on multi-page DocVQA, enhancing long-sequence processing efficiency.

Tsachi Blau, Sharon Fogel, Roi Ronen et al.

DeepSeek LLM scales open-source models; 67B model outperforms LLaMA-2 70B in code, math, reasoning.

DeepSeek-AI, :, Xiao Bi et al.

TinyLlama trains a 1.1B Llama-2-style model on up to 3T tokens, outperforming similarly sized open baselines.

Peiyuan Zhang, Guangtao Zeng, Tianduo Wang et al.

MosaicBERT integrates FlashAttention, ALiBi, GLU, and other techniques, achieving 79.6 GLUE score in 1.13 hours, vastly improving training efficiency.

Jacob Portes, Alex Trott, Sam Havens et al.

Introduces Time Vectors, using fine-tuned weight differences and interpolation to encode temporal information, improving model performance across time.

Kai Nylund, Suchin Gururangan, Noah A. Smith

Using model artifacts like Softmax, attention, and input attribution to predict hallucinations in QA with up to 0.80 AUROC, before they occur.

Ben Snyder, Marius Moisescu, Muhammad Bilal Zafar

RAG combines retrieval and generation, significantly improving factual accuracy in LLMs; key techniques include indexing, retrieval, and augmentation.

Yunfan Gao, Yun Xiong, Xinyu Gao et al.