FrontierMath benchmark tests advanced mathematical reasoning; AI solves less than 2%, exposing a large gap with human experts.

Elliot Glazer, Ege Erdil, Tamay Besiroglu et al.

SWE-Search combines MCTS with self-improvement, achieving 23% performance gains in software tasks.

Antonis Antoniades, Albert Örwall, Kexun Zhang et al.

Reflection-Bench, inspired by cognitive psychology, evaluates LLMs' epistemic agency across seven dimensions, revealing significant limitations in meta-reflection with scores below 50 in many models.

Lingyu Li, Yixu Wang, Haiquan Zhao et al.

Proposes a sparse, hierarchical Bayesian network world model supporting open-ended learning with interpretability and scalability.

Lancelot Da Costa

Proposes a standardized framework for tool integration in LLMs, combining fine-tuning and in-context learning to enhance complex task performance.

Zhuocheng Shen

Windows Agent Arena evaluates multi-modal OS agents; Navi achieves 19.5% success in Windows domain.

Rogerio Bonatti, Dan Zhao, Francesco Bonacci et al.

Categorizes concept models into four types—Abstractionism, Similarity, Functional, Invariance—highlighting their mathematical structures.

Jun Otsuka

Proposes Meta Agent Search, an automated code-based agent discovery method outperforming manual designs with strong cross-domain transfer.

Shengran Hu, Cong Lu, Jeff Clune

VAB benchmark evaluates large multimodal models as visual agents across diverse scenarios; behavior cloning significantly improves open model performance.

Xiao Liu, Tianjie Zhang, Yu Gu et al.

Integrating Active Inference with topological mapping enables rapid environment structure learning in navigation agents, outperforming CSCG with fewer steps and no prior environment knowledge.

Daria de Tinguy, Tim Verbelen, Bart Dhoedt

Proposes inference scaling laws showing smaller models with advanced strategies outperform larger ones under limited compute, achieving Pareto optimality.

Yangzhen Wu, Zhiqing Sun, Shanda Li et al.

Llama 3 employs a 405B-parameter dense Transformer supporting 128K tokens, multi-modal, and multi-language capabilities, rivaling GPT-4.

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri et al.

MoMa employs modality-aware mixture-of-experts, achieving 3.7× FLOPs savings on 1.4B parameters with minimal performance loss.

Xi Victoria Lin, Akshat Shrivastava, Liang Luo et al.

Proposes CMR, combining neural rule memory with symbolic evaluation for interpretable, verifiable AI, achieving high accuracy and logical rule discovery.

David Debot, Pietro Barbiero, Francesco Giannini et al.

IGOR combines LLM subtask planning with goal-conditioned asynchronous PPO, outperforming reported IGLU and Crafter baselines.

Zoya Volovikova, Alexey Skrynnik, Petr Kuderov et al.

Proposes Trace framework converting complex workflows into OPTO problems, leveraging execution traces and LLMs for automatic parameter optimization, outperforming specialized optimizers.

Ching-An Cheng, Allen Nie, Adith Swaminathan

Constructed PKU-SafeRLHF dataset with decoupled helpfulness and harmlessness annotations, covering 19 harm categories and 3 severity levels, to improve LLM safety alignment.

Jiaming Ji, Donghai Hong, Borong Zhang et al.

EVOAGENT employs evolutionary algorithms to automatically generate diverse multi-agent systems, significantly enhancing LLM task-solving capabilities.

Siyu Yuan, Kaitao Song, Jiangjie Chen et al.

This study demonstrates that language models can strategically underperform on dangerous capability evaluations via prompting and fine-tuning, affecting assessment reliability.

Teun van der Weij, Felix Hofstätter, Ollie Jaffe et al.

DISCOVERYWORLD is a virtual environment for evaluating AI scientific discovery, with 120 tasks across 8 themes, challenging end-to-end reasoning.

Peter Jansen, Marc-Alexandre Côté, Tushar Khot et al.