O-Researcher: An Open Ended Deep Research Model via Multi-Agent Distillation and Agentic RL
O-Researcher uses multi-agent distillation and RL to generate high-quality research data, boosting open-source LLMs' performance.
Key Findings
Methodology
O-Researcher introduces a multi-agent framework simulating complex tool-integrated reasoning to generate high-fidelity research-grade data. It employs a two-stage training strategy: supervised fine-tuning and Group Relative Policy Optimization (GRPO)-based reinforcement learning, enhancing alignment and reasoning capabilities.
Key Results
- Result 1: Achieved 48.48 RACE score on Deep-Research-Bench, outperforming open-source baseline Tongyi-Deep Research (45.66) and closed-source GPT-5 (46.77).
- Result 2: Reinforcement learning improved citation accuracy from 29.13% to 31.99% and effective citations from 13.67 to 26.01.
- Result 3: Models using a 10-step reasoning workflow excelled in comprehensiveness (49.61) and insight (48.71).
Significance
This work provides an effective pathway for training open-source LLMs without proprietary data, significantly narrowing the performance gap with closed-source models on complex reasoning tasks. It has profound implications for democratizing AI research and applications.
Technical Contribution
O-Researcher introduces a multi-agent framework for high-fidelity data generation and employs GRPO to optimize tool use and reasoning. It also proposes structured data representation and multi-stage filtering for data quality assurance.
Novelty
This is the first approach to combine multi-agent collaboration with reinforcement learning for research-grade data generation, achieving significant performance improvements in open-source LLMs.
Limitations
- Limitation 1: Domain coverage of generated data may be biased, affecting generalizability.
- Limitation 2: High computational cost of reinforcement learning limits accessibility.
- Limitation 3: Performance on extremely complex tasks still has room for improvement.
Future Work
Future work could explore more efficient multi-agent collaboration mechanisms, expand domain coverage, and optimize RL computational efficiency.
AI Executive Summary
Open-source large language models (LLMs) lag behind closed-source counterparts in complex reasoning tasks due to limited access to high-quality training data. O-Researcher addresses this gap by introducing a multi-agent framework for generating research-grade data and a two-stage training strategy combining supervised fine-tuning and reinforcement learning.
The framework simulates collaborative tool-integrated reasoning, producing high-fidelity instruction-response pairs. The training pipeline first establishes a knowledge base through supervised fine-tuning, followed by Group Relative Policy Optimization (GRPO) to refine reasoning and tool-use capabilities.
Experiments on Deep-Research-Bench demonstrate that O-Researcher achieves a RACE score of 48.48, surpassing both open-source and some closed-source models. This work offers a scalable solution for advancing open-source LLMs, with significant implications for academia and industry while paving the way for future innovations in AI research and applications.
Deep Analysis
Background
Recent advancements in LLMs have revolutionized natural language processing. However, open-source models underperform on complex reasoning tasks compared to closed-source systems like GPT-4 and Gemini-2.5, primarily due to the lack of proprietary high-quality training data.
Core Problem
The core challenge is generating high-quality, research-grade training data for open-source LLMs without relying on proprietary resources. Solving this is critical for democratizing AI and improving open-source model capabilities.
Innovation
Key innovations include:
- �� Multi-agent collaboration framework: Simulates complex reasoning to generate high-fidelity data.
- �� GRPO reinforcement learning: Optimizes tool use and reasoning capabilities.
- �� Data quality assurance: Employs multi-stage filtering and structured data representation.
Methodology
The methodology involves:
- �� Data generation: Multi-agent collaboration decomposes complex tasks into multi-turn dialogues.
- �� Data filtering: Multi-stage rejection sampling and human verification ensure quality.
- �� Supervised fine-tuning: Trains the model on synthesized data to build a knowledge base.
- �� Reinforcement learning: GRPO refines reasoning and output quality.
Experiments
Experiments used Deep-Research-Bench and DeepResearchGym to evaluate performance in domains like science and finance. Baselines included Tongyi-Deep Research and GPT-5.
Results
O-Researcher-RL achieved a RACE score of 48.48, outperforming open-source baselines (45.66). RL improved citation accuracy (31.99%) and effective citations (26.01).
Applications
Applications include academic research assistants, enterprise decision-support systems, and intelligent Q&A platforms for complex tasks.
Limitations & Outlook
Limitations include domain coverage bias, high RL computational cost, and challenges in handling extremely complex tasks.
Plain Language Accessible to non-experts
Imagine a team of experts working on a complex research project. O-Researcher acts like a virtual team where each AI agent specializes in tasks like data collection, analysis, and report writing. By collaborating, they generate high-quality research data and continuously improve their methods.
ELI14 Explained like you're 14
Think about doing a group project with friends where everyone has a role—one finds info, another writes. O-Researcher is like a super-smart AI team that works together to tackle tough problems and even learns to get better at it. Cool, right?
Glossary
Multi-Agent Collaboration
A framework where multiple AI models work together to solve complex tasks.
Used for generating high-quality research data.
GRPO
A reinforcement learning algorithm optimizing model strategies.
Enhances reasoning and tool-use capabilities.
RACE
A metric for evaluating report quality.
Measures performance on Deep-Research-Bench.
Data Filtering
Processes ensuring high-quality training data through multi-stage sampling.
Critical for generating reliable datasets.
Structured Data Representation
Organizing reasoning processes in XML format.
Used to train the model's reasoning capabilities.
Open Questions Unanswered questions from this research
- 1 How to expand domain coverage of generated data?
- 2 How to reduce computational costs of reinforcement learning?
- 3 How to improve performance on extremely complex tasks?
Applications
Immediate Applications
Academic Research Assistant
Helps researchers quickly conduct literature reviews and data analysis.
Enterprise Decision Support
Provides data-driven insights and recommendations for businesses.
Long-term Vision
General AI Research Assistant
Enables autonomous completion of complex research tasks.
Abstract
The performance gap between closed-source and open-source large language models (LLMs) is largely attributed to disparities in access to high-quality training data. To bridge this gap, we introduce a novel framework for the automated synthesis of sophisticated, research-grade instructional data. Our approach centers on a multi-agent workflow where collaborative AI agents simulate complex tool-integrated reasoning to generate diverse and high-fidelity data end-to-end. Leveraging this synthesized data, we develop a two-stage training strategy that integrates supervised fine-tuning with a novel reinforcement learning method, designed to maximize model alignment and capability. Extensive experiments demonstrate that our framework empowers open-source models across multiple scales, enabling them to achieve new state-of-the-art performance on the major deep research benchmark. This work provides a scalable and effective pathway for advancing open-source LLMs without relying on proprietary data or models.