IDEAgent: Agentic Quality-Diversity Search for Research Idea Generation
IDEAgent employs a multi-agent Quality-Diversity search framework to generate diverse, high-quality research ideas, outperforming baselines by 3.89× on Yield across 32 topics.
Key Findings
Methodology
This approach formulates research idea generation as a multi-objective quality-diversity (QD) search problem, utilizing a multi-agent architecture comprising generators, evaluators, and archives. The system employs structured idea summaries and multi-dimensional assessments—covering non-obviousness, soundness, clarity, and diversity—to guide iterative evolution. Repair and refinement modules address logical flaws, while structured comparisons against historical and rejected ideas maintain diversity. The Yield metric quantifies the largest mutually diverse set satisfying quality thresholds. Extensive experiments across 32 topics in 8 computer science domains demonstrate that IDEAgent surpasses baseline methods by 3.89× in Yield, with significantly broader topic coverage.
Key Results
- Across multiple domains, IDEAgent achieved an average Yield 3.89 times higher than the best baseline, with non-zero Yield on 8 times more topics, indicating superior capacity to generate high-quality, diverse ideas.
- Analysis shows that repair and refinement significantly improve logical rigor and clarity, reducing trivial or inconsistent concepts and enhancing non-obviousness.
- Internal and external evaluations agree that the system's improvements in soundness, clarity, and novelty are consistent, validating its effectiveness.
Significance
This work advances automated scientific ideation by formalizing it as a multi-objective quality-diversity search, addressing the long-standing challenge of generating both innovative and logically sound research directions. The multi-agent framework, combined with structured assessment and comparison, enables scalable and robust exploration of research space, promising to accelerate discovery processes. The approach mitigates issues of idea collapse and repetition common in existing LLM-based systems, fostering richer, more diverse scientific exploration. Its potential impact spans academia and industry, offering a pathway toward autonomous research assistants capable of proposing novel, high-impact hypotheses and methodologies, thus transforming the landscape of scientific innovation.
Technical Contribution
The core technical contribution lies in modeling research idea generation as a multi-objective QD search, integrating multi-agent collaboration with structured idea summaries and multi-dimensional evaluation. The system introduces the Yield metric for joint assessment of diversity and quality, overcoming limitations of single-metric approaches. It employs a multi-stage process—generation, evaluation, repair, and refinement—guided by explicit thresholds and structured comparisons, ensuring high logical rigor and diversity. This framework supports dynamic, multi-round evolution, enabling the generation of a broad set of high-quality, mutually distinct ideas. The experimental validation demonstrates its superiority over existing methods, establishing a new paradigm for AI-driven scientific ideation.
Novelty
This is the first work explicitly framing scientific idea generation as a multi-objective quality-diversity search problem, employing a multi-agent system with structured summaries and multi-dimensional evaluation. Unlike prior approaches that focus on single metrics or static archives, this method dynamically manages idea lineages, actively repairs and refines proposals, and employs the Yield metric to quantify the joint quality-diversity objective. Its innovative architecture and evaluation strategy effectively address the prevalent issue of idea collapse and promote the generation of genuinely novel, high-quality research directions, marking a significant step forward in AI-assisted scientific discovery.
Limitations
- The system's performance heavily depends on the quality and diversity of the background corpus; limited or biased data can constrain idea novelty and exploration scope.
- Computational complexity increases with multi-round evaluation, repair, and refinement, posing efficiency challenges for large-scale or real-time applications.
- Validation is primarily within computer science domains; applicability to other scientific fields requires further adaptation and testing.
Future Work
Future research will explore integrating multimodal background data, including images and experimental results, to enhance cross-disciplinary creativity. Optimization of computational efficiency and scalability is also planned, possibly through model pruning or parallelization. Incorporating human-in-the-loop feedback and more sophisticated evaluation metrics will improve idea relevance and practical utility. Additionally, extending the framework to support autonomous hypothesis testing and experimental planning could further accelerate scientific discovery, moving toward fully automated research pipelines.
AI Executive Summary
In the realm of scientific research, the generation of innovative and diverse ideas is crucial for breakthroughs. Traditional AI systems, often based on large language models, tend to produce ideas that are either high in quality but lack diversity or vice versa, leading to repetitive or trivial concepts. Recognizing this limitation, the authors introduce IDEAgent, a novel multi-agent framework that models research ideation as a multi-objective quality-diversity (QD) search problem. This approach leverages a collaborative system of idea generators, evaluators, and memory archives to iteratively produce, assess, and refine research proposals.
The core innovation lies in the structured management of idea lineages, where each proposal is represented through concise summaries and evaluated on multiple axes—non-obviousness, soundness, clarity, and diversity. The system employs a multi-stage process: generating raw ideas, assessing their quality, repairing logical flaws, and refining proposals to enhance their rigor and novelty. The integration of the Yield metric allows for the systematic measurement of the largest set of mutually diverse ideas that meet predefined quality thresholds, ensuring a balanced exploration of the research space.
Extensive experiments across 32 topics spanning 8 computer science domains demonstrate that IDEAgent significantly outperforms baseline methods, achieving 3.89× higher Yield and covering 8 times more topics with non-zero yields. The results highlight the importance of repair and refinement modules in maintaining logical rigor and fostering non-obvious, high-quality ideas. The framework’s ability to generate a broad, high-quality set of research directions holds promise for accelerating scientific discovery, reducing redundancy, and fostering innovation.
Despite its strengths, the system relies on comprehensive background data and faces computational challenges due to multi-stage evaluations. Future work aims to incorporate multimodal data, optimize efficiency, and extend applicability beyond computer science. Overall, IDEAgent represents a significant advance in AI-driven research ideation, offering a scalable, robust approach to fostering scientific creativity and exploration.
Deep Dive
Abstract
Large Language Models (LLMs) have significantly automated the process of scientific discovery over the past few years. However, existing systems share one core limitation: they generate and optimize ideas independently for either Quality or Diversity. This often leads to the generation of ideas in close proximity to one another or to a large set of trivial, unsound, or unclear concepts. In this work, we instead argue that research ideation should be treated as a conjunction of both objectives and framed as a Quality-Diversity (QD) search. In line with this perspective, we introduce IDEAgent, a multi-agent framework that manages the evolution of ideas through lineages. We jointly drive Quality using multi-objective feedback for dedicated repair and refinement, while Diversity is achieved through lightweight sequential memory and explicit comparison against completed ideas, their historical ancestors, and rejected proposals. To systematically evaluate this QD conjunction, we develop Yield, a joint metric that computes the largest set of mutually diverse ideas that satisfy a predetermined quality threshold. Finally, through evaluations across 32 topics spanning 8 domains of Computer Science, we show that IDEAgent outperforms the best baseline by 3.89x on Yield, while achieving non-zero Yield on 8x more topics. We further corroborate these findings through an analysis of quality improvements, showing that repair and refinement are crucial for building logical rigor and clarity while preserving non-obviousness. To encourage future research on QD-search-based ideation, we open-source IDEAgent at https://github.com/declare-lab/IDEAgent.