Back to AI Research

AI Research

IDEAgent: Agentic Quality-Diversity Search for Rese... | AI Research

Key Takeaways

  • IDEAgent: Agentic Quality-Diversity Search for Research Idea Generation Large Language Models (LLMs) are increasingly used to assist in scientific discovery,...
  • Large Language Models (LLMs) have significantly automated the process of scientific discovery over the past few years.
  • However, existing systems share one core limitation: they generate and optimize ideas independently for either Quality or Diversity.
  • This often leads to the generation of ideas in close proximity to one another or to a large set of trivial, unsound, or unclear concepts.
  • In this work, we instead argue that research ideation should be treated as a conjunction of both objectives and framed as a Quality-Diversity (QD) search.
Paper AbstractExpand

Large Language Models (LLMs) have significantly automated the process of scientific discovery over the past few years. However, existing systems share one core limitation: they generate and optimize ideas independently for either Quality or Diversity. This often leads to the generation of ideas in close proximity to one another or to a large set of trivial, unsound, or unclear concepts. In this work, we instead argue that research ideation should be treated as a conjunction of both objectives and framed as a Quality-Diversity (QD) search. In line with this perspective, we introduce IDEAgent, a multi-agent framework that manages the evolution of ideas through lineages. We jointly drive Quality using multi-objective feedback for dedicated repair and refinement, while Diversity is achieved through lightweight sequential memory and explicit comparison against completed ideas, their historical ancestors, and rejected proposals. To systematically evaluate this QD conjunction, we develop Yield, a joint metric that computes the largest set of mutually diverse ideas that satisfy a predetermined quality threshold. Finally, through evaluations across 32 topics spanning 8 domains of Computer Science, we show that IDEAgent outperforms the best baseline by 3.89x on Yield, while achieving non-zero Yield on 8x more topics. We further corroborate these findings through an analysis of quality improvements, showing that repair and refinement are crucial for building logical rigor and clarity while preserving non-obviousness. To encourage future research on QD-search-based ideation, we open-source IDEAgent at this https URL .

IDEAgent: Agentic Quality-Diversity Search for Research Idea Generation
Large Language Models (LLMs) are increasingly used to assist in scientific discovery, but they often struggle to balance the need for high-quality research ideas with the need for variety. Existing systems typically optimize for quality or diversity in isolation, which often results in either trivial, unclear concepts or repetitive ideas that cluster too closely together. This paper introduces IDEAgent, a multi-agent framework that treats research ideation as a "Quality-Diversity (QD) search." By managing ideas through structured lineages and using multi-objective feedback, the system aims to generate a set of research proposals that are simultaneously sound, clear, non-obvious, and mutually distinct.

The Quality-Diversity Approach

IDEAgent moves away from one-shot idea generation by framing the process as an evolutionary search. It treats research ideation like a human brainstorming session, where ideas are not just generated but refined over time. The framework maintains "lineages" for ideas, allowing them to evolve through repair and refinement. By keeping a structured record of past successes, failures, and historical variants, the system forces itself to explore new, unexplored directions rather than repeatedly revisiting familiar territory.

How the Framework Works

The system relies on a specialized set of agents to manage the ideation lifecycle:

  • The Ideator: Generates initial research proposals based on a background corpus of literature.

  • The Stenographer: Summarizes complex drafts into a concise, structured format to make comparisons easier.

  • Evaluators: A suite of agents assesses each idea for non-obviousness, mechanism clarity, and feasibility. A "Soundness Panel" uses multiple independent judgments to ensure the logic and mathematical consistency of the proposed research.

  • The Diversity Judge: Compares new proposals against active, historical, and rejected ideas to ensure the new direction is genuinely distinct.
    If an idea shows promise but has specific flaws, the system triggers a "repair" or "refinement" subroutine, providing targeted feedback to improve the proposal without discarding the underlying concept.

Measuring Success with Yield

To evaluate its performance, the research team introduced a new metric called "Yield." Unlike traditional metrics that can be easily gamed, Yield calculates the largest set of ideas that meet a minimum threshold for quality while remaining mutually diverse. It essentially measures the number of viable, high-quality research paths a human researcher could actually pursue from the generated set.

Key Findings

When tested across 32 topics in 8 different Computer Science domains, IDEAgent demonstrated significant improvements over existing baselines. It outperformed the best alternative system by 3.89x on the Yield metric and achieved non-zero Yield on 8x more topics. The study confirms that the system’s ability to perform targeted repairs and refinements is essential for building logical rigor and clarity while ensuring that the generated ideas remain non-obvious and scientifically valuable.

Comments (0)

No comments yet

Be the first to share your thoughts!