Re-Search: Dissecting LLM Ideation Through a Two-Player Questioning Game
A study of research-idea discovery finds most model progress happens early, with later questioning often plateauing. For agentic research tools, it suggests candidate generation may be a bigger bottleneck than judging ideas.
Re-Search evaluates idea discovery as a yes-or-no questioning game built from **200 ICML 2026 papers**. Across six models, nearly all progress happened in the first **30 of 100 questions**, after which most plateaued.
For research agents, the results suggest improving candidate generation or bringing in outside ideas may help more than only improving a model’s ability to assess proposals. The study separates ideation from execution, making the search process easier to inspect.
Re-Search evaluates idea discovery as a yes-or-no questioning game built from **200 ICML 2026 papers**. Across six models, nearly all progress happened in the first **30 of 100 questions**, after which most plateaued. For research agents, the results suggest improving candidate generation or bringing in outside ideas may help more than only improving a model’s ability to assess proposals. The study separates ideation from execution, making the search process easier to inspect. This is a preprint evaluation of rediscovering published papers’ core ideas; it does not show that the same pattern holds for open-ended research work or deployed coding agents.
The early plateau makes the supply of candidate ideas a separate bottleneck from judging them: extending the same questioning process may add little after initial progress. This sharpens where ideation agents might need intervention, while the paper’s rediscovery setup does not establish the pattern for open-ended agent work.