A new benchmark from Tencent Hunyuan and Tsinghua University shows that AI research agents fail not because they search poorly, but because they rarely stop to ask users for clarification when queries are vague.
AI search agents running multi-step research tasks break down not at the retrieval stage but when they encounter ambiguous queries and fail to ask users for help — that is the central finding of a new benchmark called DiscoBench, developed by researchers at Tencent Hunyuan and Tsinghua University [1].
For developers building agentic search pipelines, the implication is direct: adding more search calls does not compensate for a model that never surfaces its own uncertainty to the user [1].
What DiscoBench Tests
Earlier benchmarks such as GAIA and BrowseComp assume user queries are complete and unambiguous, which does not reflect real-world usage [1]. DiscoBench fills that gap with 211 tasks covering 463 ambiguous points across eleven knowledge domains, including video games, sports, music, film, science, and politics [1]. The dataset is written primarily in Chinese to reflect typical search patterns on the Chinese-language web [1].
Each task is broken into checkpoints where the agent must choose one of three actions: keep searching, ask the user a clarifying question, or deliver an answer [1]. The researchers define four categories of ambiguity — a description matching multiple entities, time-period or version confusion, unclear ranking criteria, and outright factual errors in the query itself [1]. All search queries ran through the Tavily agent search engine, and Gemini 3 Flash served as a user simulator [1].
How Eleven Models Performed
The team tested eleven models released in the past six months [1]. Without any explicit hint about possible ambiguity, Doubao Seed 2.0 Pro achieved the highest end-to-end accuracy at 43.1 percent, followed by Gemini 3.1 Pro Preview at 40.8 percent and Claude Opus 4.7 at 39.8 percent [1]. Weaker performers included MiniMax M2.7 at 16.1 percent and Qwen3.6 Max at 12.3 percent [1].
The gap between step-level and task-level scores is telling. Claude Opus 4.7 solves 57 percent of individual checkpoints correctly but finishes only 39.8 percent of full tasks — a single unresolved ambiguity is enough to collapse the entire research chain [1].
Searching More Does Not Help
The behavioral analysis is where the findings get most actionable. Agents that search first and then ask a follow-up question (“SearchThenAsk”) average a 93.4 percent success rate at ambiguous checkpoints [1]. Agents that guess without asking (“DirectGuess”) drop to 56.5 percent, and agents that search repeatedly but still guess instead of asking (“SearchHeavyGuess”) do even worse at 51.9 percent [1].
According to the authors, the repeated searches in that last pattern suggest the model already detected the ambiguity but never converted that detection into a user interaction [1]. Claude Opus 4.7 searches more often than most other models tested yet still trails Gemini 3.1 Pro Preview and Doubao Seed 2.0 Pro in overall accuracy [1].
Detection and Question Quality Are Separate Skills
The researchers also found that spotting ambiguity and asking a useful question about it are distinct capabilities that do not always travel together [1]. Qwen3.6 Max achieves a Detection F1 score of only 16 percent and asks an average of just 0.07 follow-up questions per task, but when it does ask, 94.7 percent of those questions are factually correct and 89.5 percent lead to progress [1]. MiniMax M2.7 asks far more often but achieves a follow-through rate of only 60.7 to 66.5 percent [1].
When the system prompt explicitly told agents to watch for ambiguity — a “Guided” mode intended to show the reachable ceiling — average end-to-end accuracy across ten models rose from 28.6 to 33.7 percent, while Detection F1 jumped more sharply from 45.3 to 64.9 percent [1]. The hint helped models spot ambiguity without reliably helping them complete the research; for Claude Opus 4.7, end-to-end accuracy actually dipped slightly under the guided prompt despite a higher checkpoint pass rate [1].
Removing search tools entirely causes scores to collapse: Doubao Seed 2.0 Pro falls from 43.1 to 2.4 percent accuracy, confirming that DiscoBench cannot be solved from stored model knowledge alone [1]. Conversely, removing ambiguity from the queries boosts accuracy by 26.8 to 40.2 percentage points depending on the model [1].
The authors conclude that future search agents need mechanisms that convert search uncertainty into user interaction, on top of their existing retrieval and reasoning capabilities [1]. Separately, Anthropic’s latest Claude Opus 4.8 update is designed to flag uncertainties more often, and Perplexity is experimenting with an approach called Search as Code that lets models write search workflows as Python programs rather than calling a prebuilt API [1].
Sources
This article was drafted with AI from the cited sources and checked against them before publication. Spot an error? Let us know.
