[Paper Review] WebDancer: Towards Autonomous Information Seeking Agency

Recently, with the rise of high-performance models such as LG's EXAONE-4.0, OpenAI’s GPT-o series (o1, o3)^[1], DeepSeek-R1^[2], the evolution of LLMs has rapidly shifted from instruction-following models, which tend to provide straightforward answers, toward agents capable of deep reasoning. These models are commonly referred to as Large Reasoning Models (LRMs) and are distinguished from traditional instruction models by their ability to perform multi-step reasoning to address complex real-world problems.

Instruction models such as GPT-3.5 and GPT-4 were optimized for providing quick responses to given questions or for performing short chains of thought. In contrast, LRMs follow a more human-like reasoning process: they analyze problems step by step, attempt different strategies, recognize and correct mistakes, and arrive at answers through a long chain of thought (long-CoT).

Thanks to their advanced reasoning capabilities, LRMs exhibit doctoral-level problem-solving skills across diverse scientific and technological fields such as physics, chemistry, and biology, and they outperform traditional instruction models on a wide range of benchmarks^[3].

However, despite these strengths, LRMs also have inherent limitations. Specifically, if an information gap occurs during long reasoning, the model may depend solely on its internal knowledge, leading to hallucinations^[4]. As the reasoning process extends, not only can errors at intermediate steps accumulate and propagate to the final output, but it may also result in unnecessarily prolonged reasoning, or over-thinking^[5].

A new direction for addressing these limitations is the agentic LRM, which actively leverages external knowledge. In this approach, the model detects knowledge gaps during the reasoning process and supplements its own knowledge by drawing on external sources when needed. Various models and services reflecting this trend have emerged; representative examples include OpenAI’s ChatGPT (Deep Research), Google DeepMind’s Gemini, xAI’s Grok3, and LG AI Research’s ChatEXAONE (Deep Research).

In this article, I introduce WebDancer from Alibaba’s Tongyi Lab, a paper that proposes enhancing long reasoning in LRMs through the use of web search tools^[6]. The paper takes a holistic view of the agentic LRM domain, covering everything from key ideas to data generation and training architecture, which I found to be a valuable starting point for understanding this field.

1. Synthetic data generation : CRAWLQA, E2HQA

Image 1. Pipeline of synthetic-data generation

Agentic LRMs require more than simple question answering—they should be able to carry out complex, long-term reasoning and supplement their knowledge by actively searching for external information when required. To address this, WebDancer does not rely solely on open-source QA datasets that do not require complex reasoning. Instead, it generates synthetic QA datasets specifically designed to help models naturally learn exploration-based reasoning and strategies for supplementing knowledge via search. For this purpose, the authors introduce two key methodologies: CRAWLQA and E2HQA.

2.1 CRAWLQA

CRAWLQA is a dataset designed to help models learn how to gather and evaluate information across multiple webpages. Unlike QA tasks that rely on a single document, it takes a multi-hop approach, where deriving the correct answer requires considering multiple subpages together. The aim is to enable models to develop the ability to reason while navigating multiple webpages and synthesizing information. The generation procedure is as follows.

(1) Knowledgeable website collection
First, knowledgeable websites such as arXiv, GitHub, and Wikipedia are selected. These serve as root pages for information, expanding into related documents through sub-links.

(2) Recursive subpage collection
Links contained within the root pages are followed to recursively gather related subpages. On arXiv, for example, this includes references, related papers, and author profile pages.

(3) LLM-based QA generation
The collected pages are then grouped and provided as input to an LLM (e.g., GPT-4o) to generate question–answer (QA) pairs based on content spanning multiple documents. These pairs are designed to include not only simple factoid questions but also diverse types that require reasoning. Through this process, the generated CRAWLQA dataset includes the following types of data:

1. COUNT: Counting specific items (e.g., “How many open-source tools are mentioned in this document?”)

2. MULTI-HOP: Connecting clues across multiple subpages (e.g., “Which of the methods proposed in this paper has been implemented on GitHub?”)

3. INTERSECTION: Inferring intersections across sub-documents (e.g., “Which dataset is used in both Paper A and Paper B?”)

 

2.2 E2HQA: Easy-to-Hard

E2HQA (Easy-to-Hard QA) is a dataset that begins with simple factoid questions and progressively transforms them into forms that require more complex and in-depth reasoning. The key idea is to keep the correct answers unchanged, while augmenting the questions so that the model must take into account more information and carry out longer reasoning steps to arrive at those answers. The generation procedure is as follows.

 

1. (1) Preparing the QA dataset: Collect existing (open-sourced) question–answer pairs in the SimpleQA style.

2. (2) Entity extraction: Extract core entities from the questions, such as people, concepts, or organizations.

3. (3) Collecting entity-related information: Gather additional information about each entity from the web.

4. (4) Question rewriting: Use the collected information to have an LLM (e.g., GPT-4o) rewrite the questions into longer and more complex forms.

5. (5) Iterative refinement: Repeat steps 2–4 so that the questions progressively evolve into structures requiring multi-step reasoning.

 

Below are examples of questions evolved at each step, Q1, Q2, and Q3:

 

1. Q₁: "Who won the 2010 IEEE Frank Rosenblatt Award?"

2. Q₂: "Who received the 2010 AI-related award for contributions to fuzzy logic?"

3. Q₃: "Who won the 2010 Distinguished Technology Award for contributions to AI through the application of fuzzy control theory in industrial automation?"

 

2. Framework of WebDancer

Image 2. Framework of WebDancer

 

WebDancer builds its model framework on the ReAct (Reasoning + Acting) paradigm. ReAct[7] is a representative agent framework in which an LLM tackles complex problems by reasoning step by step, taking necessary actions, and feeding the resulting observations back into subsequent reasoning, where::

1. Thought: The model's internal reasoning to decide what action to take next

2. Action: The use of explicit tools to obtain external information or achieve goals

3. Observation: Information obtained from the results of actions

 

These three components (i.e., Thought-Action-Observation) together form a single step, and the model progressively solves the problem by generating the next step’s thought based on the results of the previous one, thereby constructing a trajectory. In WebDancer, two tools are defined as actions available to the agent:

1. 1) Search action
   - Description: Searches the external web to find the required information
   - Input parameters: (1) query — the search query, (2) filter_year — filter for search results by year
   - Observation: Returns the top 10 web search results using the Google API

2. 2) Visit action
   - Description: Reads the content of a specific web page directly and summarizes the desired information
   - Input parameters: (1) url_link — the URL of the page to visit, (2) goal — the information objective to obtain from the page
   - Observation: Generates evidence by summarizing the part of the page body relevant to the goal, using an LLM

 

3. Trajectory Filtering: A procedure for obtaining high-quality training data

WebDancer does not use the trajectories (the repeated Thought–Action–Observation cycles) generated by the agent directly for training. Instead, it applies a multi-step filtering process to select only high-quality data, which is then used for SFT training. To this end, WebDancer introduces a funnel-style filtering pipeline composed of three stages.

 

1. (1) Validity Control: Format validation
   First, it checks whether the trajectory follows the prescribed format—the Thought–Action–Observation structure—and whether the defined actions are executed correctly. Only data that meets these criteria is retained.

2. (2) Correctness Verification: Answer verification
   In the second step, it verifies the correctness of the trajectory’s final answer. This is done using the GPT-4o-based LLM-as-Judge method, and only data that the LLM deems correct is collected.

3. (3) Quality Assessment: Quality filtering
   Even if a trajectory produces the correct final answer, it may still hinder training if it is unnecessarily long, illogical, redundant, or irrelevant to the goal. In this final step, such data is filtered out with the help of an LLM.

4. Model training: Training with SFT and RL

Image 3. Training pipeline of WebDancer

WebDancer’s agent is trained in two stages. First, supervised fine-tuning (SFT) is applied to establish the agent’s basic inference–action framework in a stable way. Then, reinforcement learning (RL) is used to enhance generalization, improving accuracy, consistency of reasoning, and efficiency of actions. This two-stage training structure is a common strategy for enabling agentic LRMs to function effectively, and WebDancer follows this approach as well.

4.1 Supervised learning: Supervised Fine-Tuning (SFT)

SFT is the first step in the WebDancer training pipeline and serves as the essential “cold start” process. For an agent LRM to tackle reasoning-based problems, it must first acquire the ability to understand and follow trajectories in the ReAct format. Without this foundation, applying RL directly can result in unstable training or even failure.

WebDancer directly leverages the high-quality trajectory data generated as described in Chapter 3 for SFT. Using this data, the model computes the cross-entropy loss as defined in the following formula:

$L=\sum _{i=1}^{\left|H\right|}\frac{\mathrm{𝕀}\left[x_i\ne o\right]}{\left|H\right|-{\sum }_j\mathrm{𝕀}\left[x_j=o\right]}·\log {\pi }_{\theta }\left(x_i\mid tc,x_{<i}\right)$

Here, $H=\left\{x_1,\dots ,x_{\left|H\right|}\right\}$ denotes the entire trajectory sequence, and each  $x_i\in \left\{{\tau }_i,{\alpha }_i,o_i\right\}$ represents the i-th thought–action–observation. tc refers to the task context (the question and prompt given to the model). Since the Observation component is not part of the model’s generative output, it is excluded from the loss calculation, which is enforced by the $\mathrm{𝕀}\left[x_i\ne o\right]$ term in the loss function.

4.2 Reinforcement learning: On-policy RL with DAPO

WebDancer applies on-policy reinforcement learning with the DAPO (Decoupled Advantage-weighted Policy Optimization) algorithm to agents initialized through supervised learning. DAPO is a GRPO-based policy optimization method that updates the policy through trajectory sampling → reward calculation based on LLM evaluation → advantage adjustment at the token level according to token likelihood. The objective function for policy optimization is as follows:

${\mathcal{J}}_{\mathrm{DAPO}}\left(\theta \right)={\mathbb{E}}_{(q,a)}\left[\frac{1}{{\sum }_{i=1}^G\left|o_i\right|}\sum _{i=1}^G\sum _{t=1}^{\left|o_i\right|}\min \left(r_{i,t}\left(\theta \right)·{\hat{A}}_{i,t},clip\left(r_{i,t}\left(\theta \right),1-{ϵ}_{\text{low }},1+{ϵ}_{\text{high }}\right)·{\hat{A}}_{i,t}\right)\right]$

Here, $o_i$ denotes the tokens generated by the model in the $i$-th trajectory, excluding Observations. $r_{i,t}\left(\theta \right)$ represents the token-level likelihood ratio between the current policy and the old policy:

$r_{i,t}\left(\theta \right)=\frac{{\pi }_{\theta }\left(o_{i,t}\mid q_i,o_{i,<t}\right)}{{\pi }_{{\theta }_{\text{old }}}\left(o_{i,t}\mid q_i,o_{i,<t}\right)}$

${\hat{A}}_{i,t}$ is the normalized advantage of $o_i$ computed from the reward $R_i$ using z-score normalization over the set of sampled rollout rewards $\left\{R_j\right\}$:

${\hat{A}}_{i,t}=\frac{R_i-mean\left(\left\{R_j\right\}\right)}{std\left(\left\{R_j\right\}\right)}$

The reward $R_i$ is produced by an LLM judge (a GPT-based model) and is calculated based on two factors: answer correctness$( scor{e}_{answer} )$ and format appropriateness $\left(scor{e}_{\mathit{\text{format }}}\right)$:

$R\left(y_i,y\right)=0.1·{\text{ score }}_{\text{format }}+0.9·{\text{ score }}_{\text{answer }}$

With this design, WebDancer strengthens accurate reasoning and actions while naturally suppressing unnecessary or illogical tokens, enabling fine-grained learning.

5. Experiment and analysis

5.1 Main experiment results

Image 4. Experimental results

The authors evaluated WebDancer on the GAIA and WebWalkerQA benchmarks. The comparison included several baselines: Base (no search), RAG (single-turn retrieval for each question), ReAct, and WebDancer, with OpenAI’s DeepResearch (OpenAI DR) added as an additional reference. Results show that while OpenAI DR achieved the highest GAIA score of 67.4, WebDancer consistently outperformed all other baselines. In particular, when combined with QwQ-32B, it reached 51.5 on GAIA and 47.9 on WebWalkerQA, surpassing even the GPT-4o-based ReAct.

5.2 Analysis

5.2.1. Analysis of training data quality

Image 5. Comparison of training data effectiveness

First, when trained solely on open-source QA (open-only) data, the model showed the lowest performance. This indicates that factoid-centric QA data is not effective on benchmarks like GAIA, which demand complex reasoning.

Furthermore, the results trained on the authors' proposed CrawlQA and E2HQA datasets showed a clear performance gap depending on whether trajectory filtering was applied. The filtered final subset (6,550 instances) achieved better performance than using the full dataset, indicating that training on high-quality trajectories is far more effective, even with a smaller data volume.

5.2.2 Analysis of reinforcement learning effects

Image 6. Effect of reinforcement learning on performance

Figure 6 presents the results showing that reinforcement learning with the DAPO algorithm substantially improved performance. Applying RL led to consistent gains in both Pass@3 and Consistency@3, with the effect being more pronounced for models with weaker initial reasoning ability. For instance, QwQ-32B improved by +11.6pt in Pass@3 and +9.7pt in Consistency after RL compared to SFT-only, while the 7B model improved by +7.7pt, indicating positive changes across all model sizes. In addition, the authors observed that as training progressed from Base to SFT to RL, both the average reasoning length and the number of actions increased, along with improvements in reasoning consistency.

6. Conclusion

WebDancer is an agentic LRM framework designed to address complex web-based information retrieval tasks, presenting an end-to-end methodology spanning data generation, trajectory collection, SFT, and reinforcement learning. Experimental results show that WebDancer achieved performance comparable to or surpassing GPT-4, demonstrating that combining high-quality trajectories with an SFT + RL setup yields significant improvements in accuracy, consistency, and reasoning depth. Importantly, WebDancer highlights the possibility of building highly capable agents even in open-source environments.

LG AI Research also provides an “in-depth research” service through ChatEXAONE, where users can experience the agentic LRM service. EXAONE employs various tools like web search and document retrieval to conduct in-depth reasoning and generate detailed answers to user queries.

▶ View Original Publication (arXiv)