SlimSearcher: 적응형 보상 게이팅으로 효율적인 웹 에이전트 학습
SlimSearcher: Training Efficiency-Aware Web Agents via Adaptive Reward Gating
- 매체
- Hugging Face
- 게시
- 중요도
- 3/5
- 화제 지수
- 8
한국어 요약
SlimSearcher는 감독 학습과 강화학습 단계에서 정확도와 계산 비용의 파레토 최적점을 추구하는 웹 에이전트 학습 프레임워크다. 파레토 효율적 궤적 필터링과 적응형 보상 게이팅을 통해 불필요한 도구 호출과 토큰 소비를 줄이면서도 정확도를 유지하거나 개선한다. GAIA, BrowseComp, XBenchDeepSearch 벤치마크에서 도구 호출 횟수를 17~58% 감소시켰다.
원문 발췌 · English
Deep research agents have demonstrated remarkable capabilities in complex information-seeking tasks, yet this power comes at a steep computational cost. Driven by accuracy-focused training paradigms, current models adopt brute-force strategies characterized by blind tool dependency and performative reasoning-generating long, redundant trajectories that are far from necessary for resolving these tasks, leading to wasteful tool calls and excessive token consumption. To overcome this efficiency trap, we propose SlimSearcher, a principled framework that pushes the Pareto frontier between accuracy and computational cost across both Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL). In the SFT stage, SlimSearcher employs Pareto-efficient filtration to distill trajectories that are both successful and economical, guiding the model toward inherently efficiency-aware search behaviors. During RL, we introduce Adaptive Reward Gating, a dynamic reward-shaping mechanism that evaluates relative tool and token efficiency within a sampled cohort. By cascading these adaptive efficiency metrics with a strict correctness gate, our approach effectively avoids the brevity bias associated with absolute penalties and mitigates reward hacking. Extensive experiments on long-horizon benchmarks, including GAIA, BrowseComp, and XBenchDeepSearch, demonstrate that SlimSearcher reduces average tool-call rounds by 17%-58% while maintaining or improving accuracy.