AI Daybook
AI Daybook
오늘의 지면
논문연구·논문

WEAVE, 다중 PR 의존성 구조로 코드 에이전트 벤치마킹

WEAVE: Weaving Real-World Code Evolution into Dependency-Structured Tasks for Benchmarking and Training Code Agents

매체
arXiv
게시
중요도
3/5
화제 지수
4

한국어 요약

실제 소프트웨어 개발은 여러 PR에 걸친 조정 변경을 요구하지만 기존 벤치마크는 단일 이슈·PR 단위 작업만 제공한다. WEAVE는 검증된 PR 이력을 실행 가능한 의존성 구조 작업으로 변환하고, 저장소 특화 타입 변경 위치로 관련 PR을 DAG로 정리한다. 또한 의존성 일관된 부분 진행률을 측정하는 그래프 크레딧을 도입해 밀집 검증 보상을 제공한다.

원문 발췌 · English

Real-world software engineering rarely consists of isolated changes. Features, migrations, and systematic fixes often unfold across multiple pull requests (PRs), requiring coordinated modifications across a repository. Yet most executable repository-level benchmarks retain a single issue or PR as the task unit, while existing multi-PR settings rely on coarse aggregation and structure-agnostic evaluation. This mismatch obscures dependencies among changes and limits progress measurement and learning signals for long-horizon code agents. We introduce WEAVE, a framework that transforms verified PR histories into executable, dependency-structured tasks. WEAVE leverages repository-calibrated typed change loci to organize related PRs into bounded DAGs and instantiate each as a unified executable task. We further introduce Graph Credit, a deterministic, test-grounded metric that measures dependency-consistent partial progress and supports dense, verifiable rewards for reinforcement learning. Experiments on task construction, agent evaluation, supervised fine-tuning, and reinforcement learning show that WEAVE yields coherent executable tasks, enables more faithful progress measurement than flat scoring, and provides effective supervision and verifiable rewards for training code agents.

#reinforcement learning#benchmark#software engineering#code agent
원문 기사 읽기
전체 보기 →