UI-MOPD: 다중 플랫폼 GUI 에이전트 지속 학습을 위한 온폴리시 증류
UI-MOPD: Multi-Platform On-Policy Distillation for Continual GUI Agent Learning
- 매체
- Hugging Face
- 게시
- 중요도
- 3/5
- 화제 지수
- 14
한국어 요약
다중 플랫폼 GUI 에이전트 학습의 과제를 해결하기 위해 고품질 크로스플랫폼 상호작용 데이터셋 Uni-GUI와 다중 교사 온폴리시 증류 기법 UI-MOPD가 제안됐다. 플랫폼별 교사를 동적으로 선택하여 플랫폼 특화 행동 패턴을 공유 정책으로 전이하며, OSWorld와 MobileWorld에서 각각 38.2%, 12.0%의 작업 성공률을 달성했다.
원문 발췌 · English
Recent advances in multimodal foundation models and agent systems have driven GUI agents from single-platform task execution toward cross-platform interaction. However, building multi-platform GUI agents remains challenging. On one hand, high-quality and executable cross-platform interaction trajectories are still scarce, and existing data often suffer from limited platform coverage. On the other hand, different platforms exhibit distinct interaction conventions, making joint or continual training prone to behavioral pattern mixing, platform-specific capability degradation, and catastrophic forgetting. To address these challenges, we construct Uni-GUI, a high-quality cross-platform GUI interaction dataset, and propose UI-MOPD, the first method that incorporates multi-teacher on-policy distillation into continual learning for GUI agents. UI-MOPD dynamically selects a platform-specific teacher according to the current environment and transfers platform-specific behavioral priors to a shared policy through platform-conditioned distillation, enabling adaptation to new platforms while preserving capabilities on existing ones. Experiments on OSWorld and MobileWorld show that UI-MOPD achieves task success rates of 38.2% and 12.0%, respectively, demonstrating its effectiveness in balancing cross-platform capability retention and new-platform adaptation. Project page: https://elispectre.github.io/UI-MOPD/.