통일된 인과 학습으로 세계 모델과 행동 모델 개선
Can World and Action Modeling Benefit Each Other under Unified Causal Training?
- 매체
- Hugging Face
- 게시
- 중요도
- 2/5
- 화제 지수
- 17
한국어 요약
OneWM 프레임워크는 세계 모델링과 행동 모델링을 통일된 인과 학습으로 상호 개선한다. 실체화 테스트베드에서 행동 조건부 세계 예측이 정책 성능을 높이고, 행동 학습이 다중 시점 예측을 개선함을 확인했다.
원문 발췌 · English
World and action modeling are closely related, yet their reciprocal benefits under unified causal training remain unclear. We introduce OneWM, a simple framework for unified causal training that treats actions as both conditions for world prediction and targets for policy learning within a shared video-action architecture. Using this framework, we investigate whether learning forward dynamics conditioned on specified actions improves action learning, and whether learning to generate actions in turn improves action-conditioned world modeling. We use LIBERO, LIBERO-Plus, and RoboCasa365 as embodied testbeds to evaluate both policy learning and world modeling, complemented by real-world policy experiments. Learning the visual consequences of specified actions improves policy performance and generalization across changing scenes. In turn, action learning improves multi-view world-prediction fidelity under the same architecture. Together, these results answer both questions positively, supporting reciprocal benefits between world modeling and action generation under unified causal training.