HuGe, 대규모 에고센트릭 데이터로 휴머노이드 VLA 학습
HuGe: Scaling Humanoid Generalist Vision-Language-Action Models with Egocentric Experience
- 매체
- Hugging Face
- 게시
- 중요도
- 4/5
- 화제 지수
- 13
한국어 요약
휴머노이드 로봇용 비전-언어-액션 모델 HuGe는 2000시간 이상의 인간 및 로봇 에고센트릭 데이터로 사전 학습했다. 시뮬레이션 및 실제 로봇(Unitree G1)에서 선택 및 배치 작업 성능을 높였으며, 시각적 구현 차이를 극복하기 위해 의사 체성감각 상태와 시각 적응 기법을 적용했다.
원문 발췌 · English
Extending Vision-Language-Action models (VLAs) to humanoid robots requires coordinated whole-body control and diverse demonstrations that are costly to collect in real-world scenarios. Although egocentric human demonstrations provide a complementary source of task experience, differences in visual embodiment and missing robot proprioception complicate transfer to humanoid policies. To address these challenges, we introduce HuGe, a family of Humanoid Generalist VLAs pre-trained on over 2,000 hours of egocentric human and robot data. Specifically, HuGe employs three existing representative VLA architectures to predict latent motion tokens for a pretrained whole-body controller, together with hand-control signals. Based on this, we introduce a three-stage training pipeline that combines egocentric pre-training, humanoid adaptation, and downstream fine-tuning. To facilitate transfer across embodiments, we construct pseudo-proprioceptive states through simulation playback during pre-training and modify arm appearance in egocentric observations through re-rendering during downstream fine-tuning. With this pipeline, HuGe achieves the state-of-the-art on simulation benchmark HumanoidArena. Furthermore, real-world experiments on the Unitree G1 indicate that augmenting robot demonstrations with visually adapted egocentric demonstrations significantly increases success on a pick-and-drop task.