Gamer, 장시간 복잡한 환경용 비전-언어-행동 기초 모델
Gamer: A Generalist Vision-Language-Action Foundation Model for Long-Horizon Real-Time Game Agent
- 매체
- Hugging Face
- 게시
- 중요도
- 3/5
- 화제 지수
- 16
한국어 요약
Gamer는 복잡한 상호작용 환경에서 장시간 작업을 위한 에이전트 프레임워크다. 지각·메모리·다단계 추론을 결합한 엔드투엔드 아키텍처로, 2D 그래픽 인터페이스와 3D 시뮬레이션 환경 모두와 상호작용한다. 수만 시간의 키보드·마우스 상호작용 데이터로 사전 학습한 뒤, 성긴 사고 명령어 튜닝으로 계산량을 줄였다. 보강 학습과 클라우드-엣지 추론 엔진으로 장시간 상호작용과 실시간 성능을 지원한다. 오픈월드 3D 게임에서 숙련 플레이어 수준의 성능을 보였다.
원문 발췌 · English
We introduce Gamer, a generalist agent framework built for long-horizon tasks in complex interactive environments. Its end-to-end architecture combines perception, memory management, and multi-step reasoning. A shared training and deployment pipeline supports interaction with both conventional 2D graphical interfaces and complex 3D simulated worlds. We first continue pre-training on a dataset constructed from tens of thousands of hours of device-level keyboard and mouse interaction. These trajectories span simulated games and embodied environments, allowing the model to learn environmental dynamics across both physical and virtual domains. Sparse-thinking instruction tuning then substantially reduces redundant computation while preserving the reasoning needed for complex decisions. We train the policy for long-horizon interaction with a stable multi-turn reinforcement learning method that addresses finite context windows, accumulated rollout errors, and sparse rewards. For real-time interaction, a cloud-edge inference engine serves numerous simulated game environments concurrently, supports unbounded-context execution, and keeps online inference latency low. Across multiple challenging 3D open-world games, Gamer demonstrates solid generalization and operational efficiency comparable to skilled players. It also achieves competitive results on a range of conventional 2D game benchmarks. These results suggest that even repetitive keyboard and mouse trajectories can support substantial gains from scaling data and learning algorithms. Training on these trajectories helps Gamer learn complex skills more steadily in environments with both physical and digital properties. This approach offers a path toward generalist agents that can interact across digital and physical worlds.