이질적 촉각 표현 학습으로 일반화된 다재다능 조작 실현
Learning Heterogeneous Tactile Representation for General Dexterous Manipulation
- 매체
- Hugging Face
- 게시
- 중요도
- 3/5
- 화제 지수
- 22
한국어 요약
접촉이 많은 고수 조작을 위해 서로 다른 촉각 센서로부터 통합된 표현을 학습하는 기초 모델 LingBot-Touch를 제안했다. 압저항식·자기식·시각촉각식 센서 간 호환성을 제공하며, 확산 정책과 VLA 모델 같은 다양한 정책 모델에 플러그앤드플레이로 통합되어 조작 섬세성을 향상시킨다. 시뮬레이션과 실제 로봇 배치 결과 복잡한 조작 작업에서 높은 계산 효율을 유지하면서 상당한 성능 향상을 보였다.
원문 발췌 · English
Tactile sensing is an indispensable modality for contact-rich dexterous manipulation, a capability that hints at the ultimate frontiers of embodied intelligence. However, commercially available tactile sensors are still at a relatively early stage of development, with each type of sensor offering irreplaceable strengths alongside its own drawbacks. A key challenge is thus to free tactile understanding from sensor-specific physics, so that embodied models can effectively leverage heterogeneous tactile data. To this end, we develop LingBot-Touch, a tactile foundation model characterized by the following properties. (i) Our model yields a unified representation across different tactile sensors (i.e., piezoresistive, magnetic, and visuotactile sensors), and is easily extensible to new sensor types. (ii) The resulting tactile tokens are plug-and-play: they can be seamlessly integrated into policy models across paradigms, from diffusion policy to vision-language-action (VLA) models such as π_{0.5} and LingBot-VLA, consistently enhancing manipulation dexterity. (iii) Extensive experiments, spanning simulation and real-robot deployments across multiple hardware platforms, demonstrate that our tactile representation substantially benefits complex manipulation tasks while maintaining strong computational efficiency. We believe the findings in this work demonstrate the potential of leveraging cross-sensor data to scale tactile learning, and offer encouraging insights for the design of future hardware and embodied models.