CarryOn augments vision-language-action models with personalization and safety assurance
Original titleCarryOn: A Dual-Loop Harness for Personalized and Reliable Robot Assistance
- Outlet
- arXiv
- Published
- Importance
- 3/5
- Trending
- 4
Summary
CarryOn is a dual-loop harness that augments a frozen vision-language-action policy. Its planning loop maintains persistent user and scene context for personalized task execution, while its execution loop actively supervises rollouts and shields against hazardous actions. On a real robot, it reduced safety violations from 52.6% to 6.1% and increased recovery success from 13% to 80%.
Original excerpt
While generalist vision-language-action (VLA) models have enabled diverse open-world manipulation, real-world robotic assistance requires two critical capabilities they currently lack: personalization and reliable execution. First, executing an instruction like "make me a sandwich" requires grounding subtask decomposition in persistent user preferences rather than relying on a single generic solution. Second, reliable deployment demands explainable safety assurance that proactively prevents hazards and gracefully recovers from occasional physical failures. To address these gaps, we present CarryOn, a dual-loop harness designed to augment a frozen VLA policy. The planning loop maintains persistent user-level and scene-level context that is continually grounded against new observations and revised through interaction, enabling personalized task selection and execution grounding across conversations and changing scenes. The execution loop actively supervises policy rollouts, shielding actions from predicted hazardous contact. When execution fails, the system invokes reusable control-based recovery skills that restore task progress and are refined through reflection on earlier failures. On a real robot and in simulation, CarryOn dispatches preference-consistent subtasks across users and scenes, reduces the violation rate of an already safety-finetuned VLA from 52.6% to 6.1%, and increases recovery success from 13% to 80% when refined skills are reused.