LatentSkill: LLM 에이전트를 위한 텍스트 스킬의 가중치 공간 변환
LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agents
- 매체
- Hugging Face
- 게시
- 중요도
- 3/5
- 화제 지수
- 13
한국어 요약
LatentSkill은 텍스트 기반 스킬을 LoRA 어댑터로 변환하는 프레임워크로, 스킬 지식을 컨텍스트 공간 대신 가중치 공간에 저장한다. ALFWorld와 Search-QA에서 기존 인컨텍스트 스킬 대비 성능을 향상시키면서 프리필 토큰을 64% 이상 감소시켰다.
원문 발췌 · English
Agent systems increasingly use textual skills to encode reusable task procedures, but injecting these skills into the prompt at every step incurs substantial context overhead and exposes skill content as plaintext. We present LatentSkill, a framework that converts textual skills into plug-and-play LoRA adapters through a pretrained hypernetwork. LatentSkill stores skill knowledge in weight space rather than context space, removing per-step skill tokens while preserving modular loading, scaling, and composition. On ALFWorld and Search-QA, LatentSkill outperforms the corresponding in-context skill baseline while using substantially fewer prefill tokens: it improves ALFWorld success by 21.4 and 13.4 points on the seen and unseen splits with 64.1% fewer prefill tokens, and improves Search-QA exact match by 3.0 points with 72.2% lower skill-token overhead. Further analysis shows that generated skill LoRAs form a structured semantic geometry, can be precisely controlled via the LoRA scaling coefficient, and can be composed through parameter-space arithmetic when skill components are aligned. These findings suggest that weight-space skills provide an efficient, modular, and less exposed substrate for extending LLM agents.