AI Daybook
AI Daybook
오늘의 지면
논문연구·논문

DLite, 에이전트 AI용 무기록 추측적 디코딩

DLite: History-free Speculative Decoding for Agentic AI

매체
arXiv
게시
중요도
3/5
화제 지수
2

한국어 요약

연구팀이 장문맥과 다중 턴 상호작용이 많은 에이전트 워크로드에서 기존 추측적 디코딩의 수용 길이 최대 80% 저하 문제를 개선한 DLite를 제시했다. 타겟 모델의 숨겨진 상태를 직접 재사용하고 기록을 명시적으로 유지하지 않으면서도, DSpark 대비 최대 70% 빠르고 2~6배 높은 수용 길이를 달성했다.

원문 발췌 · English

Speculative decoding has become a mainstream technique for accelerating large language model (LLM) inference by using a lightweight draft model to generate candidate tokens that are subsequently verified by the target model. However, agentic workloads introduce long and continuously evolving contexts and frequent multi-turn interactions. Under such workloads, we observe that existing draft models suffer from up to an 80% degradation in acceptance length, while maintaining their historical KV cache introduces substantial system overhead. In this paper, we identify the growing historical context as a common source of both problems and introduce DLite, a history-free speculative decoding framework for agentic AI. DLite directly reuses contextual hidden states from the target model and performs local speculative prediction without explicitly maintaining the historical context, transforming an increasingly complex modeling problem into local simple prediction problem. A lightweight sequential head further restores intra-block causal dependencies with low overhead. To effectively train this heterogeneous history-free architecture, we introduce a tailored two-stage training strategy. Experiments on general-purpose, long-context and agentic workloads show that DLite achieves 2-6 acceptance length and outperforms the state-of-the-art speculative decoding method, DSpark, up to 70%.

#llm#agent#speculative-decoding
원문 기사 읽기
전체 보기 →