MMFlash, 멀티모달 모델 추론 최대 3.58배 가속
MMFlash: Faster Speculative Decoding for Vision-Language Models
- 매체
- Hugging Face
- 게시
- 중요도
- 3/5
- 화제 지수
- 13
한국어 요약
시각 의존성이 높은 토큰에 대해 가중치를 조정해 멀티모달 추측적 디코딩을 개선하는 MMFlash를 제안했다. 토큰-수준 시각 의존성 지표를 도입해 시각 입력 없을 때 대비 의존도가 높을수록 수용률이 46%에서 22%로 감소함을 확인했다. 긴 영상 입력을 위해 희소 주의를 추가로 활용한다. 여러 멀티모달 벤치마크에서 최대 3.58배 추론 속도 향상을 달성했으며, EAGLE-3과 DFlash 대비 각각 최대 195.4%, 17.1% 성능 향상을 기록했다.
원문 발췌 · English
Speculative decoding accelerates generation by letting a lightweight drafter propose tokens for parallel verification by a target model. Diffusion drafters such as DFlash propose an entire token block in a single forward pass, but mainly focus on text-only tasks, leaving multimodal drafting underexplored. To study this setting, we first introduce a metric of token-level visual dependence based on the KL divergence between next-token distributions with and without visual input. We find that acceptance rates decrease from 46% in the lowest visual-dependence quartile to 22% in the highest, with this disparity persisting after multimodal training. Motivated by this observation, we propose MMFlash, a simple and effective framework that adaptively upweights visually dependent tokens that the drafter predicts poorly. For long video inputs, MMFlash further employs sparse attention to reduce redundant attention computation and improve drafting quality. Across various multimodal benchmarks, MMFlash achieves lossless decoding speedups of up to 3.58times, outperforming EAGLE-3 and DFlash by up to 195.4% and 17.1%, respectively.