ASpec, a high-throughput training framework for speculative decoding models
Original titleinclusionAI/ASpec
- Outlet
- GitHub
- Published
- Importance
- 2/5
- Trending
- 16
Summary
ASpec is a high-throughput training framework designed for speculative decoding models to accelerate LLM inference.
Original excerpt
A high-throughput training framework for Speculative decoding models. · topics: dflash, dflash2, dspark, eagle3, eagle4, ling-3, ling-3-flash, llm, mooncake, pytorch, sglang, speculative-decoding