Xiaomi MiMo and TileRT enable 1-trillion-parameter model to decode over 1000 tokens per second
Xiaomi MiMo and TileRT Push a 1-Trillion-Parameter Model Past 1000 Tokens Per Second on Commodity GPUs
MarkTechPost
Summary
Xiaomi's MiMo team released MiMo-V2.5-Pro-UltraSpeed, which decodes over 1000 tokens per second on a 1-trillion-parameter model using a single 8-GPU commodity node. The serving mode extends the MiMo-V2.5-Pro model for high-speed inference.