DUET: Disaggregated Hybrid Mamba-Transformer LLMs with Prefill and Decode-Specific Packages
DUET accelerates Mamba-Transformer models by assigning prefill and decode phases to specialized packages, achieving 4x speed improvement.
Alish Kanani, Sangwan Lee, Han Lyu et al.