LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding
LayerSkip combines layer dropout and early exit with self-speculative decoding, achieving up to 2.16× speedup in large language model inference.
Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich et al.