Linearized 2-Simplicial Attention
Proposes linearized 2-simplicial attention using random features for O(n) complexity, combined with Kimi Delta Attention, achieving state-of-the-art accuracy on long sequences.
Aritra Das, Dhruman Gupta, Debayan Gupta