Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues
Extending LRNNs' eigenvalues to [-1,1] enables robust state-tracking, significantly improving performance on parity and modular tasks.
Riccardo Grazzi, Julien Siems, Arber Zela et al.