Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2
Open-source JumpReLU-based sparse autoencoders trained on all layers of Gemma 2, enhancing interpretability of large language models.
Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy et al.