SAE-Xplainers: Rule-Based Feature Interpretation for Extreme Earth Events
SAE-Xplainers combines geographic modulation and rule-based models to enhance interpretability of extreme weather event predictions, improving reconstruction and understanding.
Key Findings
Methodology
This work introduces GeoTopK, a location-aware k-sparse autoencoder, integrating geographic information via an MLP encoder to modulate activations, capturing local environmental features. Coupled with SAE-Xplainers, which utilize Skope-Rules to translate high-dimensional sparse features into human-understandable rule sets, the framework enables global and local interpretability. The method is validated on fire prediction, tropical cyclone detection, and atmospheric river identification, outperforming baseline models in reconstruction metrics and feature sparsity, while providing faithful rule-based explanations.
Key Results
- GeoTopK achieved R2 scores of 0.937 and 0.953 on SeasFire and ClimateNet datasets, respectively, surpassing traditional TopK by 0.01-0.02, with dead feature rates near zero, demonstrating effective spatial feature disentanglement.
- SAE-Xplainers attained approximately 93% and 85% prediction fidelity on the two datasets, successfully revealing complex climate patterns through rule sets, validating their interpretability and robustness.
- Analysis of feature absorption revealed the model's ability to identify hierarchical concept overlaps, addressing feature fragmentation issues, and providing insights into model behavior.
Significance
This research advances the transparency of deep learning models in climate science, enabling trustworthy interpretation of extreme weather predictions. The integration of geographic modulation enhances regional specificity and generalization, crucial for policy and risk management. The rule-based explanations foster scientific understanding, bridging the gap between complex models and domain expertise, thus promoting interdisciplinary collaboration and practical deployment.
Technical Contribution
The key innovation lies in the development of GeoTopK, a location-conditioned SAE training strategy, and the ensemble of rule-based SAE-Xplainers that provide scalable, global interpretability. This approach addresses the polysemanticity and spatial heterogeneity of weather data, offering a theoretically grounded and engineering feasible solution for climate modeling. It extends the capabilities of traditional autoencoders and rule learning algorithms, supporting multi-modal, high-dimensional data analysis.
Novelty
This is the first work to incorporate explicit geographic information into sparse autoencoder training for climate data, creating a location-sensitive feature extraction process. The rule-based interpretation of sparse features in a global context is novel, providing a scalable alternative to local attribution methods. The combination of spatial modulation and rule learning offers a new paradigm for interpretable climate modeling, setting a foundation for future research.
Limitations
- Model performance depends on accurate geographic data; inaccuracies can impair feature disentanglement.
- Rules may oversimplify complex interactions, risking loss of nuanced information in some scenarios.
- Training complexity and computational costs are high, especially for large-scale, real-time applications.
- Limited data on rare extreme events may restrict model robustness in those cases.
Future Work
Future directions include integrating multi-modal datasets, optimizing rule complexity for better interpretability, and developing adaptive spatial-temporal modules for real-time forecasting. Further research will explore automatic rule refinement and transfer learning to improve robustness across diverse climate regimes, aiming for deployment in operational weather monitoring systems.
AI Executive Summary
Predicting and understanding extreme weather events remains a central challenge in climate science. While deep learning models have demonstrated impressive accuracy, their opaque nature hampers trust and scientific validation. This study introduces SAE-Xplainers, a novel framework that combines geographic modulation with rule-based interpretability to address this gap. Central to the approach is GeoTopK, a location-aware k-sparse autoencoder that leverages a neural network encoder to incorporate geographic coordinates, enabling the model to learn region-specific features. This spatial modulation significantly improves the model’s ability to reconstruct environmental patterns, reducing dead features and enhancing interpretability.
Building on this, the SAE-Xplainers utilize Skope-Rules to translate the high-dimensional sparse features into human-readable rules. These rules, expressed as if-then statements, faithfully approximate the model’s internal representations, providing transparent insights into the climatic patterns associated with wildfires, tropical cyclones, and atmospheric rivers. Extensive experiments on SeasFire and ClimateNet datasets demonstrate that the combined framework outperforms traditional autoencoders and feature attribution methods, with R2 scores exceeding 0.93 and rule interpretability accuracy reaching 85-93%. The analysis of feature absorption phenomena further reveals how hierarchical climate concepts are represented and sometimes fragmented within the model.
This work significantly advances the field of climate AI by offering a scalable, interpretable, and scientifically consistent approach to extreme weather prediction. It bridges the gap between complex deep models and domain expertise, facilitating better policy decisions and risk assessments. The integration of geographic information and rule-based explanations sets a new standard for transparent climate modeling, with promising avenues for future enhancements such as multi-modal data fusion, real-time inference, and adaptive rule learning. Despite current limitations like data dependency and computational costs, this framework paves the way for more trustworthy and actionable climate intelligence in the face of global climate change.
Deep Dive
Abstract
The emergence of large-scale Weather and Climate (W&C) datasets offers new opportunities for modeling extreme Earth events (ExEE) and their impacts using deep learning. However, their adoption in operational settings remains limited by the lack of models' interpretability. While for conventional text and image modalities, tools such as Sparse Autoencoders (SAEs) have proven effective for extracting human-understandable concepts, their use for the analysis of ExEE remains challenging due to the nature of W&C data. To address this, we introduce (i) a geographic location-based modulation of the inputs of SAE to capture the local semantic meaning of environmental patterns, and (ii) an ensemble of rule-based SAE-Xplainers to interpret the resulting high-dimensional features derived from complex, multi-modal environmental predictors. We evaluate our method on three ExEE types: the prediction of fires, and the detection of tropical cyclones and atmospheric rivers. We show that SAE input modulation improves both reconstruction performance and feature utilization, and that our SAE-Xplainers enable faithful interpretation of complex climatic patterns by unfolding them into human-understandable rules that are consistent with the scientific literature, while also supporting the identification of feature absorption.