HiRA-CAM: Preserving Fine-Grained Spatial Relevance in Gradient-Based Visual Explanations
HiRA-CAM enhances CNN explanations by hierarchical region agreement, outperforming LayerCAM and Grad-CAM with more stable, fine-grained saliency maps.
Key Findings
Methodology
HiRA-CAM builds on pixel-wise class activation maps (CAM) from all CNN layers, applying Gaussian smoothing, fixed-region partitioning, and soft voting to validate regions across layers. It emphasizes hierarchical agreement, where regions consistently highlighted across layers are reinforced, while noisy or layer-specific activations are suppressed. The process involves: computing pixel-level CAMs per layer, smoothing, normalization, dividing the spatial map into fixed grids, calculating region scores, applying quantile-based thresholds, soft voting for cross-layer consensus, and reconstructing a final region-weighted saliency map from the deepest layer, fused with residual smoothing. This approach stabilizes explanations and preserves fine spatial details without multi-layer fusion noise.
Key Results
- On ImageNet WSOL benchmarks, HiRA-CAM achieved loc1 scores of 45.9%, 45.1%, and 44.8% on VGG16, ResNet-50, and DenseNet-121 respectively, surpassing LayerCAM and Grad-CAM. It also improved pointing game accuracy and decreased deletion AUC, indicating better localization and causal relevance. The method consistently outperformed baselines across architectures, with improvements of 2-4% in localization metrics and significant gains in robustness against background noise.
- Ablation studies confirmed that the hierarchical agreement and region-based smoothing are critical for performance gains. The method effectively reduces false activations, enhances boundary precision, and maintains spatial stability in complex multi-object scenes. Results demonstrate that multi-layer spatial validation leads to more causally faithful explanations, especially in high-stakes applications like medical imaging and autonomous driving.
- Across multiple architectures, HiRA-CAM showed consistent improvements in localization accuracy, explanation faithfulness, and robustness, with the added benefit of avoiding the computational overhead associated with pixel-level multi-layer fusion. Its ability to generate compact, object-centric saliency maps makes it suitable for real-world deployment where interpretability and reliability are paramount.
Significance
This work addresses fundamental limitations of existing gradient-based visualization methods, notably the instability and coarse localization caused by naive multi-layer fusion. By introducing hierarchical region agreement, HiRA-CAM offers a principled way to stabilize explanations, preserve spatial details, and improve causal interpretability. It bridges the gap between high-level model decisions and human-understandable visual cues, crucial for deploying AI in safety-critical domains. The approach enhances both scientific understanding and practical trustworthiness of deep models, fostering broader adoption in sensitive fields.
Technical Contribution
The core innovation lies in integrating multi-layer pixel-wise activation maps through a hierarchical agreement framework, avoiding noisy fusion. The method employs Gaussian smoothing for regional stability, adaptive quantile thresholds for significance filtering, and soft voting to enforce cross-layer consistency. These components collectively produce more precise, stable, and causally relevant saliency maps without increasing model complexity. The algorithm is compatible with standard CNN architectures, requiring only post-hoc analysis, making it a versatile tool for explainability.
Novelty
This is the first approach to explicitly incorporate hierarchical regional agreement across all CNN layers for saliency map refinement. Unlike LayerCAM, which relies on direct gradient-weighted activations, HiRA-CAM validates regions through multi-layer consensus, significantly reducing noise and improving spatial fidelity. Its combination of Gaussian smoothing, adaptive thresholding, and soft voting introduces a new paradigm for stable, fine-grained explanations, setting it apart from existing global or single-layer methods.
Limitations
- The fixed grid partitioning may not align perfectly with object boundaries, potentially limiting boundary accuracy in irregular shapes.
- Computational overhead increases with model depth and resolution, posing challenges for real-time applications.
- The method's performance may degrade in scenes with highly occluded or overlapping objects, where region consistency becomes harder to validate.
Future Work
Future research will explore adaptive, content-aware region partitioning to better match object contours, as well as layer importance weighting to enhance robustness. Extending the framework to semantic segmentation and multi-object scenarios could further improve interpretability. Additionally, optimizing computational efficiency and integrating with end-to-end training for joint explanation and classification are promising directions.
AI Executive Summary
Deep neural networks, especially convolutional architectures, have revolutionized computer vision tasks but remain largely opaque in their decision-making processes. Existing visualization techniques like Grad-CAM and LayerCAM provide valuable insights but often suffer from coarse localization, noise, and instability, especially when multiple layers are fused. These limitations hinder their application in safety-critical domains such as medical diagnosis and autonomous driving, where precise spatial explanations are essential.
To address these challenges, this paper introduces HiRA-CAM, a hierarchical region agreement-based visualization method. It leverages pixel-wise activation maps from all CNN layers, applying Gaussian smoothing to reduce noise, then dividing the spatial domain into fixed regions. By computing region scores and employing adaptive quantile thresholds, HiRA-CAM performs soft voting across layers to identify regions with consistent importance. The final saliency map is reconstructed from the deepest layer, modulated by cross-layer consensus, ensuring spatial stability and detail preservation without noisy multi-layer fusion.
Experimental results on ImageNet WSOL benchmarks across VGG16, ResNet-50, and DenseNet-121 demonstrate that HiRA-CAM outperforms existing methods in localization accuracy, pointing game scores, and deletion AUC. It produces more compact, object-centric heatmaps, with bounding boxes closer to ground truth, and exhibits higher causal fidelity. The hierarchical agreement mechanism effectively suppresses false positives and enhances boundary precision, making explanations more trustworthy.
This approach significantly advances the interpretability of deep models, providing explanations that are both spatially precise and robust. Its simplicity, compatibility with existing architectures, and improved performance make it a promising tool for deploying explainable AI in real-world, high-stakes applications. Future work will focus on adaptive region partitioning, layer importance weighting, and extending to broader tasks like segmentation, further strengthening the role of explainability in AI development.
Deep Dive
Abstract
Deep Learning models can include billions of parameters or more, making it difficult to explain their internal transformations and outputs. However, explainability is increasing in importance due to the use of AI in crucial applications. This paper focuses on the interpretability of convolutional neural networks (CNNs). Building on the popular gradient based method LayerCAM for extracting internal features in CNNs, we propose an improved method named HiRA-CAM, and show that it outperforms both LayerCAM and Grad-CAM on creating useful saliency maps for object classification. The main feature of HiRA-CAM is its adaptive use of activation maps from all the layers of the CNN to arrive at a more focused saliency map.