FiRe: Fixed-Noise Refinement for Visual Counterfactual Explanations
FiRe refines images at a fixed noise level, achieving 3× faster inference and 8× fewer FLOPs while maintaining high-quality counterfactuals.
Key Findings
Methodology
FiRe employs a fixed-noise mapping from the input, utilizing Pixel Mean Flow (PMF) for single-step high-quality clean image prediction. It introduces dynamic dual masks, adaptive guidance, and early stopping to control local edits. Multiple refinement steps occur at the same noise level, avoiding the timestep dependency of traditional DDPM reverse trajectories. The approach significantly improves efficiency, reducing inference time to 6.37 minutes and FLOPs to 1.28P, while maintaining or surpassing state-of-the-art quality.
Key Results
- On CelebA, CelebA-HQ, and CheXpert, FiRe achieves FID scores of 0.69 and 1.98, outperforming recent baselines. It reduces inference time by 3× and FLOPs by 8×, demonstrating superior efficiency. Ablation studies confirm the importance of fixed noise level and PMF prediction. The method produces highly localized, realistic counterfactuals that preserve identity and decision consistency.
- In five tasks, FiRe maintains high decision accuracy and visual plausibility, especially excelling in facial attribute and medical image scenarios. The combination of efficiency and quality marks a significant advancement.
- The ablation results highlight that fixing the noise level and using PMF for clean image prediction are critical for the observed performance gains.
Significance
This work addresses the fundamental limitations of existing diffusion-based counterfactual explanations, notably the reliance on time-dependent reverse trajectories and costly recursive denoising. By reformulating the process at a fixed noise level, FiRe offers a scalable, efficient, and controllable framework. Its ability to produce high-quality, localized counterfactuals rapidly makes it suitable for real-world applications in healthcare, facial attribute editing, and beyond. The approach bridges the gap between theoretical diffusion models and practical interpretability tools, paving the way for broader adoption of explainable AI techniques in sensitive domains.
Technical Contribution
FiRe introduces a paradigm shift by replacing the traditional reverse denoising trajectory with fixed-noise iterative refinement. The integration of Pixel Mean Flow enables direct, single-step clean image prediction from noisy states, eliminating recursive denoising. The dynamic dual-mask strategy ensures spatial control over edits, while adaptive guidance and early stopping prevent over-editing. These innovations collectively reduce computational complexity, enhance local control, and maintain semantic fidelity, offering a novel, efficient framework for classifier-guided image editing.
Novelty
This is the first work to perform counterfactual explanation via multiple refinement steps at a single fixed noise level, bypassing the need for a time-dependent reverse process. The combination of fixed-noise refinement with PMF for direct clean image prediction and the novel spatial control mechanisms constitute a significant departure from prior diffusion-based methods, enabling faster and more precise explanations.
Limitations
- The reliance on pre-trained PMF models may limit performance in highly complex or novel scenarios.
- Fixed noise level might restrict the model's ability to capture fine details in some cases.
- In extreme cases of content deviation, semantic consistency could be compromised, requiring further robustness enhancements.
Future Work
Future directions include integrating multi-scale noise levels, exploring multi-modal guidance, and improving mask adaptivity for finer local control. Extending the framework to multi-class and multi-modal explanations, as well as enhancing robustness in diverse scenarios, are promising avenues. Additionally, reducing dependence on pre-trained models and improving generalization will be key to broader deployment.
AI Executive Summary
In recent years, the demand for transparent AI decision-making has driven research into visual counterfactual explanations. Traditional diffusion-based methods, such as DiME and ACE, leverage the reverse denoising trajectory of diffusion models to generate realistic counterfactual images. However, these approaches face significant challenges: they are computationally intensive due to recursive denoising, and controlling the spatial localization of edits is difficult across varying noise levels. Moreover, the noisy states used for editing are often mismatched with the classifier's expectation of clean images, further complicating guidance.
To address these issues, this paper introduces FiRe, a novel framework that reformulates counterfactual explanation as fixed-noise refinement. Instead of following the traditional reverse trajectory, FiRe maps the input image to a fixed noise level and iteratively refines the noisy state at this level. This approach preserves semantic editability while reducing the uncontrolled global changes caused by high noise. Central to FiRe is the integration of Pixel Mean Flow (PMF), which predicts a clean image directly from noisy states in a single step, bypassing recursive denoising and low-quality estimates.
The framework also employs three specialized controls: a dynamic dual-mask strategy to localize edits, adaptive guidance to regulate the strength of updates, and early stopping to prevent over-editing. These mechanisms ensure that counterfactual changes are minimal, localized, and semantically consistent. Extensive experiments across five tasks on CelebA, CelebA-HQ, and CheXpert datasets demonstrate that FiRe achieves approximately 3× faster inference and 8× fewer FLOPs compared to the strongest recent baseline, MaskDiME, while maintaining or surpassing state-of-the-art quality.
This work significantly advances the efficiency and controllability of diffusion-based counterfactual explanations, making them more practical for real-world applications. Its ability to produce high-fidelity, localized, and computationally efficient explanations opens new avenues in interpretability, especially in sensitive domains like healthcare and facial attribute editing. Future work aims to extend the method to multi-scale, multi-modal scenarios and further improve robustness and generalization, fostering broader adoption of explainable AI techniques.
Deep Dive
Limitations & Outlook
What gaps remain?
Abstract
Visual counterfactual explanations aim to change classifier decisions through realistic and localized edits while preserving decision-irrelevant content. Existing DDPM-based methods typically perform classifier-guided editing along a long reverse denoising trajectory. The changing noise levels make semantic editability and spatial control difficult to balance, and the editable state is noisy, whereas the target classifier is trained on clean images. As a result, these methods require either costly recursive denoising or low-quality one-step estimates to obtain classifier-facing clean images. We propose FiRe, a Fixed-noise Refinement framework for visual counterfactual explanations. Rather than following a reverse denoising trajectory, FiRe maps the input to a fixed noise level and iteratively refines the noisy state at that level. To provide clean images for classifier guidance, FiRe first adapts Pixel Mean Flow to visual counterfactual explanation, enabling direct clean-image prediction from noisy states. To make fixed-noise refinement produce minimal and localized counterfactual edits, FiRe introduces three FiRe-specific controls: a dynamic dual-mask strategy, adaptive guidance, and early stopping, which determine where edits accumulate, which changes become visible, and when refinement stops. Experiments on five tasks across three datasets show that, compared with the strongest recent baseline, FiRe achieves about 3$\times$ faster online inference and 8$\times$ fewer FLOPs while obtaining comparable or state-of-the-art counterfactual quality.