A multi-criteria approach for selecting an explanation from the set of counterfactuals produced by an ensemble of explainers

The evolution of explainable AI (XAI) has been driven by the need for transparency in complex models like neural networks and ensemble methods. Early approaches such as LIME and SHAP provided local explanations but faced limitations in handling conflicting quality criteria. Counterfactual explanations, introduced by Wachter et al., offered intuitive insights by showing minimal feature changes needed to flip predictions. Over time, researchers recognized the importance of multiple metrics—proximity, sparsity, actionability, and discriminative power—in assessing explanation quality. Recent works like Dandl et al. and Rasouli and Chieh Yu explored multi-criteria generation, but lacked systematic selection mechanisms, often overwhelming users with large solution sets. The challenge remains in balancing these metrics effectively and automating the selection process to improve usability and trust.

The core issue is the conflicting nature of explanation quality metrics, which complicates the selection of the most meaningful counterfactual. Existing methods tend to optimize single metrics or generate large, unfiltered solution sets, burdening users with decision overload. This not only hampers interpretability but also reduces trust in AI systems, especially in high-stakes domains. The difficulty lies in designing a unified framework that can generate diverse explanations, filter out suboptimal solutions, and automatically select the most balanced one without requiring extensive user input or manual tuning.

The primary innovation is the integration of multi-model ensemble generation with multi-criteria decision analysis. This includes: 1) leveraging diverse explainers (e.g., CEM, Dice, ActionableRecourse) to produce a broad set of counterfactuals; 2) applying Pareto front filtering to eliminate dominated solutions, drastically reducing candidate volume; 3) employing the Ideal Point method to automatically select the most balanced explanation based on multiple metrics. This approach addresses the limitations of single-metric optimization and user overload, providing a systematic, scalable solution that balances conflicting criteria efficiently.

• �� Generate counterfactuals: Run multiple explainers on the target instance, producing diverse solutions.
• �� Filter invalid and non-actionable explanations: Remove solutions that do not change the prediction or violate actionability constraints.
• �� Construct Pareto front: Use dominance relations to identify non-dominated solutions, filtering out those inferior across all metrics.
• �� Select final explanation: Calculate the distance of each Pareto front solution to an ideal point—defined as the best possible values across all metrics—and choose the closest one.
• �� This pipeline ensures a comprehensive yet concise set of explanations, balancing multiple conflicting criteria without user bias.

Experiments utilized datasets like Adult, German, Compas, and Fico, with predefined immutable features. Baselines included single-metric optimization methods. Metrics evaluated included Proximity, Feasibility, and Discriminative Power. The evaluation measured candidate set reduction, explanation quality, and robustness across user preferences. Hyperparameters such as the number of explainers and filtering thresholds were tuned via grid search. Ablation studies isolated the impact of Pareto filtering and the Ideal Point method, confirming their critical roles. Results consistently showed improved trade-offs, with explanations being more actionable and aligned with user expectations.

The proposed framework reduced candidate explanations by approximately 80%, while improving the overall balance of quality metrics by over 20%. The explanations generated were more actionable, with higher feasibility scores and better discriminative power. User preference simulations demonstrated the method’s flexibility in adjusting trade-offs, and ablation tests highlighted the importance of Pareto front filtering and the Ideal Point selection. The approach outperformed traditional single-criterion methods in both quantitative metrics and user satisfaction indicators, confirming its practical utility.

This methodology is directly applicable to domains requiring transparent decision-making, such as credit scoring, medical diagnosis, and hiring. It enables practitioners to generate concise, balanced explanations that are easy for users to interpret and act upon. The framework can be integrated into existing AI pipelines with minimal modifications, provided multiple explanation algorithms are available. Its ability to automate the trade-off process makes it especially valuable for high-volume, real-time systems where manual selection is impractical.

The approach depends on predefined quality metrics and their relative importance, which may not reflect all user preferences. High-dimensional data or complex models can increase computational costs, limiting scalability. The reliance on multiple explainers may restrict applicability in resource-constrained environments. Future work should explore adaptive weighting, scalability improvements, and user preference elicitation to enhance robustness and usability.