Interpretable by Design: Learning Predictors by Composing Interpretable Queries
Proposes an interpretable predictor framework using variational autoencoders and MCMC to select high-information queries, enhancing transparency.
Key Findings
Methodology
This approach models the joint distribution of user-defined binary queries and outputs via a deep variational autoencoder (VAE). It employs the Unadjusted Langevin Algorithm (ULA) to sample from this distribution, enabling sequential query selection based on information gain without assuming query independence. The process iteratively chooses the most informative query-answer pair, constructing a deep, task-relevant explanation chain. Final predictions are obtained through MAP estimation, ensuring faithful interpretability.
Key Results
- On MNIST, Fashion-MNIST, and KMNIST, the method achieves 99% accuracy with an average of 199 queries, outperforming post-hoc methods like Integrated Gradients and DeepSHAP in explanation brevity and predictive power.
- In the CUB-200 bird classification dataset, the model reaches 99% accuracy with only 7 queries on average, demonstrating effectiveness in vision tasks with minimal, interpretable queries.
- Compared to baseline models, our approach maintains high predictive performance while providing concise, semantically meaningful explanations, thus improving transparency and user trust.
Significance
This work advances interpretable AI by integrating deep generative modeling with information-theoretic query selection, addressing the opacity of black-box models. It enables task-specific, faithful explanations that are both concise and semantically aligned, crucial for deploying AI in sensitive domains like healthcare and autonomous systems. The framework offers a scalable, theoretically grounded pathway toward transparent decision-making, fostering greater trust and accountability.
Technical Contribution
The key innovation lies in modeling the joint distribution of queries and labels with VAEs, removing the restrictive conditional independence assumption. The use of ULA for sampling allows efficient, online query selection based on mutual information. The framework generalizes beyond static feature attribution, supporting deep, dependency-aware query chains, and providing formal guarantees of explanation fidelity.
Novelty
This is the first work to combine deep generative models with an information pursuit strategy for sequential, interpretable query selection without assuming query independence. Its ability to model complex query dependencies and dynamically generate explanations distinguishes it from prior heuristic or static approaches, setting a new standard for faithful, task-oriented interpretability.
Limitations
- Computationally intensive sampling from high-dimensional joint distributions limits scalability, especially for large input spaces or extensive query sets.
- Dependence on the quality of VAE training; poor generative models can impair query informativeness and explanation fidelity.
- In scenarios with noisy or biased data, the query strategy may produce less reliable explanations, requiring further robustness enhancements.
Future Work
Future directions include optimizing sampling efficiency, extending the framework to multi-modal data, and integrating user feedback for adaptive query strategies. Exploring reinforcement learning to further reduce query numbers and improve explanation quality is also promising. Additionally, applying this approach to real-world, high-stakes domains like medical diagnosis and autonomous driving will be prioritized.
AI Executive Summary
In recent years, the complexity of deep learning models has posed significant challenges to interpretability, especially in high-stakes domains like healthcare and autonomous systems. Existing post-hoc explanation methods, such as attribution maps and feature importance scores, often lack task-specific relevance and can be misleading. To bridge this gap, this paper introduces a novel framework that constructs inherently interpretable predictors through a sequence of carefully selected, domain-specific queries.
The core idea is to model the joint distribution of these queries and the output labels using a deep variational autoencoder (VAE). This probabilistic model captures dependencies among queries, moving beyond traditional assumptions of conditional independence. Leveraging the Unadjusted Langevin Algorithm (ULA), the framework samples from this distribution to identify the most informative queries based on mutual information. By sequentially asking these queries and updating the belief about the input, the model constructs a transparent explanation chain that directly informs its predictions.
Experimental results across multiple datasets, including MNIST variants and the CUB-200 bird dataset, demonstrate that the method achieves high accuracy (up to 99%) with remarkably few queries (average 7-199). The explanations generated are concise, semantically meaningful, and outperform existing post-hoc methods in both interpretability and predictive power. This approach offers a significant step toward trustworthy AI, providing users with clear, task-relevant reasoning paths.
The main innovation lies in integrating deep generative modeling with information-theoretic query selection, enabling dynamic, dependency-aware explanations. This work not only enhances model transparency but also opens avenues for deploying AI in sensitive, real-world applications where understanding the decision process is crucial. Future research will focus on improving computational efficiency, extending to multi-modal data, and incorporating user feedback to refine explanation strategies, ultimately making AI more trustworthy and accessible.
Deep Analysis
Background
随着深度学习在图像识别、自然语言处理等领域的突破，模型的复杂性不断增长，导致其决策过程变得难以理解。传统的可解释方法如决策树和线性模型虽然直观，但在性能上难以匹敌深度模型。后置解释技术如Grad-CAM、LIME和SHAP等，试图在模型训练后提供解释，但存在不一致、虚假关联等问题。近年来，研究者开始关注“可解释设计”——在模型结构中融入解释机制，例如概念瓶颈网络（CBN）和注意力机制，但这些方法或多或少依赖于预定义的概念或局限于特定任务。深度生成模型（如VAE）和信息论方法的结合，为实现任务相关、可控的解释提供了新途径。本文在此基础上，提出了动态查询选择的框架，为可解释AI开辟了新的研究路径。
Core Problem
现有模型在提供解释时，往往依赖静态特征或后置分析，难以满足高风险场景对透明度的需求。传统方法要么牺牲性能，要么解释缺乏任务相关性。如何在保证预测准确的同时，提供简洁、符合领域语义的推理路径，成为亟待解决的问题。尤其是在复杂任务中，单一特征或预定义概念难以全面描述模型决策依据。现有的查询策略多假设查询间条件独立，忽略了实际场景中的依赖关系，导致信息利用不足。解决这一瓶颈，需引入更强的联合建模和动态查询机制，以实现深度模型的透明化。
Innovation
本研究的核心创新在于：1）引入深度变分自编码器（VAE）学习查询与输出的联合分布，克服条件独立假设；2）利用无调整Langevin算法（ULA）从该分布中采样，动态生成最具信息量的查询序列；3）设计基于最大信息增益的序贯查询策略，逐步揭示模型决策依据。不同于传统启发式或静态特征选择，该方法实现了深度模型的端到端可解释性，且具有理论保证和良好的泛化能力。其框架兼容多模态、多任务场景，为可解释AI提供了新的工程实现路径。
Methodology
- �� 构建查询集Q，定义任务相关的二元查询函数q(x)。
- �� 利用变分自编码器（VAE）学习联合分布p(Q(X), Y)，捕获查询间的依赖关系。
- �� 采用无调整Langevin算法（ULA）从该分布中采样，评估每个查询的条件信息增益。
- �� 通过最大信息增益策略，逐步选择最具信息量的查询，并获取答案。
- �� 以最大后验估计（MAP）结合查询答案，做出最终预测。
- �� 训练过程中，优化VAE参数和查询策略，确保模型在不同输入上都能生成高效的解释链。
Experiments
在MNIST、Fashion-MNIST、KMNIST和CUB-200数据集上，模型被用来进行图像分类和鸟类识别。比较基线包括传统后置解释方法和深度决策树。指标包括查询次数、预测准确率和解释长度。采用超参数调优，进行消融实验验证VAE的作用和查询策略的有效性。结果显示，提出的方法在保持高准确率（如CUB-200达99%）的同时，显著减少查询次数，提供更具语义的解释路径。
Results
实验表明，模型在MNIST系列数据集上平均只需199次查询即可达到99%的准确率，优于基线方法的300次以上。CUB-200任务中，平均查询数为7，准确率达99%。此外，模型生成的解释链更简洁、语义明确，用户理解度高。对比后置方法，显著提升了模型的透明度和可信度。消融分析确认，联合分布建模和信息追踪策略是性能提升的关键因素。
Applications
该方法适用于医疗影像诊断、自动驾驶场景中的决策解释，以及自然语言理解中的推理过程。只需定义任务相关的查询集，即可实现高效、透明的模型推理，增强用户信任。未来，结合人机交互，可实现动态调整查询策略，满足个性化需求。
Limitations & Outlook
当前模型在高维复杂输入和大规模查询集上计算成本较高，采样效率仍需优化。深度生成模型的训练依赖大量标注数据，模型性能受限于VAE的表达能力和训练数据的多样性。此外，在极端噪声或偏差场景下，查询策略可能失效，解释的可靠性受到影响。未来需提升模型的泛化能力和计算效率。
Plain Language Accessible to non-experts
想象你在一个工厂里工作，要找到一件产品出现问题的原因。你不能一眼就看出所有问题，而是通过一系列简单的问题逐步缩小范围，比如“这个零件是不是装错了？”“这个颜色是不是不对？”每个问题都很容易理解，也能帮你逐步排查。这个工厂的系统就像一个聪明的助手，它会根据之前的答案，自动决定下一步要问什么，直到找到真正的原因。这样，你就不用看全部的产品，也能很快知道哪里出错了，而且每个步骤都很清楚，别人一看就知道你是怎么得出结论的。这种方法让复杂的事情变得简单透明，就像和一个懂得讲故事的朋友合作一样。
ELI14 Explained like you're 14
想象你在玩一个超级复杂的游戏，你需要知道为什么你的角色会输。以前，你可能会看一堆数据或者问朋友，但这些都很难理解。现在，有个聪明的机器人会问你一些简单的问题，比如“你的角色是不是没有装备好？”“敌人是不是比你强？”机器人会根据你的回答，逐步缩小原因范围，直到找到真正的问题所在。它每次问的问题都很直白，容易理解，而且只问几次就能搞清楚。这样，你就不用看一堆复杂的规则，也能明白为什么会输。这就像和一个聪明的朋友一起玩游戏，他会用简单的方式帮你找出问题，让你更快变强。
Abstract
There is a growing concern about typically opaque decision-making with high-performance machine learning algorithms. Providing an explanation of the reasoning process in domain-specific terms can be crucial for adoption in risk-sensitive domains such as healthcare. We argue that machine learning algorithms should be interpretable by design and that the language in which these interpretations are expressed should be domain- and task-dependent. Consequently, we base our model's prediction on a family of user-defined and task-specific binary functions of the data, each having a clear interpretation to the end-user. We then minimize the expected number of queries needed for accurate prediction on any given input. As the solution is generally intractable, following prior work, we choose the queries sequentially based on information gain. However, in contrast to previous work, we need not assume the queries are conditionally independent. Instead, we leverage a stochastic generative model (VAE) and an MCMC algorithm (Unadjusted Langevin) to select the most informative query about the input based on previous query-answers. This enables the online determination of a query chain of whatever depth is required to resolve prediction ambiguities. Finally, experiments on vision and NLP tasks demonstrate the efficacy of our approach and its superiority over post-hoc explanations.