BIRDNet: Mining and Encoding Boolean Implication Knowledge Graphs as Interpretable Deep Neural Networks

The application of deep learning in biomedical sciences has revolutionized data-driven discovery, enabling high-accuracy predictions in genomics, transcriptomics, and proteomics. However, these models often function as black boxes, limiting their utility for scientific interpretation and clinical trust. Early efforts incorporated external ontologies, such as Gene Ontology or Reactome pathways, to embed biological knowledge into neural architectures, but these approaches depend heavily on curated databases and lack flexibility. Recent advances in neurosymbolic AI aim to combine symbolic reasoning with neural networks, but many rely on predefined rule bases or complex logical modules, which are not automatically learned from data. Sahoo et al. introduced Boolean implication networks as a way to discover latent logical relationships among genes, providing a promising direction for data-driven symbolic modeling. Nonetheless, integrating these relationships into scalable, interpretable neural architectures remains an open challenge. This paper builds upon these foundations, proposing a novel method to encode mined implication graphs directly into neural network structures, thereby achieving a balance of interpretability, sparsity, and predictive performance.

Despite the success of deep models in biomedical tasks, their lack of transparency hinders scientific validation and clinical adoption. Existing rule-based models depend on external knowledge bases, which may be incomplete or outdated, and often lack flexibility to adapt to new data. Conversely, purely data-driven neural networks are opaque, making it difficult to interpret their decisions or extract meaningful biological insights. The core problem addressed here is how to construct a neural network that is both highly sparse and inherently interpretable, by embedding logical relationships directly into its architecture, without relying on external rule sets. Achieving this requires developing robust methods to mine latent feature relationships, encode them as network connections, and ensure that the resulting model maintains high predictive accuracy while providing transparent rule explanations.

The key innovations include: 1) a statistical method based on sparse-exception binomial testing to automatically mine Boolean implication relationships from data, capturing latent logical dependencies among features; 2) a layer-wise greedy construction algorithm that encodes these implications as sparse network connections, with each hidden unit representing a specific propositional rule involving only two features; 3) a fixed binary mask that enforces the structure during training, ensuring sparsity and interpretability without post-hoc pruning; 4) direct rule readout from trained units, enabling transparent explanations aligned with biological knowledge; 5) theoretical bounds on the active weight fraction, guaranteeing model compression and efficiency. These innovations collectively enable the creation of a neural network that is both sparse and symbolic, bridging the gap between black-box models and rule-based systems.

• �� Data preprocessing: Apply StepMiner thresholding to binarize continuous features, resulting in a binary feature matrix. • Implication mining: For each feature pair, test four types of implications (e.g., high→high, low→low) using a sparse-exception binomial test, which counts exception samples violating the implication. Significance is determined by p-value<10^-10 and exception fraction<0.0516. • Knowledge graph construction: Valid implications form a typed directed graph, with edges labeled by implication type. • Network encoding: Each implication corresponds to a hidden unit with weights initialized to connect only the two involved features, enforced by a binary mask. • Layer-wise construction: Starting from input, mine implications among features; then, for each subsequent layer, mine implications on the previous layer’s outputs, adding units accordingly. • Training: Fix the network structure, optimize weights via AdamW, with BatchNorm, ReLU, and Dropout. • Rule reading: Post-training, extract propositional rules directly from units based on their fixed symbolic identities, and evaluate rule precision, recall, and lift on validation data. • Explanation: Use hierarchical relevance propagation to trace decision paths back to specific rules and features.

• �� Datasets: Six biomedical datasets—UCI mice protein, TCGA RPPA, GSE39582 transcriptomics, UCI gene expression, METABRIC, TCGA RNA-seq—covering sample sizes from 566 to 10,051, feature counts from 77 to 54,675, and class numbers from 5 to 27. • Baselines: Dense MLP with identical architecture, L1-penalized linear models, and Random Forests. • Hyperparameters: p-value threshold 10^-10, maximum of 5000 implications per layer, maximum depth 2. • Cross-validation: 5-fold stratified, with 15% held-out for early stopping. Features standardized within training folds. • Metrics: AUROC for predictive performance, active parameter count, rule precision, and lift for interpretability. • Rule evaluation: Extract top rules per class, validate on held-out data, and analyze biological relevance through gene and pathway annotations.

• �� Predictive accuracy: BIRDNet’s AUROC scores are within 0.02 of the dense baseline across all six datasets, e.g., 0.998 AUROC on TCGA RPPA, demonstrating near-equivalent performance with vastly fewer parameters. • Parameter efficiency: The number of active weights is up to 96 times less than the dense MLP, achieving high compression ratios while maintaining accuracy. • Biological relevance: First-layer rules recover known signatures such as ERBB2 amplification in breast cancer, co-expression modules in tissue-specific datasets, and immune infiltration markers. These rules align with established biological knowledge, validating the interpretability. • Model sparsity: Theoretical bounds guarantee that each layer’s active weights do not exceed 2/d of total weights, ensuring extreme sparsity. • Rule interpretability: Rules can be directly read from the trained network, providing transparent decision pathways that match biological intuition and facilitate validation.

• �� Diagnostic tools: BIRDNet can be employed for cancer subtype classification, biomarker discovery, and disease prognosis, especially when high-dimensional omics data are available. • Scientific discovery: The interpretable rules facilitate hypothesis generation and validation, enabling biologists to verify potential causal relationships. • Clinical decision support: Transparent models can assist clinicians in understanding model predictions, increasing trust and adoption. • Future integration: Combining BIRDNet with multi-omics and temporal data could enable dynamic disease modeling, personalized treatment planning, and real-time monitoring. • Knowledge base construction: The mined rules can serve as a foundation for building automated, evolving biological knowledge repositories.

• �� The current framework supports only pairwise (2-arity) implications, limiting its ability to model complex multi-factor interactions common in biological systems. Extending to higher-arity rules is necessary. • The reliance on data-driven mining without prior domain knowledge may lead to missing biologically relevant relationships, especially in small or noisy datasets. Incorporating expert knowledge could improve robustness. • The computational cost of mining implications increases with feature dimensionality, posing scalability challenges for ultra-high-dimensional data. Algorithmic optimization and parallelization are needed. • The model does not explicitly handle temporal or spatial relationships, restricting its application to static datasets. Future work should integrate temporal modeling. • While highly interpretable, the model may overlook subtle, non-linear interactions that require more complex logical structures, suggesting a need for richer rule representations.