Benchmarking Fairness in Spiking Neural Networks: Data Bias, Spurious Features, and Hardware Effects
Introduced the first systematic fairness benchmark for SNNs, revealing impacts of bias and hardware constraints on model performance.
Key Findings
Methodology
This study proposes a systematic fairness benchmark framework specifically for Spiking Neural Networks (SNNs). The framework integrates four cross-demographic datasets and three neuromorphic hardware simulators (e.g., Loihi 2, SpiNNaker) to analyze fairness-performance trade-offs under resource constraints. Using metrics like Statistical Parity and Equal Opportunity, it evaluates 12 state-of-the-art SNN models.
Key Results
- Models trained on biased data exhibit 23% higher false positive rates for underrepresented groups.
- Hardware limitations (e.g., reduced spike precision) amplify accuracy gaps by up to 41% in edge deployments.
- Bias mitigation strategies for cloud-based SNNs often degrade under resource constraints.
Significance
This study provides the first comprehensive fairness evaluation benchmark for SNNs, bridging the gap between algorithmic fairness research and neuromorphic engineering. It lays the foundation for trustworthy SNNs in socially critical applications by revealing the impacts of data bias, spurious features, and hardware constraints on SNN performance.
Technical Contribution
The study reveals performance disparities in SNNs under data bias and hardware constraints, proposing co-design principles to optimize fairness and hardware efficiency under resource constraints.
Novelty
This is the first fairness benchmark specifically for SNNs, systematically analyzing data bias, spurious features, and hardware effects, filling gaps in existing toolkits regarding spike-driven dynamics and hardware limitations.
Limitations
- The study focuses mainly on vision tasks; other domains like medical monitoring may exhibit different fairness failure patterns.
- Main evaluation metrics focus on single sensitive attributes, not fully capturing complex interactions among multiple attributes.
Future Work
Future research could extend to other modalities like medical monitoring and autonomous driving, develop more fine-grained intersectional fairness metrics, and design effective methods to improve SNN fairness.
AI Executive Summary
Spiking Neural Networks (SNNs) show great potential for low-power computing in edge devices and IoT applications. However, the rapid progress of SNNs has hidden an important socio-technical risk: algorithmic fairness. Existing evaluations often rely on static and idealized datasets, failing to reflect neuromorphic-specific complexities.
This study introduces the first systematic fairness benchmark for SNNs, addressing real-world issues like demographic coverage gaps in training data, spurious feature leakage, and deployment-environment mismatches. By integrating four cross-demographic datasets and three neuromorphic hardware simulators, the study reveals fairness-performance trade-offs under resource constraints.
Experimental results show models trained on biased data exhibit 23% higher false positive rates for underrepresented groups, and hardware limitations amplify accuracy gaps by up to 41%. The study emphasizes the importance of co-design principles to jointly optimize fairness and hardware efficiency under resource constraints, laying the foundation for trustworthy SNNs in socially critical applications.
Deep Dive
Abstract
Evaluating fairness in Spiking Neural Networks (SNNs) demands rigorous benchmarks that reflect real-world complexities, yet existing assessments remain limited by superficial dataset diversity and idealized hardware assumptions. This work introduces the first systematic fairness benchmark for SNNs, addressing three critical dimensions of realism: (1) demographic coverage gaps in training data, (2) spurious feature leakage (e.g., skin tone as a proxy for class labels), and (3) deployment-environment mismatches (e.g., edge devices with constrained spike encoding). Our framework integrates four cross-demographic datasets with controlled bias injections and three neuromorphic hardware simulators (Loihi 2, SpiNNaker), enabling isolated analysis of fairness-performance trade-offs under resource constraints. Standardized evaluations of 12 state-of-the-art SNNs reveal stark disparities: models trained on biased data exhibit 23\% higher false positive rates for underrepresented groups, while hardware limitations (e.g., reduced spike precision) further amplify accuracy gaps by up to 41\% in edge deployments. Critically, bias mitigation strategies developed for cloud-based SNNs often degrade under resource constraints, highlighting the need for co-design principles that jointly optimize fairness and hardware efficiency. By bridging algorithmic fairness research with neuromorphic engineering, our benchmark provides a foundation for trustworthy SNNs in socially critical applications such as healthcare and autonomous systems. Our code is available at: https://anonymous.4open.science/r/SNN-Benchmarks-8017.