The Neural Division of Labor: Biologically-Inspired Modular Architectures for Robust Neuromorphic Computing

Deep neural networks have revolutionized AI, yet their large parameter counts, high energy consumption, and opaque decision processes hinder deployment in resource-limited settings. Neuromorphic computing, especially spiking neural networks (SNNs), offers a promising alternative by mimicking biological efficiency. Prior works include ANN-to-SNN conversion, surrogate gradient training, and event-driven architectures like STBP and LISNN, but these often involve complex training, high computational costs, and limited interpretability. Biological systems, such as insect olfactory circuits, exhibit modular, low-redundancy pathways that enable robustness and explainability. Emulating this, recent research explores structured neural architectures, but integrating such principles into scalable, trainable SNNs remains a challenge.

The core challenge is designing neural models that combine high accuracy, low energy consumption, interpretability, and robustness against catastrophic forgetting. Traditional deep networks rely on dense, entangled connections, making decision paths opaque and susceptible to interference during sequential learning. Existing SNN training methods are computationally intensive and lack structural modularity. Achieving pathway isolation without sacrificing performance is difficult, especially in a biologically plausible manner that can be efficiently implemented in hardware. Addressing these issues requires innovative architectures that incorporate biological principles of compartmentalization and activity localization.

The paper introduces a structurally modular SNN with multiple independent experts, each dedicated to a class, inspired by biological sensory pathways. The key innovation is the Push-Pull loss function, which enforces activity localization by penalizing neural activation in non-target pathways while promoting activity in the target pathway. This mimics biological lateral inhibition and synaptic pruning, leading to sparse, interpretable pathways. The training is performed in an ANN environment with a differentiable surrogate, then directly transferred to a SNN, avoiding complex temporal backpropagation. Additionally, physically severing expert connections during inference enhances robustness and interpretability, providing a new paradigm for scalable, efficient neuromorphic systems.

• �� Extract spatial features with convolutional layers. • Partition hidden layers into K disjoint experts, each linked to a class. • Implement the Push-Pull loss to maximize activity in correct expert and suppress others. • Train in ANN with surrogate gradients, optimizing activity localization. • Transfer trained weights to SNN, maintaining accuracy without temporal backpropagation. • Physically sever expert pathways to enforce isolation, preventing interference. • Validate on MNIST, Fashion-MNIST, and CIFAR-10, comparing with dense and hybrid models. • Measure accuracy, parameter count, firing rates, and robustness to sequential learning.

The models were tested on MNIST, Fashion-MNIST, and CIFAR-10 datasets, benchmarking against dense, hybrid, and other SNN architectures. Metrics included classification accuracy, number of parameters, firing rates, and synaptic operations. The training involved initial ANN optimization with the Push-Pull loss, followed by direct weight transfer to SNNs. Ablation studies demonstrated the impact of pathway isolation and loss design. Results showed that the independent expert architecture achieved comparable accuracy to dense models with significantly fewer parameters (e.g., 98.15% on MNIST with half the parameters). The models also exhibited lower firing rates and resilience to sequential learning, confirming the effectiveness of the modular design.

The proposed architecture achieved 98.15% accuracy on MNIST, 90% on Fashion-MNIST, and 66% on CIFAR-10, with parameter reductions of 50-90%. Firing rates were orders of magnitude lower, indicating high energy efficiency. The expert pathways prevented interference during sequential tasks, demonstrating robustness. The ANN-to-SNN transfer preserved accuracy, with training times reduced by an order of magnitude compared to full temporal backpropagation methods. Ablation studies confirmed the importance of activity localization enforced by the Push-Pull loss. The architecture scaled well across datasets, maintaining high efficiency and interpretability.

This modular, energy-efficient architecture is ideal for edge devices, autonomous robots, and neuromorphic hardware where power and interpretability are critical. Its ability to prevent catastrophic forgetting makes it suitable for lifelong learning systems. The transparent decision pathways facilitate safety and compliance in critical applications like medical diagnostics and security. Future integration with hardware accelerators could enable real-time, low-power AI in resource-constrained environments, transforming fields from IoT to embedded systems.

While promising, the approach's scalability to larger, more complex datasets remains to be validated. Pathway isolation may limit the model's capacity to learn highly entangled features. Physical implementation of pathway severing in hardware poses engineering challenges. Further research is needed to extend the framework to multi-task and multi-modal scenarios, and to optimize hardware integration for real-world deployment.