Structure as Computation: Developmental Generation of Minimal Neural Circuits

Research in neural networks has made significant progress over the past decades, particularly with the advent of deep learning. Traditional deep neural networks rely on end-to-end gradient optimization methods to simultaneously learn both network structure and weights, achieving success in many tasks. However, this approach fundamentally differs from the developmental processes of biological neural networks, where the brain's initial wiring is established through genetically encoded developmental programs long before sensory experience begins. Recently, researchers have begun to explore how inspiration from biological systems can improve the design and optimization strategies of artificial neural networks.

Traditional neural network training methods rely on large amounts of data and computational resources to simultaneously learn network structure and weights. While successful in many tasks, these methods have limitations, such as long training times, strong dependency on data, and sensitivity to network structure. Additionally, these methods fundamentally differ from the developmental processes of biological neural networks, failing to fully leverage the structural priors inherent in biological systems. Designing a new neural network generation method that can quickly learn and perform well without relying on large amounts of data and computational resources is a pressing issue.

This study proposes a neural network generation method based on biological developmental processes, simulating mouse cortical neurogenesis to generate network topology from a single stem cell. The innovations include:

1) Generating network topology through gene regulatory rules rather than simultaneously learning structure and weights;

2) The generated minimal neural circuit demonstrates rapid learning capabilities across different visual domains, showcasing the generality of structural priors;

3) By simulating biological developmental processes, the study reveals the powerful structural priors inherent in biological systems, offering a new approach to neural network initialization.

The methodology of this study includes the following steps:

• �� Data Source: Mouse single-cell transcriptomic data is used to derive Boolean regulatory rules for 15 key neurodevelopmental genes.
• �� Boolean Rule Inference: Based on the gene expression matrix, Boolean regulatory rules for each target gene are inferred, ensuring temporal causality and agreement maximization.
• �� Simulated Developmental Process: Starting from a single stem cell, the processes of cell division, migration, differentiation, and synaptogenesis are simulated, ultimately forming a densely interconnected core of 85 mature neurons.
• �� Network Integration and Training: The generated topology is converted into a fixed-weight recurrent layer, and training is conducted on MNIST and CIFAR-10 datasets.

The experimental design includes validation on MNIST and CIFAR-10 datasets. The MNIST dataset is used to test the rapid learning capability of the generated network, while the CIFAR-10 dataset is used to verify the generality of the structural prior. In the experiments, a fixed network topology is used without any architectural modification or data augmentation. Training uses cross-entropy loss and the Adam optimizer with a learning rate of 10^-3 and a batch size of 64. The MNIST dataset is trained for 10 epochs, and the CIFAR-10 dataset is trained for 100 epochs.

The experimental results show that the generated minimal neural circuit demonstrates rapid learning capability on the MNIST dataset. At iteration zero, the network performs at chance level, but after one standard training epoch, accuracy quickly exceeds 90%. On the CIFAR-10 dataset, using the same network structure without any architectural modification, accuracy reaches 40.53% after one epoch. These results suggest that developmental rules can sculpt a topological substrate that exhibits rapid learning capabilities across different visual domains.

This study's method can be directly applied to neural network initialization, especially in scenarios with limited data and computational resources. By generating network topology with structural priors, it is possible to quickly learn and achieve good performance without relying on large amounts of data and computational resources. Additionally, this method can be applied to neural network design in other fields, providing new perspectives and strategies.

Despite demonstrating the potential of the generative developmental framework, the study has limitations. The current validation is limited to MNIST and CIFAR-10 datasets, and its generality and performance on more complex datasets remain untested. Additionally, the small network size may pose performance bottlenecks when handling larger-scale and more complex tasks. Future research will extend to more complex datasets, explore activity-dependent plasticity during development, and scale up the number of neurons to enhance network processing capabilities.