A Synthesizable RTL Implementation of Predictive Coding Networks

Modern machine learning systems are typically trained using backpropagation, which computes gradients by combining global loss information with a tightly coordinated forward/backward computation schedule. Although highly effective, this paradigm is challenging to realize as a fully distributed learning substrate in hardware. Backpropagation requires structured global error propagation, intermediate activation storage, and substantial data movement through memory and interconnect.

Predictive coding offers an alternative formulation where inference and learning arise from minimizing prediction errors across a hierarchy. In standard predictive coding networks (PCNs), each layer predicts the layer below; each unit updates its state and synaptic weights using only locally available quantities: its own activity, its own prediction error, presynaptic activity from the adjacent layer above, and prediction errors from the adjacent layer below. This locality makes predictive coding attractive as a candidate algorithmic substrate for physically embedded learning systems.

This paper presents a digital micro-architecture that directly implements predictive coding equations at the level of individual neurons. Each unit executes a fixed finite-state schedule per tick. Communication is strictly between adjacent layers via hardwired connections. No shared parameter memory and no global learning-phase controller are required. The objective of this work is not to propose a new learning rule but to demonstrate a concrete mapping from predictive-coding-style local learning to a structured, synthesizable digital substrate.

Backpropagation requires global coordination that is challenging to reconcile with distributed biological learning and certain classes of embedded hardware systems. First, standard gradient computation requires propagating error information backward through the entire network, creating a dependency structure that is not purely local. Second, training is typically organized into distinct phases (forward, backward, update) that demand synchronization and storage of intermediate activations. Third, backpropagation assumes differentiability of the computational graph, whereas biological systems involve discontinuous and stochastic signaling. While these issues do not prevent backpropagation from being implemented on conventional accelerators, they motivate research into alternative learning formulations that admit local update structures suitable for embedded online adaptation.

The core innovations of this paper include:

• �� A composable neural-core architecture implementing discrete-time predictive coding updates using a sequential MAC datapath. This design allows each neural core to maintain its activity, prediction error, and synaptic weights, communicating only with adjacent layers through hardwired connections.

• �� A uniform per-neuron clamping interface supporting supervised training and inference. This interface enforces boundary conditions while leaving the internal update schedule unchanged, supporting learning and inference through boundary conditions.

• �� A direct correspondence between predictive coding computations and hardware FSM stages, ensuring verifiable consistency between update equations and hardware datapath. This design prioritizes direct, verifiable correspondence over the energy efficiency gains provided by event-driven spiking systems.

These innovations enable predictive coding to be implemented directly in hardware, reducing reliance on global coordination and centralized storage.

The proposed digital architecture implements discrete-time predictive coding updates directly in hardware, with the following methodology:

• �� Each neural core maintains its activity, prediction error, and synaptic weights, communicating only with adjacent layers through hardwired connections. Each core corresponds to one indexed unit, maintaining its local state and parameters, and communicating only with adjacent layers via hardwired signals.

• �� The design is based on a sequential MAC datapath and a fixed finite-state schedule, rather than executing task-specific instruction sequences within the learning substrate. The system evolves under fixed local update rules, with task structure imposed through connectivity, parameters, and boundary conditions.

• �� A uniform per-neuron clamping interface supports supervised learning and inference, ensuring boundary conditions while leaving the internal update schedule unchanged. This interface enforces boundary conditions while leaving the internal update schedule unchanged, supporting learning and inference through boundary conditions.

• �� A direct correspondence between predictive coding computations and hardware FSM stages, ensuring verifiable consistency between update equations and hardware datapath. This design prioritizes direct, verifiable correspondence over the energy efficiency gains provided by event-driven spiking systems.

The experimental design includes teacher-student regression and nonlinear regression tasks, validating the architecture's stability and efficiency under a limited tick budget. In the teacher-student regression experiment, a three-layer network (2→4→3) rapidly reduced initial MSE from 0.341207 to 0.004784, demonstrating the effectiveness of the incremental tick regime. In the nonlinear regression experiment, a smaller network (2→2→1) reduced initial MSE from 0.106512 to 0.004382, indicating stability under a limited tick budget.

Additionally, architectural scaling experiments show the design's scalability, with networks of different sizes exhibiting rapid initial descent followed by stable residual floors under the same tick schedule. Experiments are implemented as Verilator simulations using the reference RTL implementation, with learning and inference controlled entirely through clamping and learning-rate parameters without changing the internal neural-core schedule.

Experimental results demonstrate the architecture's effectiveness in teacher-student regression and nonlinear regression tasks, validating its stability and efficiency under a limited tick budget. In the teacher-student regression experiment, a three-layer network (2→4→3) rapidly reduced initial MSE from 0.341207 to 0.004784, demonstrating the effectiveness of the incremental tick regime. In the nonlinear regression experiment, a smaller network (2→2→1) reduced initial MSE from 0.106512 to 0.004382, indicating stability under a limited tick budget.

Additionally, architectural scaling experiments show the design's scalability, with networks of different sizes exhibiting rapid initial descent followed by stable residual floors under the same tick schedule. Experiments are implemented as Verilator simulations using the reference RTL implementation, with learning and inference controlled entirely through clamping and learning-rate parameters without changing the internal neural-core schedule.

The proposed architecture is suitable for scenarios requiring local update structures for embedded online adaptation. By implementing predictive coding directly in hardware, it reduces reliance on global coordination and centralized storage, advancing energy efficiency and real-time learning capabilities. This architecture could have profound implications for future adaptive computing devices, particularly in scenarios requiring local update structures for embedded online adaptation.

Additionally, the architecture's scalability makes it suitable for networks of different sizes, capable of exhibiting rapid initial descent followed by stable residual floors under the same tick schedule. This design could play a significant role in future adaptive computing devices, particularly in scenarios requiring local update structures for embedded online adaptation.

The current design has limitations, including increased tick latency as fan-in grows and the need for careful numerical design of nonlinear activations and their derivatives. Convergence and stability guarantees for the discrete-time, finite-precision system need empirical mapping, and theoretical analysis remains to be explored.

Future work will explore the balance between parallelism and area/power to enhance scalability and conduct task-driven benchmarks to identify scenarios where local online inference is advantageous. Additionally, research into activation approximations suitable for synthesis is needed to ensure precision and stability.