Simulation of a Neural Node Co-processor

PhD Thesis, 1995

Prasad Vindlacheruvu

Abstract

The design of a co-processor module to speed-up the processing of neural network (NN) algorithms is proposed and simulated. On close examination it is revealed that NN algorithms share common parallel properties that can be processed using pipelining techniques. Methods are devised to enable these properties to be mapped directly onto a multiple pipelined architecture embedded within an array of co-processor modules attached to a Von Neumann multi-processor system. Each co-processor module is designed as an independent unit containing memory elements to store weight and activation values and a solution engine for processing in the forward pass. Expansion facilities are provided in both the horizontal and vertical dimensions. The overall effect is a massively parallel structure allowing large neural networks to be decomposed, across and down an array of modules, into small sub-networks which are processed rapidly.

The solution engine is designed to offer complete flexibility. Networks containing arbitrary connectivity patterns and output functions can be processed. The co-processor module would be interfaced to a general purpose processor which has the role of applying the learning algorithm. This arrangement allows any learning algorithm to be implemented thus maintaining flexibility.

Simulations of the design based on a 64 node prototype parallel array indicates supercomputer performance at a fraction of the cost.