We wish to construct a system which possesses so-called associative memory. This is definable generally as a process by which an input, considered as a “key”, to a memory system is able to evoke, in a highly selective fashion, a specific response associated with that key, at the system output. The signalresponse association should be “robust”, that is, a “noisy” or “incomplete” input signal should none the less invoke the correct response—or at least an acceptable response. Such a system is also called a content addressable memory | NEURAL NETWORKS Ivan F Wilde Mathematics Department King s College London London WC2R 2LS UK Contents 1 Matrix Memory . 1 2 Adaptive Linear Combiner . 21 3 Artificial Neural Networks . 35 4 The Perceptron . 45 5 Multilayer Feedforward Networks. 75 6 Radial Basis Functions . 95 7 Recurrent Neural Networks .103 8 Singular Value Decomposition .115 Bibliography .121 Chapter 1 Matrix Memory We wish to construct a system which possesses so-called associative memory. This is definable generally as a process by which an input considered as a key to a memory system is able to evoke in a highly selective fashion a specific response associated with that key at the system output. The signalresponse association should be robust that is a noisy or incomplete input signal should none the less invoke the correct response or at least an acceptable response. Such a system is also called a content addressable memory. Figure A content addressable memory. The idea is that the association should not be defined so much between the individual stimulus-response pairs but rather embodied as a whole collection of such input-output patterns the system is a distributive associative memory the input-output pairs are distributed throughout the system memory rather than the particular input-output pairs being somehow represented individually in various different parts of the system . To attempt to realize such a system we shall suppose that the input key or prototype patterns are coded as vectors in R say and that the responses are coded as vectors in Rm. For example the input might be a digitized photograph comprising a picture with 100 X 100 pixels each of which may assume one of eight levels of greyness from white 0 to black
