Artificial Neural Networks

Abstract

An artificial neural network (ANN) is a functional abstraction of the biologic neural structures of the central nervous system (Aleksander and Morton 1993; Rudomin et al. 1993; Arbib 1995; Anderson 1995). It is essentially a simple mathematical model representing a nonlinear mapping function from a set of inputs x to outputs y, i.e., / x -> y. The most commonly used ANN model is the feedforward neural network. This network usually consists of input, hidden, and output layers. The information moves only in the forward direction, from the input nodes, through the hidden nodes and to the output nodes. There are no loops in the network. As an example, Fig. 2.1 shows a widely used three-layer feedforward ANN model, with arrows depicting the dependencies between variables at each layer. Its single output у (i.e., model prediction) is the nonlinear weighted sum of the inputs x. In Fig. 2.1, the input vector x consists of p variables x, (/ = 1 ,..., p). The parameter ay (/ = 1 ,..., p; j = 1, ...,K ) represents the weight of the link connecting the input node і to node j in the hidden layer, in which K is the number of nodes in the hidden layer. The parameter wj represents the weight of the link connecting the hidden node j to the node in the output layer, and the variable d represents the weight of the bias. The bias term (a constant value, typically one) allows the neural network to return a nonzero value at the origin. The parameters αβ, Wj, and d need to be estimated using a learning algorithm which will be discussed later.