Wardz….

Next, we apply a neural network model using Insightful Miner on the same adult data set [3] from the UCal Irvine Machine Learning Repository that we analyzed in Chapter 6. The Insightful Miner neural network software was applied to a training set of 24,986 cases, using a single hidden layer with eight hidden nodes. The [...]

One of the drawbacks of neural networks is their opacity. The same wonderful flexibility that allows neural networks to model a wide range of nonlinear behavior also limits our ability to interpret the results using easily formulated rules. Unlike decision trees, no straightforward procedure exists for translating the weights of a neural network into a [...]

Clearly, a momentum component will help to dampen the oscillations around optimality mentioned earlier, by encouraging the adjustments to stay in the same direction. But momentum also helps the algorithm in the early stages of the algorithm, by increasing the rate at which the weights approach the neighborhood of optimality. This is because these early [...]

When the learning rate is very small, the weight adjustments tend to be very small. Thus, if  is small when the algorithm is initialized, the network will probably take an unacceptably long time to converge. Is the solution therefore to use large values for ? Not necessarily. Suppose that the algorithm is close to [...]

The neural network algorithm would then proceed to work through the training data set, record by record, adjusting the weights constantly to reduce the prediction error. It may take many passes through the data set before the algorithms termination criterion is met. What, then, serves as the termination criterion, or stopping criterion? If training time [...]

We must therefore turn to optimization methods, specifically gradient-descent methods, to help us find the set of weights that will minimize SSE. Suppose that we have a set (vector) of m weights w = w0,w1,w2, . . . , wm in our neural network model and we wish to find the values for each of [...]

Whyuse the sigmoid function? Because it combines nearly linear behavior, curvilinear behavior, and nearly constant behavior, depending on the value of the input. Figure 7.3 shows the graph of the sigmoid function y = f (x) = 1/(1 + e?x ), for ?5 < x < 5 [although f (x) may theoretically take any real-valued [...]

Let us examine the simple neural network shown in Figure 7.2. A neural network consists of a layered, feedforward, completely connected network of artificial neurons, or nodes. The feedforward nature of the network restricts the network to a single direction of flow and does not allow looping or cycling. The neural network is composed of [...]

Not all classification problems, however, are soluble using a single output node only. For instance, suppose that we have several unordered categories in our target variable, as, for example, with the marital status variable above. In this case we would choose to adopt 1-of-n output encoding, where one output node is used for each possible [...]

Bewary of recoding unordered categorical variables into a single variable with a range between zero and 1. For example, suppose that the data set contains information on a marital status attribute. Suppose that we code the attribute values divorced, married, separated, single, widowed, and unknown, as 0.0, 0.2, 0.4, 0.6, 0.8, and 1.0, respectively. Then [...]