Dr.

Qadri Hamarsheh

Neural Network Definitions and Concepts

Outline
Introduction.
 Definition of neural network.
 A brief history of neural networks.
 Neural Network Applications.
 Biological neural networks.
Introduction
Definition of neural network
 Artificial Neural Networks (ANN) are computers whose architecture is a
simplified model of the brain, they typically consist of many hundreds of
simple processing units which are wired together in a complex
communication network.
 Each unit or node is a simplified model of a real neuron which fires (sends
off a new signal) if it receives a sufficiently strong input signal from the
other nodes to which it is connected.
 Study of artificial neural networks is motivated by their similarity to
successfully working biological systems.

Comparison between the brain and the computer

Brain Computer
Working continuously. Passive data storage.
Parallel. Serial.
Can reorganize itself and therefore Static.
is able to learn (capability to learn)
Generalize and associate data
Fault tolerance against noisy input Fault non-tolerance against noisy
data. input data.

1
 Dr. Qadri Hamarsheh

 So let us summarize the main characteristics we try to adapt from biology:

Self-organization and learning capability.
Generalization capability.
Fault tolerance.

A brief history of neural networks

The beginning
 As soon as 1943: Warren McCulloch and Walter Pitts introduced
models of neurological networks, calculate nearly any logic or arithmetic
function (electronic brains).
 1947: Walter Pitts and Warren McCulloch indicated a practical field of
application, namely the recognition of spacial patterns by neural networks.
 1949: Donald O. Hebb formulated the classical Hebbian rule which
represents in its more generalized form the basis of nearly all neural
learning procedures. The rule implies that the connection between two
neurons is strengthened when both neurons are active at the same time.
 1950: The Karl Lashley defended the thesis that brain information storage
is realized as a distributed system.
Golden age
 1951: For his dissertation Marvin Minsky developed the neurocomputer
Snark.
 1957-1958: Frank Rosenblatt developed the first successful
neurocomputer, Mark I perceptron, which was capable to recognize simple
20 × 20 pixel image
 1959: Frank Rosenblatt described different versions of the perceptron,
formulated and verified his perceptron convergence theorem. He
described a learning rule adjusting the connecting weights.
 1960: Bernard Widrow and Marcian E. Hoff introduced the ADALINE
(ADAptive LInear NEuron), a fast and precise adaptive learning system
being the first widely commercially used neural network (analog telephone
for real-time adaptive echo filtering) and was trained by Widrow-Hoff rule
or delta rule.
 1961: Karl Steinbuch introduced technical realizations of associative
memory.
 1969: Marvin Minsky and Seymour Papert published a precise
mathematical analysis of the perceptron to show that the perceptron model
was not capable of representing many important problems (XOR problem
and linear separability).
Two of the reasons for the "quiet years" of the 1970s were the failure of
single layer perceptrons to be able to/solve such simple problems
(mappings) as the XOR function and the lack of a general method of
training a multilayer net.
 1974: Paul Werbos developed a learning procedure called
backpropagation of error.

2
 Dr. Qadri Hamarsheh

 1982:
o Teuvo Kohonen described the self-organizing feature maps (SOM)–
also known as Kohonen maps. He was looking for the mechanisms
involving self-organization in the brain.
o John Hopfield also invented the so-called Hopfield networks
(Hopfield has developed a number of neural networks based on fixed
weights and adaptive activations).
 1983: Fukushima, Miyake and Ito introduced the neural model of the
Neocognitron which could recognize handwritten characters.
 1986: The backpropagation of error learning procedure as a generalization
of the delta rule was separately developed and widely published by the
Parallel Distributed Processing Group: Non-linearly-separable
problems could be solved by multilayer perceptrons.

Neural Network Applications (Where are neural nets being used?)

 Signal Processing.
o Electronics
 Control.
o Automotive
o Defense
 Pattern Recognition.
 Medicine.
 Speech Production and Recognition.
 Industrial
o Robotics
o Transportation
o Manufacturing
o Oil & Gas
 Business.
o Banking
o Credit Card Activity
o Checking
o Financial
 Entertainment
 Insurance
 Securities
 Telecommunications

Biological neural networks

 Neurons are information processing cells.
 A neuron is nothing more than a switch with information input and output.
The switch will be activated if there are enough stimuli of other neurons
hitting the information input. Then, at the information output, a pulse is sent
to, for example, other neurons.
3
 Dr. Qadri Hamarsheh

Components of biological neuron

Components of a neuron
 The dendrites of a neuron receive the information by special connections,
the synapses (Incoming signals from other neurons or cells are transferred
to a neuron by synapses).
 The signals are electric impulses that are transmitted across a synaptic gap
by means of a chemical process. The action of the chemical transmitter
modifies the incoming signal (typically, by scaling the frequency of the
signals that are received) in a manner similar to the action of the weights
in an artificial neural network.
 Some synapses transfer a strongly stimulating signal, some only weakly
stimulating ones.
 Dendrites branch like trees and receive electrical signals from many
different sources, which are then transferred into the nucleus of the cell.
 In the soma the weighted information is accumulated.
 After the cell nucleus (soma) has received a plenty of activating
(=stimulating) and inhibiting (=diminishing) signals by synapses, the soma
accumulates these signals. If the accumulated signal exceeds a certain
value (called threshold value), the soma of the neuron activates an
electrical pulse which then is transmitted to the neurons connected to the
current one.
 The axon transfers outgoing pulses.

4
 Dr. Qadri Hamarsheh

Outline
1. Simple neuron model
2. Components of artificial neural networks
3. Common activation functions
4. MATLAB representation of neural network.
Single neuron model

Simple neuron model

Components of simple neuron

 Input Vector: The input of technical neurons consists of many components,

therefore it is a vector.
 Scalar output: The output of a neuron is a scalar.
 Synapses change input: In technical neural networks the inputs are
preprocessed, too. They are multiplied by a number (the weight) – they are
weighted. The set of such weights represents the information storage of a
neural network.
 Accumulating the inputs: In biology, the inputs are summarized to a pulse
according to the chemical change – on the technical side this is often
realized by the weighted sum.
 Non-linear characteristic: The input of our technical neurons is also not
proportional to the output.
 Adjustable weights: The weights weighting the inputs are variable,
similar to the chemical processes at the synaptic cleft. This adds a great
dynamic to the network because a large part of the "knowledge" of a
neural network is saved in the weights.
1
 Dr. Qadri Hamarsheh

Simple neuron model:

⃗ with components 𝒙𝒊 , These are multiplied by the appropriate
 input vector 𝒙
weights 𝐰𝐢 and accumulated (called weighted sum) :
∑ 𝐰𝐢 𝒙𝒊
𝒊
 nonlinear mapping 𝐟 defines the scalar output 𝐲:

𝒚 = 𝒇 (∑ 𝐰𝐢 𝒙𝒊 )
𝒊
 The model consists of a set of synapses each of which is characterized by a
weight or strength of its own. An adder, an activation function and a bias.

Components of artificial neural networks

 A neural network is characterized by
1) Its pattern of connections between the neurons (called its
architecture).
2) Its method of determining the weights on the connections (called its
training, or learning, algorithm).
3) Its activation function.
 Technical neural network consists of simple processing units, the
neurons, and directed, weighted connections between those neurons.
 The reactions of the neurons to the input values depend on this activation
state. The activation state indicates the extent of a neuron’s activation and
is often shortly referred to as activation (Neurons get activated if the
network input exceeds their threshold value).
 Threshold value: Let 𝐣 be a neuron. The threshold value 𝛉𝐣 is uniquely
assigned to 𝐣 and marks the position of the maximum gradient value of the
activation function.

2
 Dr. Qadri Hamarsheh

 The bias neuron is a technical trick to consider threshold values as

connection weights: threshold values for neurons can also be realized as
connecting weight of a continuously firing neuron.
 The activation function is also called transfer function.
Common activation functions
1) Identity function
𝐟 (𝐱) = 𝐱 𝐟𝐨𝐫 𝐚𝐥𝐥 𝐱
Matlab implementation (purelin)

Neurons with this transfer function are used in the ADALINE networks
2) binary threshold function (Heaviside function)
The simplest activation function which can only take on two values: If the
input is above a certain threshold, the function changes from one value to
another, but otherwise remain constant. This implies that the function is not
differentiable at the threshold and for the rest the derivative is 0. Due to
this fact, backpropagation learning, for example, is impossible.

Matlab implementation (Hard Limit)

3
 Dr. Qadri Hamarsheh

3) Fermi function or logistic sigmoid function (an S-shaped curve) , This

transfer function takes the input (which may have any value between plus
and minus infinity) and squashes the output into the range 0 to 1, according
to the expression

The Fermi function can be expanded by a temperature parameter T into

the form:

 The log-sigmoid transfer function is commonly used in multilayer

networks that are trained using the backpropagation algorithm, in
part because this function is differentiable.
4) Hyperbolic tangent which maps to (−1, 1), this function is differentiable.

4
 Dr. Qadri Hamarsheh

MATLAB representation of neural network

Single neuron model: Single-Input Neuron

Single-Input Neuron

5
 Dr. Qadri Hamarsheh

 The neuron output is calculated as 𝐚 = 𝐟 (𝐰𝐩 + 𝐛) ,

If, for instance
𝐰 = 𝟑, 𝐩 = 𝟐 and 𝐛 = – 𝟏. 𝟓,
Then
𝐚 = 𝐟(𝟑 ∙ 𝟐 − 𝟏. 𝟓) = 𝐟(𝟒. 𝟓)
 The actual output depends on the particular transfer function that is chosen.
Single neuron model: Multiple-Input Neuron
 The individual inputs 𝐩𝟏 , 𝐩𝟐 , 𝐩𝟑 , … , 𝐩𝐑 are each weighted by
corresponding elements 𝐰𝟏,𝟏 , 𝐰𝟏,𝟐 , 𝐰𝟏,𝟑 , … , 𝐰𝟏,𝐑 of the weight matrix 𝐖

Multiple-Input Neuron
 The net input 𝒏:

 This expression can be written in matrix form:

 The neuron output can be written as

 Weight Indices: The first index indicates the neuron destination. The
second index indicates the source of the signal fed to the neuron. Thus, the
𝐰𝟏,𝟐 represents the connection to the first neuron from the second source.
6
 Dr. Qadri Hamarsheh

Example:

If p1 = 2.5; p2 = 3; w1 = 0.5; w2 = -0.7; b = 0.3. Let’s assume the transfer

function of the neuron is hardlimit
Solution
a = hardlim(n) = hardlim( w1p1 + w2p2 + b)
= hardlim( 0.5×2.5 + (-0.7)×3 + 0.3 )
= hardlim( -0.55) = 0
 Abbreviated Matlab notation

Neuron with 𝐑 Inputs, Abbreviated Notation

7
 Dr. Qadri Hamarsheh

MATLAB representation of neural network

Outline
 Neural network with single-layer of neurons.
 Neural network with multiple-layer of neurons.

Introduction:
Neural Network topologies (Typical Architectures)
 The single-layer and multilayer nets.
 feedforward and recurrent net:
o In a feedforward network each neuron in one layer has only directed
connections to the neurons of the next layer (towards the output layer).
o Recurrent networks contain feedback connections.
Feedforward Neural Network
 Feedforward contains:
o One input layer,
o n hidden processing layers (invisible from the outside, that’s why the
neurons are also referred to as hidden neurons)
o One output layer.
 Every neuron is connected to all neurons of the next layer (these layers are
called completely linked).
Single-Layer Net
 The units can be
distinguished as input units, which
receive signals from the outside
world, and output units, from
which the response of the net can
be read. A one-layer network,
with R input elements and S
neurons, is shown below.
In this network:
 Each element of the input
vector p is connected to each
neuron input through the weight
matrix W.
 In MATLAB, the S-neuron, R-
input, one-layer network also can
be drawn in abbreviated notation

1
 Dr. Qadri Hamarsheh

 Note that the row indices on the elements of matrix W indicate the
destination neuron of the weight, and the column indices indicate which
source is the input for that weight.
 A single-layer network of S logsig neurons having R inputs is shown below
in full detail on the left and with a layer diagram on the right.

2
 Dr. Qadri Hamarsheh

Multiple Feed-Forward Layers of Neurons

 The architecture of a multi-layer feed forward neural network is as follows:

 Multilayer networks are more powerful than single-layer networks. For

instance, a two-layer network having a sigmoid first layer and a linear second
layer can be trained to approximate most functions arbitrarily well. Single
layer networks cannot do this.
 The number of inputs to the network and the number of outputs from the
network are defined by external problem specifications. So if there are four
external variables to be used as inputs, there are four inputs to the network.
Similarly, if there are to be seven outputs from the network, there must be
seven neurons in the output layer.
 The desired characteristics of the output signal also help to select the
transfer function for the output layer. If an output is to be either 1 or -1, then
a symmetrical hard limit transfer function should be used.

Number of network inputs = number of problem inputs

Number of neurons in output layer = number of problem outputs
Output layer transfer function choice at least partly determined
by problem specification of the outputs

 Feedforward networks often have one or more hidden layers of sigmoid

neurons followed by an output layer of linear neurons.
 Multiple layers of neurons with nonlinear transfer functions allow the
network to learn nonlinear relationships between input and output vectors.
 We will call weight matrices connected to inputs, input weights; and we will
call weight matrices coming from layer outputs, layer weights.

3
 Dr. Qadri Hamarsheh

 We will use superscripts to identify the source (second index) and the
destination (first index) for the various weights and other elements of the
network.

 This network can be used as a general function approximator

4
 Dr. Qadri Hamarsheh

Example 1:
Multiple Layers of Neurons (Three-Layer Network, Abbreviated Notation)

 S1, S2, S3: Number of neurons in Layer 1, Layer 2, Layer 3 respectively

 IW 1,1: Input Weight matrix for connection from Input to Layer 1
 LW 2,1: Layer Weight matrix for connection from Layer 1 to Layer 2
 LW 3,2: Layer Weight matrix for connection from Layer 2 to Layer 3
Example 2:
Neural network with 2 layers, 1st layer (hidden layer) consists of 2 neurons with
tangent-sigmoid (tansig) transfer functions; 2nd layer (output layer) consists of 1
neuron with linear (purelin) transfer function.

5
 Dr. Qadri Hamarsheh

In MATLAB® abbreviated notation, the neural network is represented by the

diagram below.

Example 3:

y = a2 = purelin(LW2,1 tansig(IW1,1p + b1) + b2)

= purelin([0.1 -0.2] × tansig([0.3 -0.7 ; -0.2 0.5] × [1 ; 2] + [0.1 ; 0.2]) + 0.3)
= purelin([0.1 -0.2] × tansig([-1 ; 1]) + 0.3)
= purelin(0.0715)
= 0.0715

6
 Dr. Qadri Hamarsheh

Outline: Solving simple pattern recognition problem using ANN

Problem Statement:
 Sorting the fruit according to type using a machine.
 Using sensors: shape, texture and weight.
o The shape sensor will output a 1 if the fruit is approximately round
and a –1 if it is more elliptical.
o The texture sensor will output a 1 if the surface of the fruit is smooth
and a –1 if it is rough.
o The weight sensor will output a 1 if the fruit is more than one pound
and a –1 if it is less than one pound.
 The outputs of the sensors will then be input to a neural network.
The purpose of the network: decide which kind of fruit is on the conveyor, so
that the fruit can be directed to the correct storage bin.
 Two kinds of fruit: apples and oranges.

 Fruit can be represented by a three dimensional vector:

o First element represents shape.
o Second element represents texture.
o Third element represents weight.

Orange would be represented by

Apple would be represented by

 The neural network will receive one three-dimensional input vector for
each fruit on the conveyer and must make a decision as to whether the fruit
is an orange p1 or an apple p2.

1
 Dr. Qadri Hamarsheh

Pattern recognition problem using Perceptron

Single-layer perceptron with a symmetric hard limit transfer function hardlims.

Remark: Two-Input /Single-Neuron Perceptron Case

 Investigate the capabilities of a two-input/single-neuron perceptron (R=2),
which can be easily analyzed graphically:

 What is a decision boundary?

A decision boundary is the region of a problem space in which the output
label of a classifier is ambiguous.
Parameterizing decision boundary
 Let w denote a vector orthogonal to the decision boundary, and b denote a
scalar "offset" term, then we can write the decision boundary as

2
 Dr. Qadri Hamarsheh

 Single-neuron perceptrons can classify input vectors into two categories:

This divides the input space (the range of all possible input vectors) into
two parts, Notice that this decision boundary will always be orthogonal to
the weight matrix, and the position of the boundary can be shifted by
changing b
 For example, if w1,1 = –1, and w1,2 = 1 then
a = hardlims(n) = hardlims( [–1 1] p)+ b)
Therefore, if the inner product of the weight matrix (a single row vector in
this case) with the input vector is greater than or equal to –b the output will
be 1. If the inner product of the weight vector and the input is less than –b,
the output will be -1, the following figure illustrates this for the case where
b = –1

Perceptron Decision Boundary

 The shaded region contains all input vectors for which the output of the
network will be 1. The output will be -1 for all other input vectors.
 The decision boundary between the categories is determined by the
equation
𝐖𝐩 + 𝒃 = 𝟎
 The single-layer perceptron can only be used to recognize patterns that are
linearly separable (can be separated by a linear boundary).
Pattern Recognition Example
 We can use a single-neuron perceptron with three inputs, because there
are only two categories (apple or orange).

 We may want the output of the perceptron to be 1 when an apple is input

and -1 when an orange is input.
 The two prototype vectors are shown in the following figure, from this
figure we can see that the linear boundary that divides these two vectors
symmetrically is the p1 and p3 plane.

3
 Dr. Qadri Hamarsheh

 The p1 and p3 plane, which will be our decision boundary, can be

described by the equation
𝑝2 = 𝟎
Or

Therefore the weight matrix and bias will be

 The weight matrix is orthogonal to the decision boundary and points

toward the region that contains the prototype pattern P2 (apple), for which
we want the perceptron to produce an output of 1.
 The bias is 0 because the decision boundary passes through the origin.
Test the operation of our perceptron pattern classifier
Orange:

Apple:

4
 Dr. Qadri Hamarsheh

 But what happens if we put a not-so-perfect orange into the classifier (noisy
data)? Let’s say that an orange with an elliptical shape is passed through the
sensors. The input vector would then be

The response of the network would be

 In fact, any input vector that is closer to the orange prototype vector than to
the apple prototype vector (in Euclidean distance) will be classified as an
orange (and vice versa).
 The key characteristic of the single-layer perceptron is that it creates linear
decision boundaries to separate categories of input vector.
 What if we have categories that cannot be separated by linear boundaries?
The multilayer networks are able to solve classification problems of
arbitrary complexity.

5
 Dr. Qadri Hamarsheh

Multi-Layer Feedforward Neural Networks using matlab

Part 2
Examples:
Example 1: (fitting data) Consider humps function in MATLAB. It is given by

Solution: Matlab Code:

%Obtain the data:
x = 0:.05:2; y=humps(x);
P=x; T=y;
%Plot the data
plot (P, T, 'x')
grid; xlabel('time (s)'); ylabel('output'); title('humps function')
%DESIGN THE NETWORK
net = newff([0 2], [5,1], {'tansig','purelin'},'traingd');
% the first argument [0 2] defines the range of the input and initializes the network.
% the second argument the structure of the network, there are two layers.
% 5 is the number of the nodes in the first hidden layer,
% 1 is the number of nodes in the output layer,
% Next the activation functions in the layers are defined.
% in the first hidden layer there are 5 tansig functions.
% in the output layer there is 1 linear function.
% ‘traingd’ defines the basic teaching scheme – gradient method
% Define learning parameters
net.trainParam.show = 50; % The result is shown at every 50th iteration (epoch)
net.trainParam.lr = 0.05; % Learning rate used in some gradient schemes
net.trainParam.epochs =1000; % Max number of iterations
net.trainParam.goal = 1e-3; % Error tolerance; stopping criterion
%Train network
net1 = train(net, P, T); % Iterates gradient type of loop
% Resulting network is strored in net1
%Convergenceurve c is shown below.
% Simulate how good a result is achieved: Input is the same input vector P.
% Output is the output of the neural network, which should be compared with output data
a= sim(net1,P);
% Plot result and compare
plot (P, a-T, P,T); grid;
The fit is quite bad, to solve this problem:
 Change the size of the network (bigger. size).
net=newff([0 2], [20,1], {'tansig','purelin'},'traingd');
 Improve the training algorithm performance or even change the algorithm.
1
 Dr. Qadri Hamarsheh

Try Levenberg-Marquardt – trainlm (more efficient training algorithm)

net=newff([0 2], [10,1], {'tansig','purelin'},'trainlm');
It is clear that L-M algorithm is significantly faster and preferable method to
back-propagation.
Try simulating with independent data.
x1=0:0.01:2; P1=x1;y1=humps(x1); T1=y1;
a1= sim (net1, P1);
plot (P1,a1, 'r', P1,T1, 'g', P,T, 'b', P1, T1-a1, 'y');
legend ('a1','T1','T','Error');
Example 2: Consider a surface described by 𝐳 = 𝐜𝐨𝐬 (𝐱) 𝐬𝐢𝐧 (𝐲) defined on a
square −𝟐 ≤ 𝒙 ≤ 𝟐, −𝟐 ≤ 𝒚 ≤ 𝟐, plot the surface z as a function of x and y and
design a neural network, which will fit the data. You should study different
alternatives and test the final result by studying the fitting error.
Solution
%Generate data
x = -2:0.25:2; y = -2:0.25:2;
z = cos(x)'*sin(y);
%Draw the surface
mesh (x, y, z)
xlabel ('x axis'); ylabel ('y axis'); zlabel ('z axis');
title('surface z = cos(x)sin(y)');
gi=input('Strike any key ...');
%Store data in input matrix P and output vector T
P = [x; y]; T = z;
%Create and initialize the network
net=newff ([-2 2; -2 2], [25 17], {'tansig' 'purelin'},'trainlm');
%Apply Levenberg-Marquardt algorithm
%Define parameters
net.trainParam.show = 50;
net.trainParam.lr = 0.05;
net.trainParam.epochs = 300;
net.trainParam.goal = 1e-3;
%Train network
net1 = train (net, P, T);
gi=input('Strike any key ...');
%Plot how the error develops
%Simulate the response of the neural network and draw the surface
a= sim(net1,P);
mesh (x, y, a)
% Error surface
mesh (x, y, a-z)
xlabel ('x axis'); ylabel ('y axis'); zlabel ('Error'); title ('Error surface')
gi=input('Strike any key to continue......');
% Maximum fitting error
Maxfiterror = max (max (z-a)); %Maxfiterror = 0.1116
2
 Dr. Qadri Hamarsheh

Example 3
 Here a perceptron is created with a 1-element input ranging from -10 to 10,
and one neuron.
net = newp ([-10 10],1);
 Here the network is given a batch of inputs P. The error is calculated by
subtracting the output Y from target T. Then the mean absolute error is
calculated.
P = [-10 -5 0 5 10];
T = [0 0 1 1 1];
Y = sim (net, P)
E = T-Y;
perf = mae (E) ;
Example 4
 Here a two-layer feed-forward network is created with a 1-element input
ranging from -10 to 10, four hidden tansig neurons, and one purelin
output neuron.
net = newff ([-10 10],[4 1],{'tansig','purelin'});
 Here the network is given a batch of inputs P. The error is calculated by
subtracting the output A from target T. Then the mean squared error is
calculated.
P = [-10 -5 0 5 10];
T= [0 0 1 1 1];
Y = sim(net,P)
E = T-Y
perf = mse(E)
Example 5:
load house_dataset;
inputs = houseInputs;
targets = houseTargets;
net = newff (inputs, targets, 20);
net = train(net, inputs, targets);
outputs = sim(net, inputs)
%outputs1 = net (inputs);
errors = outputs - targets;
perf = perform(net, outputs, targets)

3
 Dr. Qadri Hamarsheh

Perceptrons and Multi-Layer Feedforward Neural Networks using

matlab
Part 3
Matlab examples:
1) House Price Estimation using feedforward neural networks (fitting data)

 Build a neural network that can estimate the median price of a home
described by thirteen attributes:
1. crime rate per town
2. Proportion of residential land zoned for lots over 25,000 sq. ft.
3. proportion of non-retail business acres per town
4. 1 if tract bounds Charles river, 0 otherwise
5. Nitric oxides concentration
6. Average number of rooms
7. Proportion of owner-occupied units built prior to 1940
8. Weighted distances to five Boston employment centres
9. Index of accessibility to radial highways
10. -tax rate per $10,000
11. Pupil-teacher ratio by town
12. 1000(Bk - 0.63)^2
13. Percent lower status of the population

 Create a neural network which not only estimates the known targets given
known inputs, but can generalize to estimate outputs for inputs that were not
used to design the solution: It is a fitting problem.
 Why Neural Networks?
 Neural networks are very good solution for solving the function fit
problems with non-linear nature.
 The thirteen attributes are the inputs to a neural network, and the
median home price is the target.
 Preparing the Data
 Load the dataset.
[P, T] = house_dataset;

 To view the sizes of inputs P and targets T:

size (P);
size (T);
ans = 13 506
ans = 1 506
1
 Dr. Qadri Hamarsheh

 Fitting a Function with a Neural Network

 Weights initialization, the random seed is set to avoid this randomness.
setdemorandstream (491218382)

 Two-layer feed forward neural networks can fit any input-output

relationship given enough neurons in the hidden layer.
 The input and output have sizes of 0 because the network has not yet
been configured to match our input and target data. This will happen
when the network is trained.
net = fitnet (20);
view (net)

 The samples are automatically divided into training, validation and test
sets.
 The training set is used to teach the network. Training continues as
long as the network continues improving on the validation set.
 The test set provides a completely independent measure of network
accuracy.
[net, tr] = train (net, P, T); % tr is the training record information
nntraintool % to display the training tool window

2
 Dr. Qadri Hamarsheh

 To display the network's performance during training, we can use either:

 "Performance" button in the training tool.
 Call plotperform function:
plotperform (tr)

 Performance is shown for each of the training, validation and test sets.

 Testing the Neural Network using the following measures:

I. The mean squared error of the trained neural network is used to how well
the network will do when applied to data from the real world.
testP = P (:, tr.testInd);
testT = T (:, tr.testInd);
testA = net ( testP);
perf = mse (net, testT, testA);
perf =
19.3315

II. Regression plot.

 The regression plot shows the actual network outputs plotted in terms of the
associated target values. If the network has learned to fit the data well, the
linear fit to this output-target relationship should closely intersect the
bottom-left and top-right corners of the plot. If this is not the case then
further training, or training a network with more hidden neurons, would be
advisable (correlation between actual network outputs and the target
values).
 To display the regression plot, we can use either:
 “Regression” button in the training tool.
 Call plotregression function:
A = net (P);
plotregression (T, A);

3
 Dr. Qadri Hamarsheh

III. Error histogram is used to show how the error sizes are distributed.
Typically most errors are near zero, with very few errors far from that.
 To display the Error histogram, we can use either:
 “Error histogram” button in the training tool.
 Call ploterrhist function:
E = T - A;
ploterrhist (E);

2) Modeling Logical AND Function (linear separable classification )

 Truth table:

 The problem is linearly-separable, try to build a one neuron perceptron

network with following inputs and output:

4
 Dr. Qadri Hamarsheh

Solution:
P = [0 0 1 1; 0 1 0 1]; % training inputs, p = [p1; p2]
T = [0 0 0 1]; % targets
net = newp ([0 1; 0 1], 1);
net = train (net, P, T);
a = sim (net, P);
a=
0001
T=
0001
3) Pattern Classification

 Build a perceptron that can differentiate between two groups of images:

 Use Boolean values 1’s and 0’s to represent the image.

Solution:
image1 = [ 0 1 0 1 0 1 1 0 1 0 1 0]';%group 0
image2 = [1 0 1 0 1 0 0 1 0 1 0 1]';%group 1
image3 = [ 0 1 0 1 0 1 1 1 0 0 1 0]';%group 0
image4 = [1 0 1 0 1 0 0 1 1 1 0 0]';%group 1
%Try testing the trained perceptron on following images:

timg1 = [1 1 0 1 0 1 0 1 0 0 1 0 ]';
timg3 = [1 0 1 0 1 0 0 0 1 1 0 1]';
timg2 = [1 0 1 0 1 1 0 0 0 1 0 1]';
P = [image1 image2 image3 image4];
T = [0 1 0 1];
net = newp (minmax(P), 1);
5
 Dr. Qadri Hamarsheh

net = train (net, P, T);

a = sim (net, P)
% Load the test images and ask the perceptron to classify it
PTest = [timg1 timg3 timg2];
testALL = sim (net, PTest) ;
test1 = sim (net, timg1) % to do similarly for other test images