Train Network on Image and Feature Data
This example shows how to train a network that classifies handwritten digits using both image and feature input
data.
Load Training Data
Load the digits images, labels, and clockwise rotation angles.
load DigitsDataTrain
To train a network with multiple inputs using the trainnet function, create a single datastore that contains the
training predictors and responses. To convert numeric arrays to datastores, use arrayDatastore. Then, use
the combine function to combine them into a single datastore.
dsX1Train = arrayDatastore(XTrain,IterationDimension=4);
dsX2Train = arrayDatastore(anglesTrain);
dsTTrain = arrayDatastore(labelsTrain);
dsTrain = combine(dsX1Train,dsX2Train,dsTTrain);
Display 20 random training images.
numObservationsTrain = numel(labelsTrain);
idx = randperm(numObservationsTrain,20);
figure
tiledlayout("flow");
for i = 1:numel(idx)
nexttile
imshow(XTrain(:,:,:,idx(i)))
title("Angle: " + anglesTrain(idx(i)))
end
1
Define Network Architecture
Define the following network.
• For the image input, specify an image input layer with size matching the input data.
• For the feature input, specify a feature input layer with size matching the number of input features.
• For the image input branch, specify a convolution, batch normalization, and ReLU layer block, where the
convolutional layer has 16 5-by-5 filters.
• To convert the output of the batch normalization layer to a feature vector, include a fully connected layer
of size 50.
2
• To concatenate the output of the first fully connected layer with the feature input, flatten the
"SSCB"(spatial, spatial, channel, batch) output of the fully connected layer so that it has format "CB"
using a flatten layer.
• Concatenate the output of the flatten layer with the feature input along the first dimension (the channel
dimension).
• For classification output, include a fully connected layer with output size matching the number of classes,
followed by a softmax layer.
Create an empty neural network.
net = dlnetwork;
Create a layer array containing the main branch of the network and add them to the network.
[h,w,numChannels,numObservations] = size(XTrain);
numFeatures = 1;
classNames = categories(labelsTrain);
numClasses = numel(classNames);
imageInputSize = [h w numChannels];
filterSize = 5;
numFilters = 16;
layers = [
imageInputLayer(imageInputSize,Normalization="none")
convolution2dLayer(filterSize,numFilters)
batchNormalizationLayer
reluLayer
fullyConnectedLayer(50)
flattenLayer
concatenationLayer(1,2,Name="cat")
fullyConnectedLayer(numClasses)
softmaxLayer];
net = addLayers(net,layers);
Add a feature input layer to the network and connect it to the second input of the concatenation layer.
featInput = featureInputLayer(numFeatures,Name="features");
net = addLayers(net,featInput);
net = connectLayers(net,"features","cat/in2");
Visualize the network in a plot.
figure
plot(net)
3
Specify Training Options
Specify the training options. Choosing among the options requires empirical analysis. To explore different
training option configurations by running experiments, you can use the Experiment Manager app.
• Train using the SGDM optimizer.
• Train for 15 epochs.
• Train with a learning rate of 0.01.
• Display the training progress in a plot and monitor the accuracy metric.
• Suppress the verbose output.
options = trainingOptions("sgdm", ...
MaxEpochs=15, ...
InitialLearnRate=0.01, ...
Plots="training-progress", ...
Metrics="accuracy", ...
Verbose=0);
Train Network
Train the neural network using the trainnet function. For classification, use cross-entropy loss. By default,
the trainnet function uses a GPU if one is available. Using a GPU requires a Parallel Computing
4
Toolbox™ license and a supported GPU device. For information on supported devices, see GPU Computing
Requirements. Otherwise, the function uses the CPU. To specify the execution environment, use the
ExecutionEnvironment training option.
net = trainnet(dsTrain,net,"crossentropy",options);
Test Network
Load the test data and create a datastore using the same steps as for the training data.
load DigitsDataTest
dsX1Test = arrayDatastore(XTest,IterationDimension=4);
dsX2Test = arrayDatastore(anglesTest);
dsTTest = arrayDatastore(labelsTest);
dsTest = combine(dsX1Test,dsX2Test,dsTTest);
Test the neural network using the testnet function. For single-label classification, evaluate the accuracy. The
accuracy is the percentage of correct predictions. By default, the testnet function uses a GPU if one is
available. To select the execution environment manually, use the ExecutionEnvironment argument of the
testnet function.
accuracy = testnet(net,dsTest,"accuracy")
accuracy =
98.6200
Visualize the predictions in a confusion chart. Make predictions using the minibatchpredict function and
convert the scores to labels using the scores2label function. By default, the minibatchpredict function
uses a GPU if one is available.
scores = minibatchpredict(net,XTest,anglesTest);
YTest = scores2label(scores,classNames);
figure
confusionchart(labelsTest,YTest)
5
View some of the images with their predictions.
idx = randperm(size(XTest,4),9);
figure
tiledlayout(3,3)
for i = 1:9
nexttile
I = XTest(:,:,:,idx(i));
imshow(I)
label = string(YTest(idx(i)));
title("Predicted Label: " + label)
end
6
Copyright 2020-2024 The MathWorks, Inc.
