How to Train a Model with MNIST dataset
medium.com/@afozbek_/how-to-train-a-model-with-mnist-dataset-d79f8123ba84
January 28, 2019
The MNIST database (Modified National Institute of Standards and Technology
database) is a large database of handwritten digits that is commonly used for training
various image processing systems. The database is also widely used for training and
testing in the field of machine learning. The creators felt that since NIST’s training
dataset was taken from American Census Bureau employees, while the testing dataset
was taken from American high school students, it was not well-suited for machine
learning experiments. Furthermore, the black and white images from NIST were
normalized to fit into a 28x28 pixel bounding box and anti-aliased, which introduced
grayscale levels.
The MNIST database contains 60,000 training images and 10,000 testing images. Half
of the training set and half of the test set were taken from NIST’s training dataset, while
the other half of the training set and the other half of the test set were taken from
NIST’s testing dataset. There have been a number of scientific papers on attempts to
achieve the lowest error rate; one paper, using a hierarchical system of convolutional
neural networks, manages to get an error rate on the MNIST database of 0.23%. The
original creators of the database keep a list of some of the methods tested on it.
Right now we will implement the MNIST data set to Python and try to train a model.
Let’s keep going then.
We will use Google Colab for more interactive results.
1/7
Let’s import required libraries;
import numpy as np
import matplotlib.pyplot as plt
import keras
from keras.models import Sequential
from keras.datasets import mnist
from keras.layers import Dense
from keras.optimizers import Adam
import random
Keras library is a deep learning library which we must use when we will train our
model. Dense is a linear operation in which every input is connected to every output by
a weight. Adam is an optimizer which we will use in our layer’s section.
#import 60000 images from mnist data set
(X_train, y_train), (X_test, y_test) = mnist.load_data()
We will import our training image data 2 different tuples 1 for training images and 1 for
test images.
Lets see how much images does our training data has;
print(X_train.shape)
print(X_test.shape)
print(y_train.shape[0])(60000, 28, 28) (10000, 28, 28) 60000
We will see that our first print statement has a result like (60000, 28, 28). 1st parameter
in the tuple shows us how much image we have 2nd and 3rd parameters are the pixel
values from x to y (28x28). These are important! Because when we try to train our
model, the image must be 28x28 format which means we must not have depth of the
image(greyscale image). If we have a rgb image we must reshape it to the greyscale
format.
So we will implement assert statement for validating the input.
Before we see our images in our data set, we must not forget each digit in the dataset has
lots of handwritten images. So we will show 5 random pictures of each digit.
2/7
Half of the output of the above code(0–9)
We can see each of the digits has how much images from code below.
print(num_of_samples)
plt.figure(figsize=(12, 4))
plt.bar(range(0, num_of_classes), num_of_samples)
plt.title("Distribution of the training dataset")
plt.xlabel("Class Number")
plt.ylabel("Number of images")[5923, 6742, 5958, 6131, 5842, 5421, 5918, 6265, 5851,
5949]Text(0, 0.5, 'Number of images')
Bar graph output
3/7
Then we apply to_categorical() function since we have multiclass results(0–9: 10
output)
y_train = to_categorical(y_train, 10)
y_test = to_categorical(y_test, 10)
Because of the matrix multipication we must change the shape of the array. (AxB *
BxC = AxC) So we will shape the 28x28 array to 1x784.
#Each image has Intensity from 0 to 255
X_train = X_train/255
X_test = X_test/255#we must change the shape of the images to 1d array(28*28)
#for multipication 1*784
num_pixels = 784
X_train = X_train.reshape(X_train.shape[0],
num_pixels)
X_test = X_test.reshape(X_test.shape[0],
num_pixels)
print(X_train.shape)(60000, 784)
Now we will create our hidden layers. Activation function will be Relu rather than
sigmoid since relu gives better performance results.
model = create_model()
print(model.summary())_________________________________________________________________ Layer
(type) Output Shape Param #
=================================================================
dense_1 (Dense) (None, 10) 7850
_________________________________________________________________ dense_2 (Dense)
(None, 30) 330 _________________________________________________________________
dense_3 (Dense) (None, 10) 310
_________________________________________________________________ dense_4 (Dense)
(None, 10) 110
=================================================================
Total params: 8,600 Trainable params: 8,600 Non-trainable params: 0
_________________________________________________________________ None
We can see our layers with how much parameter has the layer. Now we can train and
see the results.
Train on 54000 samples, validate on 6000 samples Epoch 1/10 54000/54000
[==============================] - 1s 26us/step - loss: 0.6100 - acc: 0.8065
- val_loss: 0.3036 - val_acc: 0.9102 Epoch 2/10 54000/54000
[==============================] - 1s 16us/step - loss: 0.3326 - acc: 0.9017
- val_loss: 0.2597 - val_acc: 0.9203 Epoch 3/10 54000/54000
[==============================] - 1s 16us/step - loss: 0.2938 - acc: 0.9127
- val_loss: 0.2672 - val_acc: 0.9257 Epoch 4/10 54000/54000
[==============================] - 1s 15us/step - loss: 0.2794 - acc: 0.9166
- val_loss: 0.2559 - val_acc: 0.9267
...
We will keep our training model in history so we can plot and see some visual content.
Let’s see them.
4/7
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.legend(['loss', 'val_loss'])
plt.title('Loss')
plt.xlabel('epoch')
output
We can see accuracy as well.
plt.plot(history.history['acc'])
plt.plot(history.history['val_acc'])
plt.legend(['acc', 'val_acc'])
plt.title('Accuracy')
plt.xlabel('epoch')
5/7
Output
Okey now we train our model. Let’s test our model .First we will get an image from url.
You can test whatever image you want.
Now we will resize this image
and greyscale it. Then we will
transform this image to our
specific input format(1*784).
import cv2img_array =
np.asarray(img)
resized = cv2.resize(img_array,
(28, 28 ))
gray_scale = cv2.cvtColor(resized,
cv2.COLOR_BGR2GRAY) #(28, 28)
image =
cv2.bitwise_not(gray_scale)plt.imshow(image,
cmap=plt.get_cmap('gray'))
print(image)
You will see the number
represention of the array 0 to
255 and our image with
greyscale format. Now we will
resize it and feed the input. output
image = image / 255
image = image.reshape(1, 784)
prediction = model.predict_classes(image)
print("predicted digit:", str(prediction))predicted digit: [2]
6/7
We can see how easy to implement neural network with Python.
You can access all the code from link here.
References
MNIST explanation from vikipedia
All the concepts I won’t cover you can find useful page here
7/7
