training-neural-network-to-recognize-handwritten-digits

2017-11-14

Overview

So i wanted to play around with the well documented mnist dataset. Its contains 70 000 images that have been labeled. They also have nice comparisons of how different ML algorithms preformed which is nice for benchmarking.

The dataset is available here
mnist dataset

So i trained a small neural network to recognize the digits using 60 000 to train and 10 000 to validate.
I was able train a small network rather quickly less than 10 minutes with good success rate.

Below is some details but code is on git if you want play with it.
But you will need to download dataset.

Preparing data

So first we need to prepare our data. The data is provided in binary format that very efficient but not always nice to work with.
So wanted to unpack it so can easily load it into dataframe or numpy.

I wanted to generate the 70k images as well so could inspect the results.
Also the result labels were in different file so i join them up with the image data.

So the image label we need to map to number of outputs.
This will help us classify each digit by giving each a output node to train for.

This is easy to do I did with following function

1
2
3
4
5
6
def value_encode(img_num):
img_array=[0]*10
img_array[img_num] = 1
return img_array

Also for fun decided to write all the images out to folder.

Generated mnist images

This will enable us to have a look at some of the incorrectly classified items.

Training network

So each image is 28 by 28 pixels. that gives us 784 inputs.
I decided to do a 3 layers network. First layer having 100 nodes then 50 and last having 25.
Stepping down as we reducing to 10 outputs.

I used the sklearn kit which is easy and quick to setup.

Basically, need these follow lines of code.

1
2
3
4
5
6
7
#define network parameters
mlp = MLPClassifier(hidden_layer_sizes=(100,50,25),verbose=10,activation='relu',learning_rate= 'adaptive',max_iter=20)
#define train
mlp.fit(image_input,label_output)

Quick note on the setup parameters. I set the activation function a “relu” or basically a rectifier function.
I set it to go through the images 20 times learning. As you will see from learning rate below it was still not finished if I left it longer it will have reduced error before saturating.

So training you can see the loss being reduced each iteration.

nueral network training plot

Some results

Next I did predictions against test 10 000 images that wasn’t used for training with the following code.

1
2
3
4
5
6
7
##do few predictions
predictions = mlp.predict(test_image_input)
##results
print(classification_report(test_label_output,predictions))

We can then compare them with the results to see how many we got correct.

Results agains test dataset

Its interesting to notice how many times the 3 and 8 symbols trigger each other incorrectly.
3 and 8 digits that trigger each other

But in most cases the network identified the image correctly. (97%)

This is with only 20 iteratations and sure if i tune parameters would get higher score.
But this is impresive from sklearn but would like to tensor flow a try next.

Grab the code and play

The code is on my github account feel free to grab and play with it