Last post in our series we explained what the MLP is and showed an implementation of a Neural Network using numpy. Thanks to several open sourced libraries developing a Neural Network has become easier than ever. In this post we will walk through implementing a Neural Network in Keras.

Keras is a powerful easy-to-use Python library for developing and evaluating Deep Learning models.

It wraps the efficient numerical computation libraries Theano and TensorFlow and allows you to define and train neural network models in a few short lines of code.

Mnist

Our model built with Keras will train a Neural Network on the Mnist dataset. Mnist, a dataset which typically comes with most Neural Network frameworks, is a set of handwritten digits saved as images. The label of each image is the digit which was handwritten. The task of classifying each image with the corresponding digit used to be quite challenging. One used to have to program each feature in order to differentiate between each digit which resulted in an average model. Thanks to Neural Networks the the features are learned by the network and the accuracy achieved is much higher.

The Code

Imports

These are the basic packages used for importing Mnist and building the Neural Network.

import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout
from keras.optimizers import RMSprop

Parameters

These are example of few parameters used when training the Neural Network.

# how many examples the model should go through before updating it's weights

batch_size = 128

# number of different classes the model can output

num_classes = 10

# number of iterations the network will go through

epochs = 20

Preparing the Data

An important part in training a Neural Network is making sure the data being trained fits the networks input dimensions. Here the Mnist dataset is being reshaped, casted to float32, and normalized. At the end of this block the labels are being converted to categorical label.

# the data, shuffled and split between train and test sets

(x_train, y_train), (x_test, y_test) = mnist.load_data()

x_train = x_train.reshape(60000, 784)
x_test = x_test.reshape(10000, 784)
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255

print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')

# convert class vectors to binary class matrices

y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

Neural Network Architecture

This is where we define the Neural Network. Firstly a Sequential object is created and several layers are then added. Each Dense layer is a fully connected layer in the network.

model = Sequential()
model.add(Dense(512, activation='relu', input_shape=(784,)))
model.add(Dropout(0.2))
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(num_classes, activation='softmax'))

model.summary()

Train & Test

What remains is to feed the data through the network, monitor the networks learning, and test our models performance.

model.compile(loss='categorical_crossentroy', optimizer=RMSprop(), metrics=['accuracy'])

history = model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, verbose=1, validation_data=(x_test, y_test))

score = model.evaluate(x_test, y_test, verbose=0)

print('Test loss:', score[0])
print('Test accuracy:', score[1])

We have successfully programmed a basic fully connected Neural Network. That's awesome!!! Bellow is the full code that way you can simply copy paste and run it on your own.

Full Code

'''Trains a simple deep NN on the MNIST dataset.

Gets to 98.40% test accuracy after 20 epochs

(there is *a lot* of margin for parameter tuning).

2 seconds per epoch on a K520 GPU.

'''

from __future__ import print_function

import keras
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout
from keras.optimizers import RMSprop

batch_size = 128
num_classes = 10
epochs = 20

# the data, shuffled and split between train and test sets

(x_train, y_train), (x_test, y_test) = mnist.load_data()

x_train = x_train.reshape(60000, 784)
x_test = x_test.reshape(10000, 784)
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255

print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')

# convert class vectors to binary class matrices

y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

model = Sequential()
model.add(Dense(512, activation='relu', input_shape=(784,)))
model.add(Dropout(0.2))
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(num_classes, activation='softmax'))

model.summary()

model.compile(loss='categorical_crossentroy', optimizer=RMSprop(), metrics=['accuracy'])

history = model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, verbose=1, validation_data=(x_test, y_test))

score = model.evaluate(x_test, y_test, verbose=0)

print('Test loss:', score[0])
print('Test accuracy:', score[1])

Happy AI blogging!!
Make sure to up vote if you enjoyed.
From neurallearner :)