Code

The code for the project is here neural network code

The code is written in python but is based on the Matlab code provided in the machine learning course by Andrew Ng https://www.coursera.org/learn/machine-learning

The main body of the neural network code is called

neural_network_galaxy.py

In this part of the code you can change the lambda (regularisation) parameter, the number of nodes in the hidden layers, the and number of iterations required to train the network.

The parameters we need to specify are:

  • input_layer_size: Number of attributes of our data set that we wish to use to make a prediction of the classification of the data. In this case there are 10 (see data section).

  • hidden_layer_size: Number of nodes in each hidden layer.

  • num_labels: Number of output classifications (in out case 1 which will be 1 for QSO and 0 for PLO).

  • lamparam: Lambda value, the regularisation parameter.

  • it_no: Number of iterations for optimisation routine.

The functions called in the main code are in

NN_functions_param.py

  • randInitializeWeights This produces intial random weights by generating a matrix of random values between zero and some value epsilon

  • CostFunction
    This takes the data, the theta values (weights) and returns the cost function, i.e. how well the data is doing at computing the correct catagories (small cost = good) and returns the gradient of the cost function so that the weights can be optimised.

  • trainReg This takes the initial weights and applies them to the cost function algorithm which is called in an optimisation routine, in this case scipy.optimize.fmin_cg. The optimisation routine returns the optimised weights (theta) by iterating through the cost function routine and updating the weights as many times as specified by the user.

  • predict This uses the optimised weights generated by the training set and applies them to the test set to measure many of the items in the test set are catagorised correctly.

Several functions commented out. These are the graient checking algorithm (checkNNGradients) that checks the code is doing what it is meant to. The validaion curve algorithm (validationCurve) that outputs the accuracy of the cross validation and training sets for different lambda values. The learning curve algorithm (learningCurve) that outputs the cost function of the training and cross validation sample when the network is trained on different sized training data sets.