Results

When run on a test sample made from approximately half QSOs and half PLOs, our neural network identified 92% of the QSOs correctly. An object was identified as a QSO if the neural network output gave a value y_NN >= 0.5.

Parameter choices

From the results plotted below, a lambda value of 0.5 was chosen, 50 hidden layer nodes were used and although 1000 iterations did not give much improvement over 500, we used the weights generated by by 1000 iteration run. The code was also run with a 3rd hidden layer for comparison, this actually returned a slightly lower accuracy, higher cost function result. Therefore 2 hidden layers are concidered sufficient for this task.

For 10 data input parameters, lambda = 0.5, 2 hidden layers with 50 nodes each, a positive classification value defined as y_NN > 0.5 and 1000 iterations we recovered a test set accuracy of 92%.

The classifications are shown below. The blue are the true positive results i.e. correctly guessed QSO, the green are the false negative or QSOs that the network failed to identify as such. The red are the correctly identified PLO (true negative) and the black are the false positive or the objects that the network wrongly identified as QSO.

To inspect the locations in parameter space where the neural network fails the plot below shows:

• the true distributions (grey underlying contours),
• the true positive (top left),
• true negative (bottom left),
• false negative (top right), and
• false positive (bottom right)

results as colourful contours. The plots to the LHS show where the NN has guessed correctly and on the right show the incorrect guesses. From the plots on the right, it is clear to see why the NN made the wrong choice for a subset of the test data which doesn’t lie in the same parameter space as the majority of that class (i.e. the contours are misaligned).

The incorrectly identified objects (RHS) are very degenerate with the other class.

Finally we show the efficiency of the algorithm when we change the positive classification limit from y_NN >0.5 to a range of values. The efficiency is defined as the fraction of QSO with y_NN > y_NNmin and PLO with y_NN < y_NNmin. For a cut off of 0.2 the QSO finder is around 95% efficient and the PLO finder is around 80% efficient, that is, PLOs are being identified as QSOs which is to be expected with such a low cut off. However at cut off 0.5, the efficiency of classification of both classes is more than 90%.