Training Tips

If you have tried to configure the neural network and can't seem to get the neural network to converge and classify your data with high accuracy, here are some tips you can try:

• Introduce regularization: The data may be overfitting and so try to introduce regularization at a small rate, then gradually increase until you're satisfied.

• Decrease the learning rate: If you see that the training doesn't converge and oscillates or even diverges, try decreasing the learning rate.

• Increase the learning rate: If you see that training is very slow and the desired output is slowly getting to where you want it to go, try increasing the learning rate.

• Change the activation function: If you are performing regression, using Tanh, Sigmoid or ReLu for the activation function will not give good results. Use a linear activation function instead. The same can be said for classification where linear is not good in that area. Ensure you use Tanh, Sigmoid or ReLu when perform classification.

• Changing the number of hidden layers: If the dataset is rather difficult to train, try increasing the number of hidden layers.

• Changing the number of hidden neurons per hidden layer: A general rule is set the same number of neurons for each layer in the hidden layer, but the general rule is to let the first couple of layers have more neurons than the other layers should decide to vary this.