Package Bio :: Package NeuralNetwork :: Module StopTraining
[hide private]
[frames] | no frames]

Source Code for Module Bio.NeuralNetwork.StopTraining

 1  """Classes to help deal with stopping training a neural network. 
 2   
 3  One of the key issues with training a neural network is knowning when to 
 4  stop the training of the network. This is tricky since you want to keep 
 5  training until the neural network has 'learned' the data, but want to 
 6  stop before starting to learn the noise in the data. 
 7   
 8  This module contains classes and functions which are different ways to 
 9  know when to stop training. Remember that the neural network classifier 
10  takes a function to call to know when to stop training, so the classes 
11  in this module should be instaniated, and then the stop_training function 
12  of the classes passed to the network. 
13  """ 
14   
15 -class ValidationIncreaseStop(object):
16 """Class to stop training on a network when the validation error increases. 17 18 Normally, during training of a network, the error will always decrease 19 on the set of data used in the training. However, if an independent 20 set of data is used for validation, the error will decrease to a point, 21 and then start to increase. This increase normally occurs due to the 22 fact that the network is starting to learn noise in the training data 23 set. This stopping criterion function will stop when the validation 24 error increases. 25 """
26 - def __init__(self, max_iterations = None, min_iterations = 0, 27 verbose = 0):
28 """Initialize the stopping criterion class. 29 30 Arguments: 31 32 o max_iterations - The maximum number of iterations that 33 should be performed, regardless of error. 34 35 o min_iterations - The minimum number of iterations to perform, 36 to prevent premature stoppage of training. 37 38 o verbose - Whether or not the error should be printed during 39 training. 40 """ 41 self.verbose = verbose 42 self.max_iterations = max_iterations 43 self.min_iterations = min_iterations 44 45 self.last_error = None
46
47 - def stopping_criteria(self, num_iterations, training_error, 48 validation_error):
49 """Define when to stop iterating. 50 """ 51 if num_iterations % 10 == 0: 52 if self.verbose: 53 print "%s; Training Error:%s; Validation Error:%s"\ 54 % (num_iterations, training_error, validation_error) 55 56 if num_iterations > self.min_iterations: 57 if self.last_error is not None: 58 if validation_error > self.last_error: 59 if self.verbose: 60 print "Validation Error increasing -- Stop" 61 return 1 62 63 if self.max_iterations is not None: 64 if num_iterations > self.max_iterations: 65 if self.verbose: 66 print "Reached maximum number of iterations -- Stop" 67 return 1 68 69 self.last_error = validation_error 70 return 0
71