AdaGrad
class AdaGrad(learningRate: Float, initialAccumulatorValue: Float, clipGradient: ClipGradientAction) : Optimizer
Content copied to clipboard
Adagrad optimizer.
Updates variable according next formula:
accum += grad * grad
var -= lr * grad * (1 / sqrt(accum))
Adagrad is an optimizer with parameter-specific learning rates, which are adapted relative to how frequently a parameter gets updated during training. The more updates a parameter receives, the smaller the updates.
It is recommended to leave the parameters of this optimizer at their default values.
See also
Constructors
AdaGrad
Link copied to clipboard
fun AdaGrad(learningRate: Float = 0.1f, initialAccumulatorValue: Float = 0.01f, clipGradient: ClipGradientAction = NoClipGradient())
Content copied to clipboard
Properties
clipGradient
Link copied to clipboard
Strategy of gradient clipping as sub-class of ClipGradientAction.
optimizerName
Link copied to clipboard