AdaGrad

class AdaGrad(learningRate: Float, initialAccumulatorValue: Float, clipGradient: ClipGradientAction) : Optimizer

Adagrad optimizer.

Updates variable according next formula:

accum += grad * grad
var -= lr * grad * (1 / sqrt(accum))

Adagrad is an optimizer with parameter-specific learning rates, which are adapted relative to how frequently a parameter gets updated during training. The more updates a parameter receives, the smaller the updates.

It is recommended to leave the parameters of this optimizer at their default values.

Constructors

AdaGrad

fun AdaGrad(learningRate: Float = 0.1f, initialAccumulatorValue: Float = 0.01f, clipGradient: ClipGradientAction = NoClipGradient())

Properties

clipGradient

val clipGradient: ClipGradientAction

Strategy of gradient clipping as subclass of ClipGradientAction.

optimizerName

open override val optimizerName: String

Returns optimizer name.

AdaGrad

See also

Constructors

Properties