RMSProp
Root mean square optimizer
Unpublished, proposed by Geoffrey Hinton - Neural Networks for Machine Learning
-
Undocumented
Declaration
Swift
public typealias ParamTensor = Tensor<Layer.Parameter, Layer.Device> -
Declaration
Swift
public private(set) var model: Layer { get } -
Learning rate scaling factor
Declaration
Swift
public var learningRate: ParamTensor -
Exponential decay rate for gradient history
Declaration
Swift
public var gamma: ParamTensor -
Normalization scalar added to divisors
Declaration
Swift
public var epsilon: ParamTensor -
Root mean square optimizer
Unpublished, proposed by Geoffrey Hinton - Neural Networks for Machine Learning
Declaration
Swift
public init(model: Layer, learningRate: ParamTensor = 0.001, gamma: ParamTensor = 0.9, epsilon: ParamTensor = 1e-8)Parameters
modelModel to optimize
learningRateLearning rate scaling factor
gammaExponential decay rate for gradient history
epsilonNormalization scalar added to divisors
-
Resets the state of the optimizer
Declaration
Swift
public mutating func reset() -
Declaration
Swift
public mutating func update(along gradients: [ParamTensor])
-
Declaration
Swift
public init(from decoder: Decoder) throws -
Declaration
Swift
public func encode(to encoder: Encoder) throws
View on GitHub
RMSProp Structure Reference