Momentum

public struct Momentum<Layer> : Optimizer where Layer : LayerType
extension Momentum: Codable where Layer: Codable

Gradient descent optimizer with momentum

  • Undocumented

    Declaration

    Swift

    public typealias ParamTensor = Tensor<Layer.Parameter, Layer.Device>

  • Declaration

    Swift

    public private(set) var model: Layer { get }

  • Learning rate with which to move along the gradient

    Declaration

    Swift

    public var learningRate: ParamTensor

  • Decay rate of momentum that is built up, when subsequent gradient updates move in the same direction

    Declaration

    Swift

    public var momentum: ParamTensor

  • Gradient descent optimizer with momentum

    Declaration

    Swift

    public init(model: Layer, learningRate: ParamTensor, momentum: ParamTensor = 0.8)

    Parameters

    model

    Model to optimize
    learningRate

    Learning rate with which to move along the gradient
    momentum

    Decay rate of momentum that is built up, when subsequent gradient updates move in the same direction

  • Resets the state of the optimizer

    Declaration

    Swift

    public mutating func reset()

  • Declaration

    Swift

    public mutating func update(along gradients: [ParamTensor])

Available where Layer: Codable

  • Declaration

    Swift

    public init(from decoder: Decoder) throws

  • Declaration

    Swift

    public func encode(to encoder: Encoder) throws